This page covers the following sections:
- What is a Ruleset?
- Data source settings
- Ruleset operations
- Options with individual rules
- Ruleset Templates
What is a Ruleset?
Each transform applied on the dataset is stored in DataPrep as a rule, in order of their execution. The ordered list of these rules is called a Ruleset. You can access the Ruleset from the Ruleset pane, where you can add, edit, or remove rules that have been applied.
To access the Ruleset
1. Click the ruleset icon in the top-right corner of the DataPrep Studio page to view the Ruleset pane.
2. The Ruleset pane will slide open into view. You can see the data source of your dataset added by default to the top of your ruleset.
3. If you select a rule before the latest (last) one in your Ruleset, the data grid is updated to reflect the state of the sample at that time. You can edit this transform applied in the rule using the Transforms panel.
4. When you are in a previous state of the sample, all subsequent rules will be greyed out in the Ruleset pane.
5. If you apply for a new transform in a previous state, the new rule will be added after the current selection.
6. At this point, all rules after the new rule will be recomputed for transformation and if this causes any errors in the subsequent rules, they will be highlighted in red.
7. For example, if you delete the column " Modified_Time " in the new rule when the column is being used in the following rule, it cannot be executed and will be highlighted in red so you can address the issue.
Data source details
To access the data source details:
1. Click the Ruleset icon to view the Ruleset pane.
2. The Ruleset pane will slide open into view. You can see the data source of your dataset added by default at the top of your ruleset.
3. Click on the
icon in the data source tile to open the
Data source details pane.
4. This will have the necessary details on the source of the input data which includes source type, connection details, timestamps of last import and next import, parsing details, and if the import is scheduled.
5. You can edit the data source , and connection from here, depending on the type of data source.
6. You can pause, delete, or edit your import Schedule .
7. You can click on the expand button next to the View file parsing details option to view and edit file parsing details.
8. This will show parsing details which include file encoding, delimiter, text qualifier, etc.
9. File encoding dropdown lists all different encoding mechanisms from the table below.
IBM00858
| IBM857
| ISO-8859-6
| US-ASCII
| windows-1253
|
UTF-8
| IBM862
| ISO-8859-7
| UTF-16
| windows-1254
|
IBM437
| IBM866
| ISO-8859-8
| UTF-16BE
| windows-1255
|
IBM775
| ISO-8859-1
| ISO-8859-9
| UTF-16LE
| windows-1256
|
IBM850
| ISO-8859-2
| ISO-8859-13
| windows-1250
| windows-1257
|
IBM852
| ISO-8859-3
| KOI8-R
| windows-1251
| x-IBM737
|
IBM855
| ISO-8859-4
| KOI8-U
| windows-1252
| x-IBM874
|
x-UTF-16LE-BOM
| ISO-8859-5
|
|
|
|
10. The Delimiter dropdown lists separator values such as comma (,), tab, semicolon (;), space, and pipe(|).
11. Text qualifier lets you qualify data with text data type only if they are enclosed within single or double quotes.
12. When you select the Data contains header option, you can enter which row number should be set as the header in the Row as header field.
The data from the selected row number replaces the column names of their corresponding column.
Ruleset operations
You can perform ruleset operations on the all the rules or on selective rules as needed. Let's look at each of these options:
- Search
- Save as template
- Import ruleset
- Export ruleset
- Reorder rules
- Ruleset history
- Disable rules
Search
Click the
icon on the top-right corner of the
Ruleset pane to perform a search in your Ruleset. For example, you can search the column name to find all the rules applied in that column.
Save as template
The Save as template option is used to save the Ruleset as a template. Ruleset templates are the rulesets from a dataset saved for reuse. You can save the entire ruleset as a template or select specific rules and save them as a ruleset template.
To save as template,
1.Click the
icon and choose the
Save as template option to save all the rules as a ruleset template. You can also select specific rules and save them as a ruleset template.
2. Enter a
Template Name, Description (optional) and click
Save .
Click here to know more about ruleset templates.
Note: The parsing details are saved only if all the rules in the ruleset are saved as a template.
Import ruleset
Using the Import ruleset option, you can import a saved ruleset template from a file (.ZDRT format) or a workspace to your current dataset. When you import ruleset, rules from the template will be added to the last rule in your current ruleset.
Note: The rules from the imported ruleset template will be added after the last rule in your current dataset.
To import ruleset,
1. Click the
icon and choose the
Import ruleset option
2. If you want to import ruleset template from an existing workspace, select the From workspaces option a ruleset template from an existing workspace
3. If you want to import ruleset template from a file (.ZDRT format), choose the From file option.
4. If you want to replace your current parsing details in the data source configuration panel from the ruleset template you import, select the Override parsing details checkbox.
Export ruleset
You can download the ruleset as a file in a .ZDRT format using this option and import it in a ruleset later. You can also select specific rules and export them as a file.
To export ruleset,
1. Click the
icon and choose the
Export ruleset option.
2. If you want to export the entire ruleset, simply choose the Export ruleset option.
3. If you want to export only specific rules from the ruleset, select the required rules, and select Export ruleset .
4. Enter a File name and Description and click Export .
Reorder ruleset
You can reorder one or more rules in the ruleset using this option. For example, when you select more than one rule and move up, all the rules will be moved above the rule selected first.
To reorder rules,
1. Click the
icon and choose the
Reorder rules option.
2. Select the required rules and move them to top, bottom, up or down as needed. You can also drag and drop rules to reorder.
3. Click Apply changes .
Note: Whenever an error occurs after reordering, you will see a pop-up with an option to proceed or undo.
Ruleset history
Click Ruleset history from the three dots icon to view the changes in the Ruleset's timeline. The history of the Ruleset includes user information such as, who created or modified the rules, the timestamp when the changes were made, and the various transforms added to the Ruleset.
Disable rules
You can disable one or more rules using this option. To disable rules, select the required rules and choose the Disable rules option from the three vertical dots menu. This will disable the rule and re-compute transform on data for the remaining rules. You can Enable the rule again from the three vertical dots menu.
Note: The Disable rules option is displayed only when one or more rules are selected.
Actions on individual rules
You can modify individual rules using the actions associated with them. Click the three dots icon menu and select the required options listed below:
- Edit
- Move
- Remove
- Duplicate
- Disable
- Insert rule before
- Show info
Let's look into each one in detail.
Edit
Click Edit to to edit the transform applied on a rule. You can make changes to the transform in the Transform panel and verify the output in the preview . Click Apply to save your changes. The transform will be recomputed to reflect the changes.
Move
You can move one or more rules to the top, bottom, down or up in a ruleset using the Move option. For example, when you select more than one rule and move up, all the rules will be moved above the rule selected first.
Remove
Click Remove when you want to delete one or more rules from the ruleset. This will remove the rule and recompute transform on data for the rules left.
Tip: You can always undo delete in case you deleted a rule accidentally. The undo icon is situated at the top-left portion just above the data grid.
Duplicate
You can duplicate or reapply the same rule again using the Duplicate option. This can be used when you have to apply the same transform more than once.
Disable
You can use this option to disable a rule in your ruleset. This will disable the rule and re-compute transform on data for the remaining rules. You can Enable the rule again from the three vertical dots menu.
Insert rule before
Use the Insert rule before option to include a previous step before a rule. You can also insert a rule after a rule using the empty placeholder that appears when you click on a rule. Once you apply a transform, the rule of this transform will be stored in the placeholder.
Show info
Click the Show info option for information about a specific rule such as who created or modified the rules. It also includes other details such as, when the rule was created or modified, and the list of users who made the changes.
Note: You can also use the
Ruleset history option for more information on all the rules in the Ruleset.
Ruleset templates
Ruleset templates are the Rulesets from various datasets saved for reuse.
To save ruleset as template,
1. Open your dataset and click the Ruleset icon in the top-right corner to view the Ruleset pane.
2. Click the
icon and choose
Save as Template .
3. Give a Name and Description to the template (e.g. "Marketing data") and click Save .
Customize ruleset template
You can select specific rules and save them as a ruleset template.
To customize ruleset template
1. Select the rules that you want to save as a template
2. Click the
icon and choose the
Save as Template option
3. Provide a Template Name and Description to the template
4. Click Save .
Manage Ruleset templates
To view all the Ruleset templates available in your workspace,
1. Select the Ruleset Templates tab from your Workspace's details page.
2. The ruleset templates available for the workspace will be shown in the order of Last modified templates.
4. You can click on a ruleset template to view the list of rules. It also shows the user who created the template and when it was created.
5. You can also Share template using the share icon in the Ruleset template tab.
6. You can share it with Individual users or with multiple users in Groups along with an invitation mail.
7. You can also share or delete multiple rulesets using the checkboxes.
FAQs
1. When I apply a ruleset template, will it be applied to an already scheduled dataset automatically?
