Over the years of working with Datastage I have come across some tips that I find can be huge time savers when working with the Standardization Rules Designer (SRD) in Quality Stage is designed to aid in the enhancement of Standardization rule sets. We use Standardization Rule Sets in a Quality Stage to do things like the following:
Assign data to its appropriate metadata fields. Standardize ensures that the data within a specific field is being used for the business purpose defined in the metadata. For example, credit records might have a driver’s license number in the address line 1 field and a customer’s address in the address line 2 fields. To synchronize data with its appropriate metadata field, the driver’s license number can be moved to a separate field for the driver’s licenses.
Decompose free-form fields into single component fields. For example, the customer’s address can be decomposed into House number, Street name, PO Box, Rural Area, and other smaller component fields.
Identify new data fields based on the underlying data. New fields that do not exist on input, such as Gender Flag, Individual/Business Record Indicator can be populated by the application, based on table or file look-ups.
Break up records storing multiple entities. It might be necessary to create a separate record for each person or entity that is represented on a single input record (such as joint accounts). A separate record allows for a more complete linkage of all entities in the input files.
Exclude records that do not meet minimum criteria. Based on defined business rules, the application can be required to exclude or reject records that do not meet basic requirements (for example, records that do not contain a name or address).
We can also use the browser-based interface to add or modify classifications, lookup tables, and rules. We can also import sample data to validate the enhancements to the rule set by adding or modifying a rule by mapping input values from an example record to output columns. This rule splits concatenated values in an input address record by mapping each part of the input value to a different output column.
To conclude this can be a very powerful tool as it provides an intuitive and efficient framework that we can use to create or enhance standardization rule sets.