Metadata loaders synchronize hierarchical information from source systems into the hierarchies of Adaptive Insights. Your hierarchies should already exist in your source system before you attempt to synchronize to Adaptive Insights.
The Planning Attribute Loader lets you map metadata about attributes from the staging area into Adaptive Insights. Use the attribute loader to import attributes in bulk from an external system and synchronize them, instead of creating attributes one at a time in the attributes admin. Once you create a loader, you can run it manually or schedule it to run as part of an integration task.
Watch the video: 1m 32s
- Enter data source settings
- Configure column mapping
- Create business rules
- Preview the loader output
- Run or schedule the loader
Before You Begin
The Planning Attribute Loader requires staging data from an import of an existing Data Source. Make a note of what data source you want to use.
Required Permissions: Integration Operator, Data Designer, Model
How You Get There
Go to nav menu > Integration > Design Integrations
Enter Data Source Settings
The Data Source Settings let you choose the source table you want to use from staging data, and indicate whether to create new attributes automatically.
Best Practice: Save frequently as you create and edit your loader. Click Save in the Actions panel. To discard any unsaved changes click Close, then click No.
- Select Create New Loader In the Loaders section of the Component Library.
- Select Planning Attribute Loader as the loader type. Enter a name for the loader.
- Select Create.
- Enter the Attribute Loader general properties information:
- Source Table: Select the source table from the dropdown list. The list contains all of the tables available in the staging area you can access.
- Planning Attribute: Select the attribute within Adaptive Insights to load into.
- Create attribute value if not found in Planning: Select to automatically create new attribute values found in the source system, but not found in Modeling > Model Management > Attributes. Deselecting this updates only existing attribute values in your model.
- Publish pending admin publishing changes: If Admin Publishing has pending workflow changes, select to publish changes.
- Log Level: Choose the type of details to capture when running the loader.
- Off: No errors logged.
- Error: Log serious errors only.
- Info: Log basic information, such as when the loader was updated.
- Verbose: Detailed information about all phases and actions. Used primarily for debugging or auditing.
Configure Column Mapping
Column Mapping lets you map columns from the data source to attribute properties available in Modeling > Model Management > Attributes. You can unmap columns to ignore the staging data you don't want to load. You can also unmap columns one at a time following loader runs to troubleshoot which columns generate errors in the logs.
If there are several columns to map, use the Show filter to select:
- All: Shows all Planning columns.
- Required: Shows all required Planning columns, like Parent ID and Attribute.
- Unmapped: Shows all unmapped Planning columns.
- Mapped: Shows all mapped Planning columns.
Use Category to select.
- All: Shows all columns. Attributes do not use categories.
You can use the Search field to find columns. Enter a complete or partial name to search for and click the magnifying glass. Any staging columns matching the search appear in the results.
The Column Mapping tab has these columns:
- Status: A checkmark indicates a mapped column. An exclamation point indicates an unmapped column.
- Planning Column: Displays all of the Planning columns available for mapping on a sheet. The list of Planning columns include a set of mandatory Planning columns, followed by all other Planning columns. Mandatory Planning columns are indicated by bold text with an asterisk and include a value column. Non-mandatory columns include Description.
- Source Id Column: Displays the columns within the staging table chosen from the Data Source Settings.
- Source Display Name Column: Indicates the display names to simplify the mapping process.
- Click Column Mapping.
- Select an unmapped column from the drop-down. Columns available for mapping display in the list. As you map the columns, the status indicator changes from an exclamation point to a check mark .
Map Parents of Attributes
The Parent Id column indicates the id of an attribute value's parent. To load to the highest attribute in a hierarchy, the Parent Id value in source data must be empty. Parent Id must also be empty If you want to create child attribute values underneath the top level.
See Load Attributes From a Spreadsheet Data Source for an example of how attributes and their parents map to planning.
To unmap columns, click Unmap and choose:
- Unmap All: Unmaps all of the the columns.
- Unmap Selected: Unmaps only the columns that are selected.
Unmapping a column changes the status icon to an en exclamation point .
Best Practice: Along with unmapping to troubleshoot errors and ignoring columns you can use unmap to help with change management. When new columns get introduced in a data source, you can unmap the older columns to map the new ones.
Create Business Rules
You can use Business Rules to create SQL expressions that limit the staging data that is available for loading. Only records that meet your filter criteria will load.
The Business Rules tab contains a text area for entering SQL:
To create an SQL filter:
In the Business Rules tab, select SQL Filter. Click Edit.
Enter an SQL expression. You can click an item in the Available Columns list to bring that column into the SQL expression instead of typing it.
Click Apply to check your SQL syntax. Errors in your syntax turn the border around the expression red. Hover your cursor over the SQL editor to see syntax error information.
For detailed SQL syntax help, click 'online help' in the Notes section of the Edit SQL Filter. See the SQL Expression Reference for more.
Correct any errors in your syntax and click Apply. Only staging rows that match the SQL expression will import when you run the loader.
Preview the Loader Output
You can preview the loader output to check the mappings and fix validation errors before running the loader.
When the preview loader output or manual load runs, the source data is validated for errors. Examples of the validations include checks if:
- An attribute value is created without indicating its parent attribute.
- An attribute is assigned an attribute value it can't find.
- The source ID value for the attribute is missing (empty) in the data.
- There are duplicate Source IDs or attribute names in the data. Source IDs must be unique.
- A ParentId is the same as the Source ID in the data. If so, the column is referring to itself as its parent.
- An existing node moved under a new node and the Create New flag is disabled. An existing attribute is moving into a new node when the loader was set to not make new attribute values.
- The data has a cyclic reference in its hierarchy. A ParentId is referring to its own child as its parent.
You can also download the preview output as an XML file to manually verify the attributes from your data source against your requirements before loading them.
Click Preview loader output in the Actions panel.
(Optional) Enter Preview Loader Output settings. The information you can enter varies depending on the data source you chose for this Planning Attribute Loader. Changing the settings here imports new data into Staging tables and columns.
- Click Preview loader output.
The loader runs, including all transformations and business rules. A status popup displays the steps as the loader executes. If attributes are available to load, the loader creates an XML file and sends an email notification with the output as a zipped XML attachment. If no attributes load, the zip will contain a message indicating none loaded.
An additional popup lets you download the preview loader output as a zipped XML file after the loader succeeds. The download popup may take a moment to appear depending on the number of attributes and the size of the file.
This is an example of a downloaded unzipped output XML file:
Run the Loader
After saving a Planning Attribute Loader with the required settings, run the loader manually or run it as a scheduled integration task.
Wait about 15 - 30 minutes between each Planning Attribute Loader run so that Adaptive Insights has time to synchronize with your import. Larger imports take longer to synchronize. Wait time depends on model size and other factors.
To run the loader manually, click Run Manually in the Actions panel.
You can load attributes directly from the source system or you can run the loader using staging values that were already imported. When the loader runs, select which of these behaviors you want. Choose Bypass data import to use the existing staging table for loading. Bypassing data import is not available when using spreadsheets as data sources.
As part of the run process, the loader checks that the data in the staging table has been mapped correctly. Multiple validations are applied during the load. Any errors that occur during the load indicate what needs to be resolved for a successful import.
Include the Planning Attribute Loader in an Integration Task
Any integration task can have one or more loaders. A task can contain other integration tasks, like Dashboards metric loaders and scripted loaders.
Best Practice: Have separate Integration tasks for each loader.
If a task contains multiple loaders, parameters from each loader display in a prompt when the task runs. If there is a common or shared parameter in the loaders within a task, the task prompts for the parameter only once. You can choose to override parameter prompts.
For scheduled runs of the task, default values of the parameters for the loaders get used.