Configuring synchronization
To enable automated provisioning or import users from another system, the Synchronization functionality in OpenIAM is essential. Synchronization serves two primary purposes:
- Importing data from a source into OpenIAM.
- Importing data from a source, bringing it into OpenIAM, and applying rules for downstream provisioning and deprovisioning.
At a high level, the synchronization engine is responsible for:
- Retrieving data from the connector or CSV file and bringing it into OpenIAM.
- This can occur either on an ad-hoc basis or as a regularly scheduled task.
- Mapping incoming data from the source to objects in the OpenIAM system.
- Passing the incoming data to the provisioning service to update downstream systems.
Configure Synchronization
To configure synchronization, follow these steps:
- Go to the webconsole > Provisioning > Synchronization.
- Several examples have been provided to help you get started. If you are new to OpenIAM, please leverage these examples instead of creating a new configuration. The following steps describe how to create a new configuration.
- Click on Create Synchronization. You will see the screen below. Complete the form based on the table below.
| Field Name | Description |
|---|---|
| Name | A descriptive value to identify this configuration. |
| Records count in one batch | The default value is 1000. This controls how many users/lines will be created to process data coming from the connector or CSV file. This is a performance optimization for processing large datasets. However, creating too many threads can take away resources from other operations, potentially having a negative impact. The synchronization service processes incoming data in batches, aiming to complete synchronization within 30 minutes (with the default value of 1000 records). However, some customers may have extremely heavy logic in transformation scripts, where it takes >10 seconds per user, meaning processing 1000 records takes ~2 hours. Therefore, we recommend reducing the batch number to ensure processing within 30 minutes (the limit in RabbitMQ message in attending in queue). |
| Is Active? | A flag determining if the synchronization configuration can be executed. Making a configuration Inactive disables the task. |
| Detect orphan | Orphan management should not be enabled for processing data from a source system. Orphan management is used to detect records in a target system that are not in the source. This will be covered in detail in the Orphan Management section. |
| Provision to target systems? | This flag enables downstream provisioning to the target system. Once you have configured your synchronization and managed systems, you MUST enable this checkbox to allow for downstream provisioning. This checkbox provides an easy way to stop downstream impact. |
| Synchronization Source | Determines if you will import the data using connectors or from a CSV file. Based on this, select either 'CSV file' or the name of the connector. |
| Managed System | Indicates which managed system the user should automatically be added to. When implementing automated provisioning, set this value to OpenIAM as you will determine which managed system the user will be provisioned to in the synchronization scripts below. |
| Synchronization Object | Defines the type of object that will be imported. Select User in this case. |
| Synch Type | Allows you to define if this should be an incremental or complete sync. While developing your script, you can start with Complete. In production, it will be used to enable incremental sync if you are working with large datasets and your source system has an attribute similar to whenChanged in LDAP to determine which objects have changed since the last synchronization. |
| Synch Frequency | Describes how often the synchronization process should run automatically. If this field has a value, a new batch task will be created with the name 'Synchronization=sync config name'. If the task wasn't created, check your cron expression; it may have the wrong format. The frequency is expressed as a CRON expression. A CRON expression is a string of 6 or 7 fields, separated by a white space, that represents a schedule. In this case, select Cron Job. |
| An example expression to run a task every 24 hours: 0 /24 * * ? | |
| Pre-Processor Script | Runs before synchronization starts. |
| Post-Processor Script | Runs after synchronization has been completed. |
| Validation Rule | A Groovy script to validate the incoming data from the file. |
| Transformation Implementation | Can be based on a policy map or a transformation script (also known as a Groovy script). Select Transformation Scripts. |
| Transformation Rule | Select the Groovy script responsible for mapping data from the CSV file to objects that OpenIAM understands. Much of the work during synchronization is performed by this script. The sections below describe how you can create your own transformation script. |
| IDM Repository Field | The field that uniquely identifies a user in OpenIAM. Select from: EMPLOYEE ID, IDM USER ID (internal GUID), PRIMARY EMAIL ADDRESS, PRINCIPAL NAME. If none of these apply, select CUSTOM ATTRIBUTE and enter the attribute name. |
| Source Attribute Name | Attribute name from your source (via connector and CSV) that uniquely identifies a user. |
| Custom Rule for Matching | In cases where it's not possible to match on a single field, you can create a custom match rule using a Groovy script, allowing for more complex matching algorithms. |
| Attribute Names Lookup | When getting data from connectors, this script defines the list of attributes from the source system available to the transformation script. For example, if you're working with LDAP or Active Directory, you'll only be able to map attributes in the transformation script that have first been defined in the Attribute Names Lookup script. |
| File Name | Name of the CSV file that has been uploaded. Use the Choose file button to upload the file. |
| Splitter | Separator in CSV. This value can be ',', ' |
Develop Synchronization Scripts
The synchronization framework allows for the introduction of custom Groovy scripts at several points. These scripts provide an open framework that can be adapted to meet your business requirements. Refer to this document to learn how to develop your own synchronization scripts.
Synchronization Troubleshooting
The table below provides descriptions for common synchronization error messages captured in the OpenIAM audit logs to assist administrators with troubleshooting.
| Synchronization Error Message | Description |
|---|---|
| TRANSFORMATION_RETURN_SKIP | Returned when specified conditions in the transformation Groovy script fail to be met. |
| Unparseable Date: "N" | Returned when the date attribute in an OpenIAM user object cannot be assigned a value due to an incorrect format. |
| DUPLICATE_PRINCIPAL | Returned when the synchronization process attempts to create a principal name that already exists. |
| INTERNAL_ERROR | Unspecified error. Administrators can troubleshoot by examining the synchronization logs. |
| VALIDATION_ERROR | Returned when criteria in the validation script are not met during synchronization, or if system validation for an attribute fails within OpenIAM (such as due to regex syntax, for example). |
| MATCH_ATTR_EMPTY | Returned when the matching attribute in the source record is empty. |