Configuring synchronization

To enable automated provisioning or import users from another system, the Synchronization functionality in OpenIAM is essential. Synchronization serves two primary purposes:

  • Importing data from a source into OpenIAM.
  • Importing data from a source, bringing it into OpenIAM, and applying rules for downstream provisioning and deprovisioning.

At a high level, the synchronization engine is responsible for:

  • Retrieving data from the connector or CSV file and bringing it into OpenIAM.
    • This can occur either on an ad-hoc basis or as a regularly scheduled task.
  • Mapping incoming data from the source to objects in the OpenIAM system.
  • Passing the incoming data to the provisioning service to update downstream systems.

Configure Synchronization

To configure synchronization, follow these steps:

  1. Go to the webconsole > Provisioning > Synchronization.
  2. Several examples have been provided to help you get started. If you are new to OpenIAM, please leverage these examples instead of creating a new configuration. The following steps describe how to create a new configuration.
  3. Click on Create Synchronization. You will see the screen below. Complete the form based on the table below.

Synchronization configuration

Field NameDescription
NameA descriptive value to identify this configuration.
Records count in one batchThe default value is 1000. This controls how many users/lines will be created to process data coming from the connector or CSV file. This is a performance optimization for processing large datasets. However, creating too many threads can take away resources from other operations, potentially having a negative impact. The synchronization service processes incoming data in batches, aiming to complete synchronization within 30 minutes (with the default value of 1000 records). However, some customers may have extremely heavy logic in transformation scripts, where it takes >10 seconds per user, meaning processing 1000 records takes ~2 hours. Therefore, we recommend reducing the batch number to ensure processing within 30 minutes (the limit in RabbitMQ message in attending in queue).
Is Active?A flag determining if the synchronization configuration can be executed. Making a configuration Inactive disables the task.
Detect orphanOrphan management should not be enabled for processing data from a source system. Orphan management is used to detect records in a target system that are not in the source. This will be covered in detail in the Orphan Management section.
Provision to target systems?This flag enables downstream provisioning to the target system. Once you have configured your synchronization and managed systems, you MUST enable this checkbox to allow for downstream provisioning. This checkbox provides an easy way to stop downstream impact.
Synchronization SourceDetermines if you will import the data using connectors or from a CSV file. Based on this, select either 'CSV file' or the name of the connector.
Managed SystemIndicates which managed system the user should automatically be added to. When implementing automated provisioning, set this value to OpenIAM as you will determine which managed system the user will be provisioned to in the synchronization scripts below.
Synchronization ObjectDefines the type of object that will be imported. Select User in this case.
Synch TypeAllows you to define if this should be an incremental or complete sync. While developing your script, you can start with Complete. In production, it will be used to enable incremental sync if you are working with large datasets and your source system has an attribute similar to whenChanged in LDAP to determine which objects have changed since the last synchronization.
Synch FrequencyDescribes how often the synchronization process should run automatically. If this field has a value, a new batch task will be created with the name 'Synchronization=sync config name'. If the task wasn't created, check your cron expression; it may have the wrong format. The frequency is expressed as a CRON expression. A CRON expression is a string of 6 or 7 fields, separated by a white space, that represents a schedule. In this case, select Cron Job.
An example expression to run a task every 24 hours: 0 /24 * * ?
Pre-Processor ScriptRuns before synchronization starts.
Post-Processor ScriptRuns after synchronization has been completed.
Validation RuleA Groovy script to validate the incoming data from the file.
Transformation ImplementationCan be based on a policy map or a transformation script (also known as a Groovy script). Select Transformation Scripts.
Transformation RuleSelect the Groovy script responsible for mapping data from the CSV file to objects that OpenIAM understands. Much of the work during synchronization is performed by this script. The sections below describe how you can create your own transformation script.
IDM Repository FieldThe field that uniquely identifies a user in OpenIAM. Select from: EMPLOYEE ID, IDM USER ID (internal GUID), PRIMARY EMAIL ADDRESS, PRINCIPAL NAME. If none of these apply, select CUSTOM ATTRIBUTE and enter the attribute name.
Source Attribute NameAttribute name from your source (via connector and CSV) that uniquely identifies a user.
Custom Rule for MatchingIn cases where it's not possible to match on a single field, you can create a custom match rule using a Groovy script, allowing for more complex matching algorithms.
Attribute Names LookupWhen getting data from connectors, this script defines the list of attributes from the source system available to the transformation script. For example, if you're working with LDAP or Active Directory, you'll only be able to map attributes in the transformation script that have first been defined in the Attribute Names Lookup script.
File NameName of the CSV file that has been uploaded. Use the Choose file button to upload the file.
SplitterSeparator in CSV. This value can be ',', '

Develop Synchronization Scripts

The synchronization framework allows for the introduction of custom Groovy scripts at several points. These scripts provide an open framework that can be adapted to meet your business requirements. Refer to this document to learn how to develop your own synchronization scripts.

Synchronization Troubleshooting

The table below provides descriptions for common synchronization error messages captured in the OpenIAM audit logs to assist administrators with troubleshooting.

Synchronization Error MessageDescription
TRANSFORMATION_RETURN_SKIPReturned when specified conditions in the transformation Groovy script fail to be met.
Unparseable Date: "N"Returned when the date attribute in an OpenIAM user object cannot be assigned a value due to an incorrect format.
DUPLICATE_PRINCIPALReturned when the synchronization process attempts to create a principal name that already exists.
INTERNAL_ERRORUnspecified error. Administrators can troubleshoot by examining the synchronization logs.
VALIDATION_ERRORReturned when criteria in the validation script are not met during synchronization, or if system validation for an attribute fails within OpenIAM (such as due to regex syntax, for example).
MATCH_ATTR_EMPTYReturned when the matching attribute in the source record is empty.