Configuring synchronization
To enable automated provisioning or import users from another system, the Synchronization functionality in OpenIAM is essential. Synchronization serves two primary purposes:
- Importing data from a source into OpenIAM.
- Importing data from a source, bringing it into OpenIAM, and applying rules for downstream provisioning and deprovisioning.
At a high level, the synchronization engine is responsible for:
- Retrieving data from a connector or CSV file and importing it into OpenIAM.
- This can occur either on an ad-hoc basis or as a scheduled task.
- Mapping incoming data from the source to OpenIAM objects.
- Passing the data to the provisioning service to update downstream systems.
Configuring synchronization
To configure synchronization, follow these steps:
- Go to webconsole > Provisioning > *Synchronization.
- Several examples are provided to help you get started. If you are new to OpenIAM, we recommend leveraging these examples rather than creating a configuration from scratch. The following steps describe how to create a new configuration.
- Click Create Synchronization. You will see the screen below. Complete the form as described in the table.
Field Name | Description |
---|---|
Name | A descriptive name to identify this configuration. |
Records count in one batch | Default is 1000. This determines how many users/records are processed per batch. Adjust this for performance: a higher number increases throughput, but excessive threads may reduce performance. If your transformation scripts are resource-intensive (e.g., >10 seconds per user), consider reducing this number to avoid exceeding the 30-minute RabbitMQ queue limit. |
Is Active? | Determines whether this synchronization configuration can be executed. If inactive, the sync will not run. |
Detect orphan | This should not be enabled for data from a source system. Orphan detection is used for target systems and is covered in the Orphan Management section. |
Provision to target systems? | Enables downstream provisioning. After configuring synchronization and managed systems, this checkbox must be enabled for downstream provisioning to occur. |
Synchronization Source | Specifies whether data will be imported via a connector or CSV file. Choose accordingly. |
Managed System | Defines which managed system the user should be added to. For automated provisioning, use OpenIAM ; provisioning targets are then defined in the transformation script. |
Synchronization Object | Select the type of object being imported. Typically, choose User . |
Synch Type | Choose between Complete or Incremental . Use Complete during development. For production, use Incremental if the source supports change detection (e.g., using whenChanged in LDAP). |
Synch Frequency | Defines how often synchronization runs. If a value is entered, a scheduled task will be created. Ensure the CRON expression is valid. |
Example: To run every 24 hours, use 0 */24 * * * ? | |
Pre-Processor Script | Groovy script that runs before synchronization starts. |
Post-Processor Script | Groovy script that runs after synchronization completes. |
Validation Rule | Groovy script used to validate incoming data. |
Transformation Implementation | Choose between Policy Map or Transformation Script . Select Transformation Scripts . |
Transformation Rule | Groovy script that maps incoming data to OpenIAM-compatible objects. Most sync logic resides here. |
IDM Repository Field | Unique identifier for users in OpenIAM. Options: EMPLOYEE ID , IDM USER ID , PRIMARY EMAIL ADDRESS , PRINCIPAL NAME , or CUSTOM ATTRIBUTE . |
Source Attribute Name | Unique identifier from the source system. |
Custom Rule for Matching | Use a Groovy script for complex matching logic when a single attribute is insufficient. |
Attribute Names Lookup | Script that defines which source attributes are available in the transformation script. Must be set if using connectors like LDAP/AD. |
File Name | The name of the uploaded CSV file. Use the Choose file button to upload. |
Splitter | CSV delimiter. Examples: , , ` |
Developing synchronization scripts
The synchronization framework supports custom Groovy scripts at multiple points, providing flexibility to meet specific business needs. Refer to this document to learn how to develop your own synchronization scripts.
Synchronization troubleshooting
The table below describes common synchronization error messages found in the OpenIAM audit logs to help with troubleshooting.
Synchronization Error Message | Description |
---|---|
TRANSFORMATION_RETURN_SKIP | Triggered when conditions in the transformation script are not met. |
Unparseable Date: "N" | A date attribute could not be assigned due to an invalid format. |
DUPLICATE_PRINCIPAL | Attempt to create a principal name that already exists. |
INTERNAL_ERROR | General/unspecified error. Check synchronization logs for details. |
VALIDATION_ERROR | Triggered when validation fails in the validation script or due to OpenIAM constraints (e.g., regex mismatch). |
MATCH_ATTR_EMPTY | The matching attribute in the source record is empty. |
Synchronizing objects with non-unique names
There may be cases where you need to synchronize objects that have the same name, but belong to different managed systems. A common example of this scenario is synchronizing groups that share identical names.
If synchronization is configured to match solely by name, the system will not be able to distinguish between the two groups and synchronization will fail. This is because, as of now, the synchronization engine does not support matching based on multiple fields directly.
To address this, you should use linked attributes, such as a combination of name and managed system, where the managed system acts as a unique identifier.
Starting from OpenIAM version 4.2.1.14, a custom matching script is provided to solve this issue:
/sync/group/GroupCustomMatchObjectRule.groovy
.
This script includes detailed comments explaining how it works. Specifically, it enables you to match group objects by their name in combination with their associated managed system, ensuring uniqueness.
To use this script select it from the list of available Transformation scripts while configuring synchronization. The code for it is given below.
// Import necessary libraries and classesimport org.apache.commons.collections.CollectionUtilsimport org.openiam.base.request.BaseSearchServiceRequestimport org.openiam.base.response.list.GroupListResponseimport org.openiam.common.beans.mq.RabbitMQSenderimport org.openiam.idm.searchbeans.multivalue.SynchronizationRequestSearchBeanimport org.openiam.idm.srvc.grp.dto.Groupimport org.openiam.idm.srvc.recon.dto.MatchConfigimport org.openiam.idm.srvc.synch.dto.LineObjectimport org.openiam.match.MatchObjectRuleimport org.openiam.model.GroupPairimport org.openiam.mq.constants.api.OpenIAMAPIimport org.openiam.mq.constants.api.SynchronizationObjectProcessingAPIimport org.openiam.mq.constants.queue.synchronization.SynchronizationBulkGetQueueimport org.springframework.context.ApplicationContext/*** Script to match a group by a combination of name + managedSysId.*/class GroupCustomMatchObjectRule implements MatchObjectRule {ApplicationContext context // Spring application context to fetch beans@OverrideList lookup(final MatchConfig matchConfig, final List list) {// Retrieve required beans from the application contextfinal SynchronizationBulkGetQueue syncBulkGetQueue = context.getBean(SynchronizationBulkGetQueue.class)final RabbitMQSender rabbitMQSender = context.getBean(RabbitMQSender.class)// Cast input list to LineObject listfinal List<LineObject> lineObjectList = listprintln "Found ${lineObjectList.size()} objects in target"final List<GroupPair> pairList = new ArrayList<>() // Resulting matched pairsfinal List<Group> iamGroups = new ArrayList<>() // IAM groups fetched from OpenIAM// Process only if target list is not emptyif (CollectionUtils.isNotEmpty(lineObjectList)) {final Set<String> names = new HashSet<>()// Extract group names from the incoming line objects based on the match source fieldfor (final LineObject lineObject : lineObjectList) {names.add(lineObject.columnMap.get(matchConfig.matchSrcFieldName).getValue())}println("Collected ${names.size()} names from target ")// Fetch matching groups from OpenIAMif (CollectionUtils.isNotEmpty(names)) {for (int i = 0; i < names.size(); i = i + 100) {// Prepare search bean with collected namesSynchronizationRequestSearchBean synchronizationRequestSearchBean = new SynchronizationRequestSearchBean()synchronizationRequestSearchBean.setNames(names)// Wrap the bean in a base search requestBaseSearchServiceRequest searchRequest = new BaseSearchServiceRequest<>(synchronizationRequestSearchBean)searchRequest.setFrom(0)searchRequest.setSize(names.size())// Call OpenIAM service via MQGroupListResponse groupListResponse = rabbitMQSender.sendAndReceive(syncBulkGetQueue,(OpenIAMAPI) SynchronizationObjectProcessingAPI.FIND_GROUPS, searchRequest, GroupListResponse.class)// Filter by managed system IDif (groupListResponse != null && CollectionUtils.isNotEmpty(groupListResponse.getList())) {println("Found groups ${groupListResponse.getList().size()} in OpenIAM ")iamGroups.addAll(groupListResponse.getList().findAll { it.getManagedSysId() == matchConfig.getManagedSysId() })}println("After filter by managed system rested ${iamGroups.size()} groups in OpenIAM ")}// Match each target LineObject with IAM groups by namefor (LineObject lineObject : lineObjectList) {String targetGroupName = lineObject.columnMap.get(matchConfig.getMatchSrcFieldName()).getValue()println("try to match ${targetGroupName}")// Perform case-insensitive name matchGroup group = iamGroups.find({ Group it ->it.getName().equalsIgnoreCase(targetGroupName)})// If a match is found, pair them; otherwise, use empty Group objectif (group) {println("Group was found in IAM")pairList.add(new GroupPair(lineObject, group))} else {println("Group was NOT found in IAM")pairList.add(new GroupPair(lineObject, new Group()))}}} else {println "No names were collected from result."}}// Return final list of matched (or unmatched) group pairsreturn pairList}@OverrideList getMatchRegex() {return null}@OverrideSynchronizationObjectProcessingAPI getApi() {return SynchronizationObjectProcessingAPI.FIND_GROUPS}@OverrideClass getDataResponseClassName() {return GroupListResponse.class}}
This approach allows accurate synchronization of objects with non-unique names across different managed systems.