> For the complete documentation index, see [llms.txt](https://docs.openiam.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.openiam.com/administration-guide/lifecycle/synchronize-and-import-data/synchronization-matching-and-processing.md).

# How a synchronization matches and processes a record

How a synchronization decides which OpenIAM object a source record is, what it creates when nothing matches, and how a run is divided into parts.

## What this is, and when you need it

Every synchronization does the same three things to every source record: it decides which existing OpenIAM object the record is, it hands the record to your transformation, and it saves or discards the result. This page is that shared behaviour, in one place, so that the CSV load, the entitlement feed and the scheduled HR feed do not each explain it again.

Read it before you design a feed, and again the first time a run reports success but changes the wrong thing. Two of the behaviours below produce no failed record, no error and no report row, so they reach production looking like a clean run.

The procedure that creates a configuration is [Set up a synchronization](/administration-guide/lifecycle/synchronize-and-import-data.md). The Groovy contracts — the return codes, the bindings and the base classes — are [Edit a synchronization transformation script](/administration-guide/automation/synchronization-scripts.md).

## Match before transform

Matching answers *which existing OpenIAM object is this record*. Transformation answers *what should it become*. Do not compensate for an ambiguous match with logic later in the transformation.

All of the matching fields live in the **Attribute Mapping & Rules** section of `/webconsole/synchronization`, below **Synchronization Source Details** and **Type & Frequency**. There is no "Synchronization Criteria" section — if a runbook or a support thread names one, it is describing a screen that does not exist.

Four fields decide it:

* **OpenIAM field name** (`MATCH_FIELD_NAME`) — how to look the value up. `EMPLOYEE_ID` and `USERID` are the safe choices; `MANAGED_SYS_PRINCIPAL` searches logins on `MANAGED_SYS_ID`; `PRINCIPAL` searches logins on the default managed system; `ATTRIBUTE` uses `CUSTOM_MATCH_ATTR`; `EMAIL` is mutable and reusable; and `NAME` on a `USER` synchronization is not a name match at all — see [Two silent degradations](#two-silent-degradations).
* **Source Attribute Name** (`MATCH_SRC_FIELD_NAME`) — the source column. OpenIAM reads this column directly when it builds the key for a failed record, so a name that is not actually in the source makes the *failure* path throw as well as the match: you get a run that fails records and then fails to report which ones. Spell it exactly as the header does, case included.
* **OpenIAM Custom Attribute Name** (`CUSTOM_MATCH_ATTR`) — the OpenIAM attribute name, used only when **OpenIAM field name** is `ATTRIBUTE`. The field is only rendered in that case, so if you cannot see it on the form, the match field is something else.
* **Custom rule for matching** (`CUSTOM_MATCH_SCRIPT_ID`) — when one field cannot identify the object safely. The rule must handle zero, one and multiple candidates explicitly, and **stop** on multiple, because the built-in rule does not.

Prove all four outcomes before anything else: a new record creates exactly one object; an existing record updates exactly one; an **empty** match value is rejected and reported; and an **ambiguous** match value — one that resolves to two or more OpenIAM objects — never occurs, because you have proved the value is unique. That last one is not enforced for you.

{% hint style="danger" %}
**A match value that hits more than one OpenIAM object does not fail — the first candidate silently wins.** OpenIAM takes whichever candidate it finds first, in no defined order, and discards the rest. There is no count check, no error, no audit failure and no report row: the run records a successful update of the wrong identity, and next run it may pick the other one.

Rejection-and-reporting applies only to the *empty* case. When the match value is blank, `MATCH_ATTR_EMPTY` is audited as a failed record, written to the run report as `<the whole column map>,F,MATCH_ATTR_EMPTY` and counted in `FAIL_COUNT`.

A second code covers the case where the value was present but could not be used:

| Code                  | What it means                                                                                                                               |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| `MATCH_ATTR_EMPTY`    | The matching attribute in the source record is missing or empty, so the record could not be looked up in OpenIAM.                           |
| `MATCH_LOOKUP_FAILED` | The source record carried a matching attribute, but the lookup in OpenIAM did not complete — for example it timed out or returned an error. |

Records dropped during matching are reported for every source type — CSV, database and connector-based synchronizations alike. Each such record appears in the run report with its code and is counted in `FAIL_COUNT`, so a run that silently processes fewer records than the source contained can be reconciled against the report rather than guessed at.

So uniqueness of the match value on the OpenIAM side is **your** responsibility, and there is no check anywhere in the product that will do it for you. Before the first run against real data, query OpenIAM for duplicates on the value you intend to match — an estate with two identities sharing a principal, an email or an employee ID will import one person's data onto the other, repeatedly, with a clean run report.
{% endhint %}

### The valid match field values

`MATCH_FIELD_NAME` is stored as plain text and resolved at run time, not validated on save, so a typo is accepted by the form and simply never resolves. Seven values are valid, and the drop-down offers exactly those seven — which means a config whose match field is not one of them was not created through the drop-down.

| Value                   | Label                    | What it compares                                                                                                                        |
| ----------------------- | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `USERID`                | User ID                  | The OpenIAM user ID. Safe, and available only when the source already stores it.                                                        |
| `EMPLOYEE_ID`           | Employee ID              | `employeeId`, case-insensitively. The usual right answer for an HR feed.                                                                |
| `PRINCIPAL`             | Principal                | The primary principal on the **default** managed system (`openiam.default_managed_sys`, shipped `0`).                                   |
| `MANAGED_SYS_PRINCIPAL` | Managed System Principal | The primary principal on the config's `MANAGED_SYS_ID`. The right choice for a directory inventory.                                     |
| `EMAIL`                 | Email                    | Any of the user's email addresses, case-insensitively. Mutable and reusable — a leaver's address handed to a joiner matches the leaver. |
| `ATTRIBUTE`             | Custom Attribute         | The user attribute named by `CUSTOM_MATCH_ATTR`, compared against its value set.                                                        |
| `NAME`                  | Name                     | Meaningful for `ROLE`, `GROUP`, `ORG` and `RESOURCE`. **Not a name match on a `USER` synchronization** — see below.                     |

### Two silent degradations

Neither of these produces a failed record, so neither appears in the run's failure count. They are the two ways a matching mistake reaches production looking like success.

{% hint style="danger" %}
**`MATCH_FIELD_NAME = NAME` on a `USER` synchronization is not a name match.** OpenIAM does not compare names at all in that case. It logs `can't filter user by name, will be used principal. Check you configuration!` at **warn** and falls back to matching on the principal on the default managed system. The value in **Source Attribute Name** is therefore compared against the login on the default managed system. A `USER` config saved with `NAME` behaves as `PRINCIPAL`, and the only trace is one warn line per run. If you meant to match on a person's name, you cannot — pick an immutable identifier instead. The shipped `CSV ROLE Example` uses `NAME` legitimately, because a role *is* matched by name.
{% endhint %}

The second is the ambiguous match in the callout above: two candidates, no error, the first one wins. The only defence against both is a match field that is unique by construction — `EMPLOYEE_ID` or `USERID` — and a pre-import duplicate check on the OpenIAM side.

### Creation on no match is gated three ways

When nothing matches, OpenIAM does not stop; it hands the transformation a brand-new empty object with a null ID — which is exactly what `isNewUser` tests in your script. Whether that object is ever persisted is decided by three independent gates. A run that reads every record and creates nothing has tripped one of them, and knowing which saves a support cycle:

1. **Detect orphans** (`IS_ORPHAN_DETECTION`). When it is on, an unmatched row is written to the orphan index, audited `SYNCHRONIZATION_ORPHAN`, and the record ends **before** the save is attempted. Nothing is created. See [Orphan detection changes what a run does](#orphan-detection-changes-what-a-run-does).
2. **The transformation's return code.** `SKIP` (`-1`), `SKIP_OK` (`11`) and `DELETE` (`1`) all end the record without creating anything, and `DELETE` against an object with no ID is a silent no-op. Of the three, only `SKIP` is counted as a failure — so a transformation returning `SKIP_OK` for unmatched rows creates nothing and reports a perfect run.
3. **Provision to target systems?** unticked (`IS_PROVISION = 'N'`). OpenIAM then refuses a new object that carries no login, with `NOT_FOUND` and the verbatim message `Logins collection must not be empty when provisioning is not enabled.` This is the field label's warning made concrete: with provisioning off, the transformation must build `pUser.principalList` itself or every new user fails.

### The match value also keys your audit and report evidence

The match fields are not used only for correlation. OpenIAM records **Source Attribute Name** as the row's key attribute before it reads the value, and that key is what names the row everywhere afterwards:

* The record's name in the audit log, on every `SYNCHRONIZATION_RECORD` entry.
* The row identifier in the per-run CSV report, and the report's header column.
* The key on an error record.
* The orphan's identity, when the object has no login on the config's managed system.

So the choice of **Source Attribute Name** decides whether your evidence is usable. A column whose values are blank or repeated produces an audit trail and a report in which rows cannot be told apart — which is the same problem as the ambiguous match, one layer further out.

### One synchronization processes the whole record

**Scope is the object type, not the attribute.** `MATCH_FIELD_NAME` and `MATCH_SRC_FIELD_NAME` define *correlation*, not scope. The transformation receives every source field the connector returned or the CSV header declared, as `columnMap`, and can map all of them in one run — **one configuration processes every column in the file.** This is the most frequent misunderstanding on the whole feature, and it is worth stating outright before anyone builds a config per attribute.

For a CSV with `worker_id`, `name`, `email`, `application` and `entitlement`, create **one** synchronization for the object type and source boundary — not five, one per column. Match `worker_id` to the approved immutable identifier, then map the name and email and create the membership in the transformation. Split configs when the object type, source authority, schedule, failure ownership or safe-replay boundary differs; never because the file has several columns.

If the file carries one row per entitlement assignment, the same person appears several times. Prove each row matches the same existing person and adds only its named entitlement. Then decide separately how a complete run *removes* assignments that are no longer present — an add-only transformation establishes no authority over removal. Which shape of feed to build is [Consolidated feed or one row per assignment](/administration-guide/lifecycle/synchronize-and-import-data/import-roles-groups-and-memberships.md#consolidated-feed-or-one-row-per-assignment).

{% hint style="info" %}
**OpenIAM handles one object type the Webconsole cannot select.** **Synchronize Object** offers seven values — `USER`, `ROLE`, `ORG`, `GROUP`, `RESOURCE`, `MANAGED_SYSTEM` and `LOG` — and those seven are the drop-down. A `POLICY` synchronization is handled internally but no control on the form can produce that value, so treat `POLICY` as not configurable from the Webconsole and **not confirmable from it either**: if you need one, raise it with support rather than editing `PROCESS_RULE` by hand.
{% endhint %}

## Create managed-system identities deliberately

Matching a source record to an existing user answers *which person this is*. It does **not** create a `LOGIN` row on the source or on any other managed system. If an imported user updates successfully and **User Identities** still shows only OpenIAM, the transformation is where to look — not a business rule.

Add a `Login` only when the source genuinely proves that account exists, or when the flow is deliberately pre-creating it. Three ownership patterns, and picking one is the whole decision:

| Pattern                          | Who creates the managed-system identity                                                   | When to use it                                                |
| -------------------------------- | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------- |
| **Authoritative account import** | The transformation, because the source *is* the managed system's own inventory            | Reading a directory to record what it already has             |
| **OpenIAM-driven provisioning**  | Provisioning, when a role, request or rule grants the resource and the connector succeeds | The normal case for a managed system OpenIAM manages          |
| **Correlation only**             | Nobody; reconciliation matches the existing account                                       | Governing accounts OpenIAM did not create and will not create |

Test all six cases: new user, existing user without the identity, existing correlated identity, renamed login, duplicate login, and removal. Check both ends every time — the identity on **User Identities** and the real account on the managed system. Either one alone is incomplete, and the two disagreeing is the finding.

## Orphan detection changes what a run does

`IS_ORPHAN_DETECTION = 'Y'` is not a report flag. Orphan handling runs **after** the transformation and **before** the save, and when the flag is on and the match found nothing, OpenIAM:

1. Writes an orphan record — identity, first name, last name, employee ID, email or the literal `"No email"`, managed system ID, create date — to the `orphan` index.
2. Audits `SYNCHRONIZATION_ORPHAN` with `OBJECT_TYPE` set to `PROCESS_RULE`, description `Orphan detected`, target attribute `ORPHAN = true`, and the whole source row as the request body. That request body is why the audit entry is worth reading: it is the only place the discarded row survives.
3. Ends the record **before the save is attempted at all**.

So with orphan detection on, unmatched records create nothing. That is exactly right for an account inventory and exactly wrong for a joiner feed. If the match value itself is missing, the audit entry **fails** with *"Can't define orphan's principal, value not presented in match field `<MATCH_SRC_FIELD_NAME>`"*.

Orphans are then reviewed at `/webconsole/orphan-management`, and the actions are audited `LINK_ORPHAN`, `CREATE_NEW_USER_FROM_ORPHAN`, `UNLINK_IDENTITY_TO_ORPHAN` and `REMOVE_ORPHAN`. **`REMOVE_ORPHAN` deletes the real account on the managed system** — it is not a list cleanup, and there is no undo. Confirm on the managed system itself after the first one you remove.

## Why aggregating inside a transformation script cannot be made correct

The tempting shortcut is to keep a map in the transformation script, accumulate each person's rows into it and save once. It cannot be made correct, for five independent reasons:

1. **Rows are split into parts.** A run is divided into parts of **Records per Batch** records, `1000` by default, and each part is dispatched as its own RabbitMQ message. Two rows for the same person can land in different parts.
2. **One script instance per part.** OpenIAM builds the transformation script once per part, so in-script state cannot span parts, and parts run concurrently on the synchronization thread queue.
3. **Matching itself is sub-batched.** Matching searches in batches of ten, so two rows naming the same person are usually resolved into two separate user objects before your script ever sees them.
4. **Saves are queued, not immediate.** Saves are buffered and flushed 100 at a time every 500 ms. "Save once at the end" is not a point in time you control.
5. **The concurrency guard does not cover new users.** Two saves for the same **existing** user id are serialised — the second is rejected with `ANOTHER_SAVE_IN_PROGRESS` and the message `User with id %s is saving in another thread. Current request will be sent back to the queue to be retried.`, then requeued and retried. For a **new** user there is no id to lock on, so two concurrent creates of the same person are not serialised and both proceed. That is the concrete risk of script-side aggregation on an unconsolidated feed: duplicate people, produced by a run that reports success. The check is a count: rows in, people out.

{% hint style="warning" %}
**The preprocessor script cannot reshape rows either.** A preprocessor script returns a single `int` for the whole run and is called once, before any row is read. It is a valid/invalid gate on the run, not a row transformer. There is no hook between reading the source and matching that can merge rows.
{% endhint %}

## Related

* [Set up a synchronization](/administration-guide/lifecycle/synchronize-and-import-data.md) — the configuration procedure these rules apply to.
* [Edit a synchronization transformation script](/administration-guide/automation/synchronization-scripts.md) — the transformation contract and the complete return-code set.
* [Import roles, groups, organizations and entitlement memberships](/administration-guide/lifecycle/synchronize-and-import-data/import-roles-groups-and-memberships.md) — feed shape and the ordering rule that follows from the concurrency rules above.
* [Enumerated values](/administration-guide/reference/enums.md) — the raw constants behind the match-field and object-type drop-downs.

*Applies to OpenIAM 2026.8.1. Last reviewed 7 August 2026.*


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.openiam.com/administration-guide/lifecycle/synchronize-and-import-data/synchronization-matching-and-processing.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.