Rebuilding OpenIAM's in-memory authorization graph

The Rebuild Graph operation fully resets and reconstructs OpenIAM's in-memory authorization graph. The authorization graph is the data structure the system uses to answer every entitlement question in real time: "Is this user in this group?", "Does this user have access to this resource?", etc.

Triggering a rebuild causes the Authorization Manager service to:

1. Drop the current graph entirely (all vertices and edges).
2. Clear all associated caches (local + Redis).
3. Re-fetch all authorization data from the relational database.
4. Rebuild the graph from scratch and repopulate the caches.

For end-users / System administrators

There are several situation in which the graph rebuild function should be used, for example:

• After a database migration or direct DB modification.
• As s routine operation / scheduled maintenance. Here, the graph rebuilding is optional since the graph rebuilds automatically on first startup if the graph is empty.

In normal day-to-day operations, the function is not to be used since the graph stays in sync automatically.

How to trigger?

Send an HTTP GET request to the Authorization Manager REST endpoint:

GET /authmanager/rebuildGraph

Example (curl):

curl -X GET http://<your-openiam-host>:9080/authmanager/rebuildGraph

Expected behavior

• The operation is asynchronous — the API returns immediately; the actual rebuild happens in the background.
• The rebuild can take a significant amount of time depending on the size of your data (users, groups, roles, resources, organizations).
• During the rebuild the Authorization Manager instance is unavailable for authorization checks until the process completes.
• If the rebuild fails for any reason, the service will shut itself down (to avoid serving stale/incorrect data). It will need to be restarted — on restart the service checks whether the JanusGraph is empty; if it is, it triggers a rebuild automatically in a background thread.
• Concurrent rebuild requests are safe: a Redis-distributed lock ensures only one rebuild runs at a time across all instances.

Monitoring progress

Check the Authorization Manager service logs for entries like:

[WARN]  Creating graph from current data.  This may take a long time...
[INFO]  Time to get all data from relational database: <N> ms
[INFO]  Done inserting <N> vertices into graph database...
[INFO]  Creation (or fail) of authorization objects in graph database took <N> ms

A successful completion produces the final timing log line. Any error will be logged at ERROR level.

For Developers

REST Entry Point

Controller: AuthManagerRestController File: openiam-esb/src/main/java/org/openiam/esb/rest/AuthManagerRestController.java

GET /authmanager/rebuildGraph

Calls authManagerMQService.refreshCache() — a fire-and-forget async message.

Full call chain

GET /authmanager/rebuildGraph
  │
  ▼
AuthManagerRestController.rebuildGraph()
  │
  ▼
AuthManagerMQServiceImpl.refreshCache()
  │  Sends async RabbitMQ message:
  │    API: AMManagerAPI.RefreshAMManager
  │    Payload: EmptyServiceRequest
  │    Exchange: AM_EXCHANGE  (virtual host: AM_HOST)
  ▼
RabbitMQSenderImpl.sendAndReceive()
  │  Routes to the correct RequestServiceGateway by vhost
  ▼
AMManagerQueueListener  (RabbitMQ consumer on AMManagerQueue)
  │  getEmptyRequestProcessor() → case RefreshAMManager
  ▼
AuthorizationManagerServiceImpl.rebuildGraph()
  │
  ├─ graphOperations.deleteAllIndicies()        // drop all vertices from JanusGraph
  ├─ remoteEntitlementsCache.delete(keys)       // clear Redis remote entitlements cache
  └─ synchronized(localEntitlementsCacheLock)
       ├─ sweep()                               // the main rebuild (see below)
       ├─ localEntitlementsCache.invalidateAll()
       ├─ graphIdCacheSweeper.forceSweep()
       ├─ entitlementsObjectsCacheSweeper.forceSweep()
       └─ edgeIdCacheSweeper.forceSweep()

Graph rebuild detail

The sweep() method steps are given below.

File: auth-manager/src/main/java/org/openiam/authmanager/service/impl/AuthorizationManagerServiceImpl.java

This is where the actual graph is constructed. It runs under a Redis distributed lock (GRAPH_BUILDER_LOCK_NAME, 10-second acquisition timeout).

1. Acquire distributed lock — prevents concurrent rebuilds across multiple service instances.

2. Fetch relational data (see SQL Queries below):

   • dataProvider.getModel(NonCachedEntitlementRequest) — loads all organizations, roles, groups, resources, and their membership relationships.
   • dataProvider.getUsers() — loads all users.

3. Build vertices — buildVertex2OpeniamGraphTuple() creates JanusGraph vertices for each entity type:

   • Users
   • Organizations
   • Roles
   • Groups
   • Resources

4. Persist graph IDs to relational DB — each vertex gets a graph ID that is written back to the RDBMS in batches (batchSize) inside a transaction (via authManagerDAO.updateGraphId()).

5. Build edges — addEdges() creates directed edges in JanusGraph representing all membership and entitlement relationships (user→group, group→role, role→resource, etc.).

6. Refresh caches:

   • graphIdCacheSweeper.forceSweep()
   • entitlementsObjectsCacheSweeper.forceSweep()
   • edgeIdCacheSweeper.forceSweep()

7. On failure — all indices are dropped and SpringApplication.exit() is called with exit code 1. The service shuts down to avoid serving incorrect data.

Error Handling

Startup behavior

Does it always rebuild?

The graph is only rebuilt on startup if JanusGraph contains no vertices.

This means:

• First deploy / fresh environment — graph is empty → full rebuild runs automatically.
• Normal restart — JanusGraph already has data → rebuild is skipped; existing graph is used immediately.
• After a crash that called SpringApplication.exit() — the crash first drops all indices, so on the next restart the graph is empty and rebuilds automatically.

This also implies that if JanusGraph data is manually cleared outside of the application, the next service restart will trigger a full rebuild.

Service: auth-manager (AuthorizationManagerServiceImpl) File: auth-manager/src/main/java/org/openiam/authmanager/service/impl/AuthorizationManagerServiceImpl.java

On startup, @PostConstruct init() runs the following check:

if (isEmptyGraph()) {
    Executors.newSingleThreadExecutor().submit(() -> {
        sweep();   // full graph rebuild in a background thread
    });
} else {
    log.info("Graph not empty - not populating.  Found at least one vertex");
}

SQL queries used during rebuild

All queries are issued by JdbcMembershipDAO and JDBCAccessRightDAO (both in openiam-common-boot-module). The {schema} prefix is a configurable table-name prefix (empty by default).

Entity queries

-- Users
SELECT USER_ID AS ID, GRAPH_ID FROM {schema}USERS

-- Resources
SELECT GRAPH_ID, RESOURCE_ID AS ID, NAME, DESCRIPTION, RESOURCE_TYPE_ID,
       RISK, COORELATED_NAME, IS_PUBLIC, TYPE_ID
FROM {schema}RES

-- Groups
SELECT GRAPH_ID, GRP_ID AS ID, GRP_NAME AS NAME, GROUP_DESC AS DESCRIPTION,
       STATUS, MANAGED_SYS_ID, TYPE_ID
FROM {schema}GRP

-- Roles
SELECT GRAPH_ID, ROLE_ID AS ID, ROLE_NAME AS NAME, DESCRIPTION,
       STATUS, MANAGED_SYS_ID, TYPE_ID
FROM {schema}ROLE

-- Organizations
SELECT GRAPH_ID, COMPANY_ID AS ID, COMPANY_NAME AS NAME, DESCRIPTION, STATUS
FROM {schema}COMPANY

-- Access Rights
SELECT ACCESS_RIGHT_ID AS ID, NAME FROM {schema}ACCESS_RIGHTS

Membership (Edge) query

All membership relationships are loaded in a single query — a UNION ALL of 15 cross-reference tables wrapped in a subquery:

SELECT MEMBER_ENTITY_ID, ENTITY_ID, MEMBERSHIP_ID, TYPE, START_DATE, END_DATE, EDGE_ID
FROM (
    SELECT ... FROM {schema}USER_ROLE            -- user → role
    UNION ALL
    SELECT ... FROM {schema}USER_GRP             -- user → group
    UNION ALL
    SELECT ... FROM {schema}USER_AFFILIATION     -- user → organization
    UNION ALL
    SELECT ... FROM {schema}RESOURCE_USER        -- user → resource
    UNION ALL
    SELECT ... FROM {schema}COMPANY_TO_COMPANY_MEMBERSHIP  -- org → org
    UNION ALL
    SELECT ... FROM {schema}ROLE_ORG_MEMBERSHIP  -- org → role
    UNION ALL
    SELECT ... FROM {schema}GROUP_ORGANIZATION   -- org → group
    UNION ALL
    SELECT ... FROM {schema}RES_ORG_MEMBERSHIP   -- org → resource
    UNION ALL
    SELECT ... FROM {schema}role_to_role_membership  -- role → role
    UNION ALL
    SELECT ... FROM {schema}GRP_ROLE             -- role → group
    UNION ALL
    SELECT ... FROM {schema}RESOURCE_ROLE        -- role → resource
    UNION ALL
    SELECT ... FROM {schema}grp_to_grp_membership    -- group → group
    UNION ALL
    SELECT ... FROM {schema}RESOURCE_GROUP       -- group → resource
    UNION ALL
    SELECT ... FROM {schema}res_to_res_membership    -- resource → resource
    UNION ALL
    SELECT ... FROM {schema}ORG_STRUCTURE        -- user → user (hierarchy)
) OPTIMIZED_SUBQUERY

When a date parameter is provided (e.g., when only fetching recently changed memberships), each sub-select adds a date-range filter on START_DATE / END_DATE.

Membership rights query

Access rights attached to each membership edge are loaded separately — another UNION ALL across 14 rights tables:

SELECT MEMBERSHIP_ID, ACCESS_RIGHT_ID, '{Type}' AS TYPE
FROM {schema}USER_ROLE_MEMBERSHIP_RIGHTS
UNION ALL
SELECT MEMBERSHIP_ID, ACCESS_RIGHT_ID, '{Type}' AS TYPE
FROM {schema}USER_GRP_MEMBERSHIP_RIGHTS
UNION ALL
-- ... (USER_AFFILIATION_RIGHTS, USER_RES_MEMBERSHIP_RIGHTS,
--      ORG_TO_ORG_MEMBERSHIP_RIGHTS, ROLE_ORG_MEMBERSHIP_RIGHTS,
--      GRP_ORG_MEMBERSHIP_RIGHTS, RES_ORG_MEMBERSHIP_RIGHTS,
--      ROLE_ROLE_MEMBERSHIP_RIGHTS, GRP_ROLE_MEMBERSHIP_RIGHTS,
--      RES_ROLE_MEMBERSHIP_RIGHTS, GRP_GRP_MEMBERSHIP_RIGHTS,
--      RES_GRP_MEMBERSHIP_RIGHTS, RES_RES_MEMBERSHIP_RIGHTS)

Summary of tables read

EDGE_ID Lifecycle

EDGE_ID is a column present on every membership xref table (e.g., USER_ROLE, GRP_ROLE, RESOURCE_USER, etc.) and on every membership rights table (e.g., USER_ROLE_MEMBERSHIP_RIGHTS). It stores the ID of the corresponding edge in JanusGraph, linking relational membership records to graph edges.

When addEdges() creates an edge in JanusGraph via Gremlin, the traversal uses .as(selectKey) to label each created edge and then .select(...) to retrieve the resulting Edge objects. The edge.id().toString() value returned by JanusGraph is the EDGE_ID that gets written back to the relational DB.

All writes go through AuthManagerDAOImpl (auth-manager module). Three SQL patterns are used:

-- Membership with an access right attached
UPDATE {schema}{xref_rights_table} SET EDGE_ID = ? WHERE MEMBERSHIP_ID = ? AND ACCESS_RIGHT_ID = ?

-- Membership with no access right
UPDATE {schema}{xref_table} SET EDGE_ID = ? WHERE MEMBERSHIP_ID = ?

-- Nulling out a stale/expired EDGE_ID
UPDATE {schema}{xref_table}        SET EDGE_ID = NULL WHERE EDGE_ID = ?
UPDATE {schema}{xref_rights_table} SET EDGE_ID = NULL WHERE EDGE_ID = ?

The target table is resolved at runtime from an internal map keyed on (parentVertexType, childVertexType) — e.g., (USER, ROLE) → USER_ROLE / USER_ROLE_MEMBERSHIP_RIGHTS.

The EDGE_ID is written in the following cases.

fixDataInconsistencies() runs on a schedule (org.openiam.authorization.manager.gremlin.fix.data.time.ms) in addition to being triggerable via the AMManagerAPI.FixDataInconsistencies endpoint.

After writing to the DB, the EDGE_ID → ACCESS_RIGHT_ID mapping is synced into a Redis cache and local in-process cache via EdgeIdCacheSweeper. This cache is used during authorization checks to resolve edge rights without hitting the DB. The sweeper also runs on a fixed schedule (org.openiam.edge.id.threadsweep).

The complete flow for a single-edge update:

addEdges(SaveGraphEdgeRequest)
  │
  ├─ deleteEdge(oldEdgeId)                        // remove stale edge from JanusGraph
  ├─ createEdgeTraversal(...)                      // create new edge; Gremlin returns edge.id()
  ├─ authManagerDAO.updateEdges(...)               // UPDATE {table} SET EDGE_ID=? WHERE MEMBERSHIP_ID=? AND ACCESS_RIGHT_ID=?
  ├─ authManagerDAO.updateEdgesWithoutRights(...)  // UPDATE {table} SET EDGE_ID=? WHERE MEMBERSHIP_ID=?
  ├─ edgeIdCache.refreshTemporaryCacheEntry(...)   // update local in-process cache
  └─ authManagerAdminMQService.refreshEdgeId(...)  // broadcast to other cluster nodes

GRAPH_ID Lifecycle

GRAPH_ID is a column on every entity table (USERS, GRP, ROLE, COMPANY, RES). It stores the JanusGraph vertex ID for that entity, linking each relational row to its vertex in the graph.

JPA note: The graphId field on all entity classes (UserEntity, GroupEntity, RoleEntity, OrganizationEntity, ResourceEntity) is mapped with insertable = false, updatable = false. JPA never writes this column — all writes go through direct JDBC in AuthManagerDAOImpl.

Write-back SQL looks like the following.

-- Users
UPDATE {schema}USERS   SET GRAPH_ID = ? WHERE USER_ID = ?

-- Groups
UPDATE {schema}GRP     SET GRAPH_ID = ? WHERE GRP_ID = ?

-- Roles (only for roles not excluded from auth)
UPDATE {schema}ROLE    SET GRAPH_ID = ? WHERE ROLE_ID = ? AND EXCLUDE_FROM_AUTH = 'N'

-- Organizations
UPDATE {schema}COMPANY SET GRAPH_ID = ? WHERE COMPANY_ID = ?

-- Resources
UPDATE {schema}RES     SET GRAPH_ID = ? WHERE RESOURCE_ID = ?

The target table and primary-key column are resolved from a map keyed on VertexType (USER, GROUP, ROLE, ORGANIZATION, RESOURCE).

The GRAPH_ID is written in the following cases.

The single-entity path (addVertex) is the normal path when a new user, group, role, org, or resource is provisioned through the application — no full rebuild is needed.

During a full rebuild buildVertex2OpeniamGraphTuple() calls addObjectsToGraph() for each entity type. addObjectsToGraph() batches entities (graphBatchSize), sends them to JanusGraph via Gremlin, reads back the Vertex objects, and returns List<Tuple<openiamId, vertex.id()>>. Those tuples are then persisted back in RDBMS batches.

After writing, GraphIdCacheSweeper.forceSweep() is called. It:

1. Reads all current GRAPH_ID values from every entity table into a Map<VertexType, Map<openiamId, graphId>>.
2. Stores the maps in Redis (one key per VertexType).
3. Refreshes the local in-process Guava cache (AbstractGraphIdProvider.sweep()).
4. Broadcasts refreshGraphIdCache to all cluster nodes via RabbitMQ so every instance updates its local copy.

The local cache has a 10-minute write TTL and also a scheduled periodic sync (org.openiam.graph.id.threadsweep). For new single entities, graphIdProvider.refreshTemporaryCacheEntry() updates the local cache immediately without waiting for the next sweep.

The complete flow for a single new entity:

GraphOperations.addVertex(type, entity, properties)
  │
  ├─ graphIdProvider.contains(type, id)?  →  skip if already exists
  ├─ graphSource.addV(type).property(...).next()  →  vertex.id() is the new GRAPH_ID
  ├─ authManagerDAO.updateGraphId(type, entityId, graphId)
  │     UPDATE {schema}{table} SET GRAPH_ID=? WHERE {pk}=?
  ├─ graphIdProvider.refreshTemporaryCacheEntry(type, id, graphId)  →  local Guava cache
  ├─ entitlementsObjectCache.addCacheEntry(entity)
  └─ authManagerAdminMQService.refreshGraphId(entity)
         → broadcasts to all nodes via RabbitMQ
         → each node: AMCacheQueueListener → graphIdProvider.refreshTemporaryCacheEntry()

GRAPH_ID is treated as a String in the DB in all cases. The type conversion (e.g. Long for JanusGraph, String for Neptune / CosmosDB) is handled by the GraphIdProvider implementation at query time, not at write time.

Key classes

Caches cleared during rebuild

Graph technology

The authorization graph is stored in JanusGraph (accessed via Gremlin traversal API — graphSource.V(), .drop(), etc.). GraphOperations.deleteAllIndicies() iterates and drops vertices in batches of 100 until the graph is empty.

Thread safety

• rebuildGraph() uses a synchronized block on localEntitlementsCacheLock to serialize local cache operations.
• sweep() uses a Redis distributed lock (redissonClient.getLock(GRAPH_BUILDER_LOCK_NAME)) to serialize the graph rebuild across all service instances in a cluster.