Powered by Algolia

    Backend exception error when running authentication manager

    Symptoms

    Authentication manager may return the following errors when checking its log.

    Can't get entitlements graph for User: ID_of_User java.util.concurrent.CompletionException: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: Could not execute operation due to backend exception

    OR

    Cannot run application - shutting down java.util.concurrent.CompletionException: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: Could not execute operation due to backend exception

    OR

    tinkerpop.gremlin.driver.exception.ResponseException: The traversal source [g] for alias [g] is not configured on the server.

    Solution

    To troubleshoot and put the authentication manager up again, follow the steps below.

    1. First, validate the status of Cassandra nodes by running the following command. 
    nodetool -h ::FFFF:127.0.0.1 status

    If any node is down start Cassandra service by running

    systemctl start cassandra

    Then restart JanusGraph and authentication manager and repeat nodetool command.

    Note that the order of services to be up has to be Cassandra > JanusGraph > Authentication manager, since Cassandra is a database for JanusGraph service and Authentication manager is using JanusGraph API. Also, they must be restarted in reverse order.

    In case Cassandra will not start, check Cassandra's logs /var/log/cassandra/ and its status by running

    systemctl status cassandra

    To move on you want a cluster of Cassandra to be running fine. At this point you can restart JanusGraph service and then openiam-auth may already work fine, if not follow the recommendations in step 2.

    1. If all nodes are ok, go into /usr/local/openiam/janusgrpaph/bin and run the following command
    gremlin.sh

    Once started, run the following commands.

    :remote connect tinkerpop.server conf/remote.yaml
    :> g.V().count()

    If count gives an error, then probably Cassandra experienced some issues and you need to clear data in Cassandra DB and repopulate it. To do it

    • Stop OpenIAM and all JanusGraph nodes.
    • Run the following on all nodes.
    nodetool -u cassandra -pw passwd00 cleanup janusgraph
    • Make sure /var/lib/cassandra/data/janusgraph/ is empty, if not empty stop Cassandra service all nodes and remove this directory manually, then start Cassandra and start JanusGraph.
    • Clear redis cache. Perform clear redis cache operation, when OpenIAM services stack are all stopped.
    • Login into master sentinel node by running the following command
    redis-cli -h[ip] -> auth password->flushdb -> flushall
    • Start OpenIAM and check authentication manager logs.
    • Run the following command once openiam-esb and authentication manager are started to force graph rebuilding.
    curl http://localhost:9080/openiam-esb/authmanager/rebuildGraph
    • Wait until the console is released of this command and try to login.
    1. If none of the steps above didn't help, create a support ticket.
    Previous
    Error when sending activation link
    Next
    Database reset