[Helm v16 RC - ALS] Party lookup fails intermittently with ALS caching enabled

[oderayi]

Summary:
There are intermittent party look up failures in Helm charts v16.0.0 RC Helm tests when ALS caching is enabled.
These failures happen mostly on a fresh install of the backend and mojaloop during the first run of the Helm tests. Subsequent runs may pass or fail too.

Examples of failing TTK tests:

1. P2P_MONEY_TRANSFER - P2P_UNHAPPY_PATH WITH INVALID SUBID AT PAYEEFSP

• Get party information

Error:

{
  "errorInformation": {
    "errorCode": "3200",
    "errorDescription": "Generic ID not found - Requester FSP not found"
  }
}

2. RESERVE NOTIFICATION POSITIVE - TESTFSP1 TESTFSP2

Error:

{
  "scenario1": {
    "result": {
      "message": "Got an error response resolving party: {\n  errorInformation: { errorCode: '3204', errorDescription: 'Party not found' }\n}",
      "transferState": {
        "from": {
          "displayName": "PayerFirst PayerLast",
          "idType": "MSISDN",
          "idValue": "17039811904"
        },
        "to": {
          "idType": "MSISDN",
          "idValue": "17039811903"
        },
        ...
        },
        "getPartiesResponse": {
          "body": {
            "errorInformation": {
              "errorCode": "3204",
              "errorDescription": "Party not found"
            }
          },
          "headers": {
            "content-type": "application/vnd.interoperability.parties+json;version=1.1",
            "date": "Fri, 16 Feb 2024 09:38:58 GMT",
            "fspiop-source": "switch",
            "connection": "close",
            "traceid": "cf0d77ba-e7c1-45d6-a4e3-b0b37e6bc6ca",
            "fspiop-destination": "testfsp2",
            "fspiop-http-method": "PUT",
            "fspiop-uri": "/parties/MSISDN/17039811903/error",
            "user-agent": "axios/1.6.2",
            "content-length": 78,
            "accept-encoding": "gzip, compress, deflate, br",
            "host": "moja2-sim-testfsp2-scheme-adapter:4000"
          }
        },
        "lastError": {
          "httpStatusCode": 500,
          "mojaloopError": {
            "errorInformation": {
              "errorCode": "3204",
              "errorDescription": "Party not found"
            }
          }
        }
      },
      "statusCode": "3204"
    }
  }
}

Severity:
(Low, Medium, High)

Priority:
(Critical, Medium, Low)

Expected Behavior
All test runs should pass without any failure.

Steps to Reproduce

1. Deploy v16.0.0 RC of the Helm charts
2. Run helm tests
3. GP and other tests should fail

Specifications

• Component (if known): ALS
• Version: v15.2.1
• Platform: K8s
• Subsystem:
• Type of testing: Helm test
• Bug found/raised by: @oderayi

Notes:

• Severity when opened:
• Priority when opened: