spike: standup a credo-based mediator service hosted in OpenShift

[cvarjao]

• Check Animo Mediator code and check license

Tasks

• stand up a running mediator service
• Setup Build/Deployment Pipeline
• Deployed and running in OpenShift

[jleach]

Tested and messages are not cached if the client (wallet) is off-line. It makes no differences if SQLite or Postgres is used. This little bit of logic is used to setup the BC Wallet to use the OOB invitation.

Add to initalize-agent.ts

const startMediation = useCallback(async (agent: Agent) => {
    const invitationId = 'a4915f12-4593-4bf2-a55e-778cd7389840'
    // const invitationId = '6636325c-5e4b-4573-86bc-13bb98c78110'
    const outOfBandRecord = await agent.oob.findByReceivedInvitationId(invitationId)

    let [connection] = outOfBandRecord ? await agent.connections.findAllByOutOfBandId(outOfBandRecord.id) : []

    if (!connection) {
      agent.config.logger.debug('Mediation connection does not exist, creating connection')
      // We don't want to use the current default mediator when connecting to another mediator
      // const routing = await agent.mediationRecipient.getRouting({ useDefaultMediator: false })

      // agent.config.logger.debug('Routing created', routing)

      const invite = await agent.oob.parseInvitation(Config.MEDIATOR_URL!)
      const { connectionRecord: newConnection } = await agent.oob.receiveInvitation(invite)

      // agent.config.logger.debug('Mediation invitation processed', { mediatorDid })

      if (!newConnection) {
        console.log('No connection record to provision mediation.')
        // throw new CredoError('No connection record to provision mediation.')
      }

      connection = newConnection!
    }

    const readyConnection = connection.isReady
      ? connection
      : await agent.connections.returnWhenIsConnected(connection.id)

    return agent.mediationRecipient.provision(readyConnection)
  }, [])

[swcurran]

So that starting point seems to be worse than for the current implementation with an ACA-Py-based mediator.

Should we consider updating ACA-Py to add the PickUp 2 protocol to eliminate the bug we have discovered? Happy not to, but wondering if that would be the easiest path forward?

What code base did you start from? https://github.com/openwallet-foundation/didcomm-mediator-service and the “afj” code within that repo?

Have you sorted out the status of the use of Pickup Protocol V2, and the “live” component that you have heard about?

[swcurran]

Looking a bit more at the ACA-Py-based Mediator implementation at https://github.com/openwallet-foundation/didcomm-mediator-service, which is what we are using at BC Gov.

• The caching in the ACA-Py instance is implemented with Redis. It is doing the persistent caching. I suspect that is why the AFJ/Credo version is not working.
• The ACA-Py mediator also has Expo support for Notifications, the feature that was implemented by BC Gov about a year ago (I think). That functionality will also have to be replicated in a Credo-based mediator. Please add that to the to-do list. :-(

[jleach]

@swcurran

I started with this. It's based on Animo's public mediator but updated for Credo-ts 0.5.0. The one you link above is getting outdated. I have a repo for it here and am waiting for Clecio to create one in bcgov so I can do a PR back to it with my changes.

Going with ACA-py would save us some work for sure, but, we would still have lots of work to do and the community would need to commit to seeing it all done. From the meeting last Thursday, If we went with ACA-py we'd still need to address:

1. Ownership (needs to be owned and run like a prod system);
2. HA (needs to have more than 1 pod and support fail over, probably DR also in Gold cluster);
3. Scalability (Socket Doc or some other mechanism for multiple pods to serve one WebSocket).

If we went with a Credo-ts based mediator we would still have to do the above, in addition to:

4. Message queueing when agents are off-line;
5. Maybe some work on push notifications to work as it does now with Firebase.

Going with credo also simplifies the tech stack in that our team can easily manage the code base along with the attestation agent, VDR proxy, and BC Wallet (all typescript based).

In short, my though is that unless the ACA-py team is ready and willing to pull the trigger on 1-3 we should continue with a Credo based solution. Not my call though.

CC. @cvarjao

[swcurran]

My $0.02CDN. I’m not a fan of the levels of forking going on. Would be much happier if we were contributing to something directly — and in particular to the OWF "didcomm-mediator-service" repo that is the “best known" mediator implementation. I’m surprised that Credbl is using a fork of the Animo project. Credbl was just accepted as an LFDT project — weird that they would have a fork of something as a core component.

I’m not sure what you mean about 1. The mediator is owned and run as a prod system today. The VC Services team was not aware of the pickup bug until last week and so has never investigated it, and the time (Wallet and VC Services) had previously agreed we would not work on the HA aspects. There was a thought about doing a Code With Us back in the day, but that didn’t happen. The VC Services team did add the Redis capability and previously load tested the service and found it satisfactory.

Since AFAIK from looking around that the PickUp protocol 2 is relatively well-known is “just another protocol” — unlike things like adding redis and Firebase.

I think you listed a few other items that will need to be done on the Credo version — invitation coordination, migration at least come to mind. There are also things needed in the BC Wallet, such as updating to PickUp protocol 2.

[jleach]

I’m not sure what you mean about 1. The mediator is owned and run as a prod system today. The VC Services team was not aware

I though it was noted in the meeting that it's on v0.10.4 and needs more active updates.

[swcurran]

There are dependabot updates needed, but few requests for updates. The PickUp bug report (which isn’t in the repo) is the first in a very long while. I’m a bit concerned that we won’t be continuing with the mediator coming from a neutral location such as its current home (https://github.com/openwallet-foundation/didcomm-mediator-service). So even if we switch to Credo — I hope we update that repo with the work that gets done.

[ajile-in]

My $0.02CDN. I’m not a fan of the levels of forking going on. Would be much happier if we were contributing to something directly — and in particular to the OWF "didcomm-mediator-service" repo that is the “best known" mediator implementation. I’m surprised that Credbl is using a fork of the Animo project. Credbl was just accepted as an LFDT project — weird that they would have a fork of something as a core component.

Hi @swcurran - Thanks for your feedback. CREDEBL team is open to collaborate & contribute the mediator efforts in the OWF didcomm-mediator-service repo. Although we are working on the forked repo (JS/TS Mediator), we have added many features required for high availability & scaling. We will certainly reach out to right people at OWF to see how we can join hands. I am sure it will benefit the community.

[swcurran]

Thanks @ajile-in — we can help with that as you look to move things to OWF. We have agreed that a new repo — didcomm-mediator-service-credo be added for a Credo implementation (vs. having two implementations in one repo), and the current didcomm-mediator-service have the -acapy extension added. If one (or some other implementation) becomes dominant, we can reconsider the names.

[swcurran]

Would love to see/hear about the features of the CREDEBL mediator — things like a Redis-like persistent queue and scalable websocket handling. BC implemented a Firebase-based mobile app notifications systems for the current mediator and Bifold, and would likely help to get that added to a common Credo-based mediator.

[jeznorth]

Moving this back to the sprint backlog while we wait for 3rd tasks to be completed. @jleach will submit a PR to finalize this work once those tasks are no longer blocking the work.

[jleach]

@swcurran @cvarjao Joined the Credo-TS group call this morning. We all agreed to create and contribute back to a single repo. It's being tracked here so I'm going to close this issue we can carry on any discussion in there for better visibility.