Payload type matching for workers talking with multiple protocols

[hairyhum]

The problem

Sometimes workers need to communicate with multiple services with messages encoded in multiple protocols.

On the routing level payload is a binary data, which is encoded with BARE encoding.
BARE encoding (just as any other structured binary encoding) expects both sides to have a schema.

When a worker needs to talk to a single service communicating in different protocols, it can use union types to distinguish between types of messages. But when it needs to talk to independent services they cannot share a union type.

Proposal

To separate protocols and user code some sort of metadata can be used to match which code should handle which type of message.

Ideally the library would handle the matching, while user code should only implement the message handling.

The routing protocol data should not be affected by adding the metadata.

Ideas

Use an address per type:

For each service a worker talks to use a different return_route address.
This way an address on which it receives a response will mark the type of response it should handle.

This may be implemented with multiple workers for different protocols, which would share some state or communicate with a "coordinating worker" or with matching on onward_route address.

In this case address will act as a metadata

Add metadata to payload:

Make the message format look like this

{
  onward_route: ...
  return_route: ...
  payload: {
    data: opaque
    metadata: something
  }
}

The payload might be encoded as a binary with BARE scema:

type Payload {
  data: data
  metadata: MetadataType
}

Use protocol ID in the metadata

type Payload {
  data: data
  protocol_id: uint
}

In this case each protocol would have to define an id it talks in.

The main challenge is that the protocol id mapping to an actual protocol should be shared between nodes

As an improvement to that, the protocol id may be a more descriptive string

type Payload {
  data: data
  protocol_id: string
}

Use session ID in the metadata

type Payload {
  data: data
  session_id: uint
}

Unlike the protocol id, session id may be chosen dynamically by the initial sender, the protocol depends on the receiver of this message.

The sender would have to associate expected type with a session id and store this association somewhere. Also if responder wants to send more messages to the sender, it would have to have such association as well.

Use request ID in the metadata

type Payload {
  data: data
  request_id: uint
}

Unlike the session id, request id is generated for each request from the sender, so the sender doesn't need to save it.
Responder would need to use a request_id received from the sender when sending responses

To allow responder to initiate new requests to the sender, this type can be extended to distinguish request id and response id

type Payload {
  data: data
  metadata: {
    request_id: uint
    response_id: uint
  }
}

Responder will use request id from the requests as response id and may add request id allowing two-way request-response communication.

Combining request id with protocol/session id

Request-response identity may be useful for identifying a specific message and not only the protocol.

This might be an overkill for this particular problem

type Payload {
  data: data
  metadata: {
    protocol_id: string
    request_id: uint
    response_id: uint
  }
}

[spacekookie]

One suggestion that isn't listed here is adding the protocol metadata to the underlying message:

type Message {
  version: uint,
  onward_route: Route,
  return_route: Route,
  protocol: {
    // ... whatever selection of metadata we want
  },
  payload: Data
}

The advantage of that is that a Rust worker could ignore the payload (and thus might be able to be handled internally to ockam_node) and proxy messages to a child worker that can then explicitly handle a particular payload.

[hairyhum]

Good point.
I was thinking of keeping the volatile part of the routed message (routes) separate from the forwarded message (payload) to shape this protocol handling more like a next layer rather then a part of the routing protocol.
Mostly to allow things like forwarding and wrapping

[spacekookie]

Having thought about possibly implementations of this, I think keeping the protocol metadata out of the payload field would make things much easier for the Rust implementation.

I don't really see the need to keep all the non-volatile state of a message to one field, considering we already have a version field which isn't touched.

[hairyhum]

Message types

Use a type per message to name formats:

type Payload {
  data: data
  type: string
}

Each message has a string-tagged type and a BARE schema, which can be defined as a part of "a protocol" like this:

{
  "my_type": BARE_SCHEMA,
  "my_other_type": BARE_SCHEMA
}

With message types there would be no need for union types and it makes it easier to combine multiple types from different protocols, but naming might be hard as all types need to be unique in a context of client/service interaction. If multiple services use the same type name, the client cannot use both of them at the same time.

[hairyhum]

Looking at the examples, to make them work with a stream protocol publisher to look the same as the other examples, it would make sense to implement the "pass through" protocol to send binary data and force all the messages to have a protocol.
It might look like this:

type Message {
  version: uint
  onward_route: ...
  return_route: ...
  payload: data
}
type Payload {
  protocol: ...
  data: data
}

Or like this:

type Message {
  version: uint
  onward_route: ...
  return_route: ...
  payload: Payload
}
type Payload {
  protocol: ...
  data: data
}

But ALL the payloads have to contain the protocol.
If a worker wants to send a binary data in there, it MUST use:

{
  protocol: "binary"
  data: ...
}

This also addresses the issue of double-encoding of binary payloads by Rust code.

[hairyhum]

Formalizing protocol API: https://github.com/ockam-network/proposals/pull/81
Example implementation: 163c1e5

[hairyhum]

After implementing protocol matching for stream protocol, it appears to be fairly complex.
As most of the message structures are organized as payload bearers, there is another idea on how to organize the data structures and distinguish between data types proposed in https://github.com/ockam-network/proposals/pull/87