Full support for Pulsar in LangStream

[cdbartholomew]

We should be able to use Pulsar as the streaming source for all parts of LangStream.

[eolivelli]

The main things that must work are:

• schema-less topics or schema = STRING
• all the standard agents
• Python code
• Gateways
• intermediate topics

this that are to be checked and probably there is something to do:

• how we handle subscriptions, ids and types
• message acknowledgments
• make it work with both Failover, SHARE and KEY_SHARED subscriptions
• Kafka Connect (this is a big work, @eolivelli has some ideas about how to make it work quite easily)

Notes:

• DLQ -> we use our DLQ, not Pulsar buillt-in
• ackTimeout -> disabled !
• retention, for topics read by the gateways retention must be set by time (because we use the Reader API)

Kafka Connect is "nice-to-have", it is not required to build all the most common pipelines (webcrawler, chatbots...)

[cbornet]

Pulsar now works correctly in LangStream.
Remaining work is:

• [feature] Avro support for Pulsar #610
• [feature] Support Kafka Connect connectors with Pulsar #635
  Closing this issue and will follow work in those issues.