Extensions should be able to publish internal messages

[rgov]

I believe messages published by the Node Playground are "internal" to Studio in that they can be seen by other panels but don't actually get sent out over the network.

How about an extra flag to PanelExtensionContext.publish() or PanelExtensionContext.advertise() that allows an extension to do the same?

Use case: I was thinking of a potential way to support video codecs; I could have an extension that decodes the video messages and then internally re-publishes the decoded image so that an Image panel can render it, with zoom support and markers and everything else "for free".

Related to foxglove/studio#2212.

[amacneil]

This seems like a duplicate of https://github.com/foxglove/studio/issues/2212? How is it different?

[rgov]

foxglove/studio#2212 is about a completely new type of extension for doing A → B internal message transformations.

The proposal here is more about making it possible to vend these internal messages from panel extensions as well, as potential low-hanging fruit for refactoring / feature parity.

My video codec use case example above may have been confusing, since it sounds like it would be implemented with a data transform extension, but (a) the proposed API may not be sufficiently flexible, as I commented; (b) this has a shorter path to being implemented.

As a more contrived example, one could build a panel with some UI elements that lets the user control the generation of, say, an internal foxglove.GeoJSON message that they want displayed by a Map panel, for example by providing a list of lat/lon markers they want displayed for context. This would not work as a data transformation extension.

[jtbandes]

To clarify, I think our plan is for each extension package to be able to implement more than one extension point. For example, today, there is no reason one extension can't provide 5 new kinds of panels via multiple calls to registerPanel. But I can understand that it would be useful for user interactions with the panel to be able to change/redo the message transformations, so we'll need to keep in mind that extensions using multiple extension points may have side effects between their extension points.

[rgov]

That sounds great, but in the case of the GeoJSON example, you're not really transforming anything, you're purely synthesizing messages.

A fair question is also: Why not just publish regular messages? Except now Studio is sending out messages that potentially wind up in a rosbag or mcap unless you are very careful with your exclusion regex.

[amacneil]

A fair question is also: Why not just publish regular messages? Except now Studio is sending out messages that potentially wind up in a rosbag or mcap unless you are very careful with your exclusion regex.

This is an interesting question, I don't think there is a first principles answer other than that is the way it currently works. Perhaps we should make node playground and transform extensions always publish back to the live data source (although not for log files). One challenge is that we would be limited to message types where we have an existing schema (e.g. you wouldn't be able to publish arbitrary json).

[amacneil]

Fwiw I still don't see this as a distinct request from https://github.com/foxglove/studio/issues/2212, it seems we are only discussing different ways of implementing it.