You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When a Waltz log is shared amongst several applications, it might be useful for some applications to ignore messages that they don't care about. One way to ignore irrelevant message types would be to do server-side filtering, where the Waltz server or storage node drops unwanted messages. This is useful in a case where the application cares about messages across the whole timeline of the log.
A second use case is when the application is cares about only a subset of messages that might or might not be distributed all the way back to the beginning of the log. In such a case, it would be useful to have the application request the lowest watermark for the union of all the message types, so that the application can start reading from that point.
For example, suppose we have a Waltz log that's 2 years old. Four months ago, a new message type was added. A new application is being written that pays attention only to the new events. Having it fetch the low watermark for the new message type means that it can start reading from exactly the point in the log where the first message type first occurs, and allows the application to skip the vast majority of the irrelevant messages.
Note that this will have to be per-partition.
The text was updated successfully, but these errors were encountered:
When a Waltz log is shared amongst several applications, it might be useful for some applications to ignore messages that they don't care about. One way to ignore irrelevant message types would be to do server-side filtering, where the Waltz server or storage node drops unwanted messages. This is useful in a case where the application cares about messages across the whole timeline of the log.
A second use case is when the application is cares about only a subset of messages that might or might not be distributed all the way back to the beginning of the log. In such a case, it would be useful to have the application request the lowest watermark for the union of all the message types, so that the application can start reading from that point.
For example, suppose we have a Waltz log that's 2 years old. Four months ago, a new message type was added. A new application is being written that pays attention only to the new events. Having it fetch the low watermark for the new message type means that it can start reading from exactly the point in the log where the first message type first occurs, and allows the application to skip the vast majority of the irrelevant messages.
Note that this will have to be per-partition.
The text was updated successfully, but these errors were encountered: