Stream Health not finding active streams

[iameli]

Stream Health can't find my stream either, even though it uses a completely different mechanism for finding active streams.

[pglowacky]

Got another report of this from a customer, who attached documentation. Since customers seem to be noticing this, this item grows in prioritity. @iameli @victorges Can we work on this as soon as the e2e stream health replication test is complete?

https://app.zenhub.com/files/386768945/3c26eade-9a71-40e0-8c1e-a5a5dfeb9f29/download

[victorges]

We do have the e2e global replication tests going on now!

Would be nice to know if these still happen very often / when we are not aware of them by a record-tester alert. @pglowacky have you had any other reports about this recently?

I know that there is 1 thing which may still happen on the dashboard, which is because we still depend on the API's own personal opinion of whether the stream is active or not. If the problems are only happening because of that, we could get started with the plan to replace the isActive completely by the data provided by Stream Health. Next year's @iameli, WDYT?