Jepsen for JetStream

[vdwees]

Earlier this year Redpanda went through the Jepsen audit process: http://jepsen.io/analyses/redpanda-21.10.1

Wondering if there is a plan to do something similar for JetStream?

[derekcollison]

Yes there is.

[vdwees]

That's great, I'm really looking forward to reading about it! Is there a rough time frame yet for when it will happen?

[derekcollison]

Possibly 2.9, or 3.0 release. 2.9 is scheduled for next week, fingers crossed.

The testing process is quite pricey as well, so need to budget.

[hwinkel]

Is there any progress in this regards?

[derekcollison]

Yes we have been working with a stealth startup for almost a year now that is incredible. Will let @mprimi divulge more.

[mprimi]

I'm not going to comment on the official Jepsen audit, but I can tell you that for some time we have been hammering NATS and JS with a Jepsen-like approach (e.g. invariant testing with fault injection). Jepsen(-like) testing is great to uncover bugs in code that deals with fault-tolerance and recovery, that are hard to test otherwise.
We've also been relying on Antithesis (https://antithesis.com/) find even more insidious issues. Antithesis is like Jepsen on steroids and can help to seek and destroy one-in-a-million heisenbugs that are so hard to track down in distributed systems.

[atropos112]

With it being couple months, have there been any news about audits?

[mprimi]

We keep continue expanding on testing.

What I can tell you is we're putting NATS/JetStreams through scenarios and trials much more evil, complex and comprehensive than what Jepsen (the software) would do.
Happy to share privately, but we're not ready to be completely open about this just yet.

On the other side, you probably want to see a third-party report and stamp of approval (i.e. Jepsen the blog).
I don't have comments or an ETA on that.