Performance test /transactions endpoints.

[mgoelswirlds]

Load test /transactions endpoints using a perf tool.

[MarcKriguerAtHedera]

Today's subtask was to try to determine why so few of the what-was-supposed-to-be-100B transactions only come in as 4.4B transactons; I found nothing helpful in the pinot-minion logs over the weekend, and I posted a question on the Apache Pinot Slack, but (so far) just a referal to someone else, not a specific answer yet. Just in case Pinot just wants to ingest .avro and not .avro.gz files, I am rerunning a few of the parallel branch copy-and-change-timestamp tasks again, but it would take ~4 days to do it all the way through. This is just to see if .avro vs .avro.gz is at all a factor.

[MarcKriguerAtHedera]

It definitely appears that the "snappy" (interally compressed, but still with an .avro suffix, not a .avro.gz suffix) files are the problem. In trying to speed things up, we wound up with an .avro format that the avro tools are happy with, but which Pinot is not happy with.

[mgoelswirlds]

The results from /transactions endpoints were as follows