Ability to compare two or more files

[amacneil]

Use case: An engineering team records data from their robot. Later, they rerun some of the recorded data against an updated codebase (new algorithms / models), and record the results. In some cases, they may replay the same recorded data against several new alternative versions of the code, and record the results.

Then they would like to compare the output from the original recording with the new recording, in the same studio instance. For example, overlaying the new behavior visualization on top of the old behavior.

This feature was partially supported in original Webviz code, but was removed due to being hard coded to Cruise conventions in many places and not flexible.

Workaround: It is possible to achieve this today by rewriting the newer bag using a topic prefix. I'm not sure if existing CLI tools can do so, but a very short python script using the rosbag library would achieve it.

[HugoFeng]

just to elaborate:
Use case: load raw_data.bag(very large) + gt.bag(small) + result1.bag(small) + result2.bag(small)
As mentioned by @amacneil , users are allowed to append a name prefix for each topic.
So that we can add this link to CI report something like: ?remote_bag=[raw.bag;gt.bag(/gt);result1.bag(/result1);result2.bag(/result2)] . We can open the link to check out the performance against the gt as well as the current SOTA.

benifit:

• raw bags are the most "heavy" ones, but results are light weight with almost all structured data ( positions and semantics). In this way we don't need to save all data for each "run".
• the comparison is generate on the fly, no new bags are created. we may also add additional bags to compare while viewing

Additionally, by loading multiple results with gt results, we can create a compare panel that computes all the "error or difference" for each result against "gt". WIth the help of ploting view, we then can quickly visualize the error peaks and jump between them with timeline synced.

[luqnaim]

Thanks to @amacneil for mentioning this issue. Linking my comment here for reference: #109

@amacneil ah, thank you for the link. I did not come across that issue during my search. Answering your question, issue #135 is exactly what i am looking for. I would like to inspect the behavior of multiple algorithms on the same video sequence and be able to compare them frame by frame. I was able to do this with webviz but was limited to only 2 simultaneous bag files.

[DDDBUG001]

hope the feature can be implemented soon, it would be better to have "showdiff" feature (same like what webviz implemented), so that we could compare the results generated from two models.