Usage of QReLU #117 by Lithum 07

[chyomin06]

[chyomin06]

Hi,

Yes, you are right. QReLU was initially introduced in J. Ballé et al., "Integer networks for data compression with latent-variable models," ICLR'19

As you may have noticed, computed scales play a critical role to parse your coded latent using the entropy decoder.
You can imagine there exist machines that use different floating precision.
Unmatched precision ruins the parsed latent variables, significantly impacting the downstream output.

Therefore, we believe the original intention of using QReLU for the scale decoder prevents such discrepancies between machines.

Nonetheless, we couldn't find any statement that either input or output of QReLU is quantized to integers for the scale-space flow model in their paper.
So, the input, output, and intermediate computations of QReLU are still in floating precision in our implementation.
Only it does clip the output to be within the minimum and maximum for a given bit-depth.

Overall, for the implementation of QReLU, we try to follow all the details described in the paper mentioned above.
Thank you.