know if a serialized piece of Arkworks structure has been compressed

[a-stevan]

i couldn't find anything related in the issue tracker, so here i am 😇

let's say i have the following piece of code

fn ser(obj: impl CanonicalSerialize, compress: Compress) -> Vec<u8> {
    let mut serialized = vec![0; obj.serialized_size(compress)];
    obj.serialize_with_mode(&mut serialized[..], compress)?;
    serialized
}

is it possible, on the other end of the protocol, to deserialize the bytes without passing around the same compress: Compress, e.g. maybe the information about compression is inside the bytes or something?
i.e. i'm wondering if something like the following is possible

fn deser(bytes: &[u8], validate: Validate) -> Result<MyStruct> {
    let compress = is_it_compressed_or_not(bytes); // this is the part i couldn't find

    let my_struct = Block::deserialize_with_mode(&bytes[..], compress, validate)?
    Ok(my_struct)
}

thanks and cheers 🥳 👋

[mmagician]

Compression is very data specific, so it's hard to answer this question in full generality without knowing what comprises your struct.
You can always try to deserialize with, and then without compression, and hopefully one of them fails - but note that it's not always the case:

• some types might not even support compression, so both versions work. This is not much of an issue, just double the work.
• some types might give you different results. I think the clearest example is Vec. Given e.g. a sequence of 96 bytes, you might successfully deserialize into either 2 EC points (assuming compressed) or 1 non compressed point on the curve. Goes without saying that they represent completely different underlying data.

[burdges]

Afaik, you should always compress for network wire formats, because compression's extra CPU time costs less than the extra network bytes, and blockchains magnify this. In particular, I'd expect the projective to affine conversions cost more, making batch_normalize more significant.

It's different when you're only storing cached data locally, well then maybe uncompressed and/or unvalidated makes sense. In principle, one could even serialize in projective locally, but in general projective serialization is extremely dangerous.

[a-stevan]

@mmagician @burdges
thanks for the quick answers, this is valuable information i think 🙏