Incompatible Value binary serialization format

[shesek]

It appears that the bincode value fix resulted in a different serialization format compared to before the regression (v0.16).

With current master (fe3e946), the following code:

let commitment = PedersenCommitment::from_slice(&Vec::from_hex("09b88680778d3d5530785ff12bb62e6ac98c7be5a8b9b0ea392e166bf61ea77ee4").unwrap()).unwrap();
let value = Value::Confidential(commitment);

println!("BE bytes: {}", bincode::serialize(&value).unwrap().to_hex());
println!("LE bytes: {}", bincode::DefaultOptions::default().with_little_endian()
    .serialize(&value).unwrap().to_hex());

Results in the following serialization:

BE bytes: 020000000000000002210000000000000009b88680778d3d5530785ff12bb62e6ac98c7be5a8b9b0ea392e166bf61ea77ee4
LE bytes: 02022109b88680778d3d5530785ff12bb62e6ac98c7be5a8b9b0ea392e166bf61ea77ee4

While with v0.16, the following code:

let value = Value::from_commitment(&Vec::from_hex("09b88680778d3d5530785ff12bb62e6ac98c7be5a8b9b0ea392e166bf61ea77ee4").unwrap()).unwrap();

println!("BE bytes: {}", bincode::serialize(&value).unwrap().to_hex());
println!("LE bytes: {}", bincode::DefaultOptions::default().with_little_endian()
    .serialize(&value).unwrap().to_hex());

Results in the following serialization:

BE bytes: 020000000000000009b88680778d3d5530785ff12bb62e6ac98c7be5a8b9b0ea392e166bf61ea77ee4
LE bytes: 0209b88680778d3d5530785ff12bb62e6ac98c7be5a8b9b0ea392e166bf61ea77ee4

[stevenroose]

(The links in the first line don't work.)

[stevenroose]

Seems like #77 introduced this. We definitely could use more tests where all serializations are tested for regressions.

[shesek]

(The links in the first line don't work.)

Oops, sorry, fixed them now.

[shesek]

To give some more context, this incompatibility resulted in electrs being unable to read db entries that were created with rust-elements v0.16 using the current master. Working around this issue required deleting the old database and running a re-index.

[stevenroose]

It's the danger of having all our releases be breaking. 0.15-0.16 is breaking, so both rust-elements, electrs and espora should have had regression tests for data formats..