Serde Bincode writes file 1byte short?

help
1

I'm trying to learn serde and have a small example with bincode. I serialize to a file, but I cannot read back the data. Let me show the example, and explain:

use bincode;
use bincode::config;
use bincode::serde::encode_into_slice;
use bincode::serde::encode_into_std_write;
use serde::{Deserialize, Serialize};
use std::fs::File;
use std::io::Write;

#[derive(Serialize, Deserialize, Clone, Debug, Default)]
struct MyStruct {
    a: u32,
    b: u32
}


fn main() {
    let data = &vec![MyStruct::default(); 10];

    let config = config::standard();

    // Save to file directly
    let mut file = &mut File::create("test.bincode").unwrap();
    let size = encode_into_std_write(data, file, config::standard()).unwrap();
    println!("Bytes written: {}", size);
    file.flush().expect("Error at flush");

    // Save to buffer then save
    let destination = &mut [0u8; 1000];
    let size = encode_into_slice(data, destination, config).unwrap();
    let sub_dest = &destination[0..size+1]; //+1 makes it readable
    let file2 = &mut File::create("test.bincode2").unwrap();
    file2.write_all(sub_dest).expect("Error at write_all");

    let file_read = &mut File::open("test.bincode").unwrap();
    // This panics
    //let (res, size): (Vec<MyStruct>, usize) = bincode::serde::decode_from_std_read(file_read, config).unwrap();
    let file_read2 = &mut File::open("test.bincode2").unwrap();
    let (res, size): (Vec<MyStruct>, usize) = bincode::serde::decode_from_std_read(file_read2, config).unwrap();
    println!("{:?}", res);
}

I create two files:

  1. Serialize directly to a file, using: encode_into_std_write
  2. Serialize to a buffer, shorten to the relevant size, and then save

In the last step, I try to read back both versions: 1/ panics, 2/ works. Oddly, file 1/ looks as if the last byte was missing, and indeed: if I change &destination[0..size+1] and remove +1 I get the exact same binary.

So in conclusion: encode_into_std_write writes the data 1byte short, hence making it unreadable. Can this be a bug or am I missing something basic here?

1 Like
2

I don't know what exactly might be going wrong, but I notice your program is ignoring some Results. Please check your compiler warnings and add unwrap()s in the appropriate places. That will make sure that the problem is not due to an overlooked error.

1 Like
3

Modified accordingly - the issue persists.

4

See: Issue when decoding a simple Enum with no member values. · Issue #695 · bincode-org/bincode

Your type annotations are forcing the decode methods to look for a usize that does not exist.

Remove the +1 from the slice you take:

-    let sub_dest = &destination[0..size+1]; //+1 makes it readable
+    let sub_dest = &destination[0..size];

Then remove the trailing usize from the type annotation on the decode calls:

-    // This panics
-    //let (res, size): (Vec<MyStruct>, usize) = bincode::serde::decode_from_std_read(file_read, config).unwrap();
+    let res: Vec<MyStruct> = bincode::serde::decode_from_std_read(file_read, config).unwrap();
     let file_read2 = &mut File::open("test.bincode2").unwrap();
-    let (res, size): (Vec<MyStruct>, usize) = bincode::serde::decode_from_std_read(file_read2, config).unwrap();
+    let res: Vec<MyStruct> = bincode::serde::decode_from_std_read(file_read2, config).unwrap();
6 Likes
5

Oh, so obvious - thanks a lot!