How to use rust-cbor Encoder/Decoder


#1

This is more of a question to @BurntSushi i guess :smile: but anyone can help of-course.

The thing is I have several types, say Type0, … , Type6. So I implement the usual traits for them to encode their content. All these types come in with separate tags of their own which are u64. Now at the deserialization end I do not know what i received. So what do you think is the best way:

<1> Try decoding it to all types Type0 - Type6 one by one till it succeeds. Is this slow or does it use some llvm name-mangler to quickly determine the discrepancy and abandon it giving error so that I can try decoding it to the next type etc ? Or is this going to be slow ? Or is this possible at all in the sense that if layouts of TypeX and TypeY are same (identical members) then serialized content of one can be deserialized into other ?

<2> I can put the tag i mentioned for the type separately like (pseudocode) CborTagEncode::new(tag_u64, &(&self.name, &self.owner, &self.value)).encode(e). But now at the decoding end i want to extract tag_u64 thingy separately before i decide what to decode the rest of the stream into. Is this possible ? If yes (which will be of tremendous help) how to do it ?

<3> This is obvious if the above two fail: So i serialize TypeX, get Vec then serialize this along with the tag. So while decoding i deserialise into tag and Vec and then depending on tag, deserialise Vec into the concrete type. This will and does work but i want to avoid this dual deserialization. Is it possible using <1> <2> or any other method ?


#2

First and foremost, have you studied the example for decoding/encoding custom tags? I think it should at least answer #2: at the decoding end, you must read the tag number (read_u64), and then you can decode your actual data structure: http://burntsushi.net/rustdoc/cbor/#example:-encode-and-decode-custom-tags — There should be tests that demonstrate this too.

For (1), there’s no name mangling going on. If all your types have the same representation, then the decoder/encoder won’t care. I think the ideal situation is to go with (2): decode the tag number, then decode into the appropriate Rust type.

I don’t think you need to resort to (3).

Finally, be warned, it is not fast. You should check the benchmarks. rustc-serialize essentially forces the implementation to decode into an intermediate data structure before decoding into your types. There is a “direct” decoder that is purposefully incomplete that is much faster. But it may eat your laundry. Hopefully we can all move to serde soon…


#3

Ah sorry, that seemed to be inside the impl for MyDataStructure in that example so I don’t know what you mean.
I want to do it outside. Once into that function it’s too late. If I could do:

let d = Decoder::from_bytes(.....);
let tag = d.read_u64();
match tag {
123 => { let concretetype: Type0 = d.decode().next().unwrap().unwrap(); }
 .....
}

then it would be useful.


#4

Well, did you try it? :slight_smile: It seems like it should work.

Why? e.g.,

enum MyTypes {
    Type0(Type0),
    Type1(Type1),
    ...
}

impl Decodable for MyTypes {
    fn decode<D: Decoder>(d: &mut D) -> Result<MyTypes, D::Error> {
        Ok(match try!(d.read_u64()) {
            567 => MyTypes::Type0(try!(Decodable::decode(d))),
            568 => MyTypes::Type1(try!(Decodable::decode(d))),
            ...
        })
    }
}

Now you can let val: MyTypes = d.decode().next().unwrap().unwrap().


#5
spandan@spandan-Lenovo-Y50-70:~/virtualization/coding/rust-maidsafe/maidsafe-types/maidsafe_types$ cargo test  data::structured_data::test::serialisation_structured_data
   Compiling maidsafe_types v0.2.0 (file:///home/spandan/virtualization/coding/rust-maidsafe/maidsafe-types/maidsafe_types)
src/data/structured_data.rs:177:21: 177:31 error: no method named `read_u64` found for type `cbor::decoder::Decoder<std::io::cursor::Cursor<collections::vec::Vec<u8>>>` in the current scope
src/data/structured_data.rs:177         let val = d.read_u64();
                                                    ^~~~~~~~~~

still dont understand u here - must be missing something silly but dont know what

Also:

i dont want that extra thing … Now i nned a swich case again


#6

Try let tag: u64 = d.decode().next().unwrap().unwrap().

OK, but it should still solve your problem.


#7

Ok so that worked but subsequent decode fails.

/// StructuredData
#[derive(Clone, PartialEq, Debug)]
pub struct StructuredData {
    type_tag: StructuredDataTypeTag,
    name: NameType,
    owner: NameType,
    value: Vec<NameType>,
}

impl Encodable for StructuredData {
    fn encode<E: Encoder>(&self, e: &mut E) -> Result<(), E::Error> {
        CborTagEncode::new(5483_002, &(&self.name, &self.owner, &self.value)).encode(e)
    }
}

impl Decodable for StructuredData {
    fn decode<D: Decoder>(d: &mut D) -> Result<StructuredData, D::Error> {
        // try!(d.read_u64());
        let (name, owner, value) = try!(Decodable::decode(d));
        let structured = StructuredData {
            type_tag: StructuredDataTypeTag,
            name: name,
            owner: owner,
            value: value
        };
        Ok(structured)
    }
}

    #[test]
    fn serialisation_structured_data() {
        let _obj_before = StructuredData::generate_random();
        let _obj_before_clone = _obj_before.clone();
        let _obj_before1 = StructuredData::generate_random();

        let mut e = Encoder::from_memory();
        e.encode(&[&_obj_before]).unwrap();

        let mut d = Decoder::from_bytes(e.as_bytes());
        let tag: u64 = d.decode().next().unwrap().unwrap(); println!("===============> {:?}", tag);
        let _obj_after: StructuredData = d.decode().next().unwrap().unwrap(); // This Panics
       /*
        assert_eq!(obj_before, obj_after);
        assert!(!(obj_before != obj_before_clone));
        assert!(obj_before != obj_before1);
        */
    }

So this fails
let _obj_after: StructuredData = d.decode().next().unwrap().unwrap(); // This Panics

thread 'data::structured_data::test::serialisation_structured_data' panicked at 'called `Option::unwrap()` on a `None` value', ../src/libcore/option.rs:362

but if i uncomment the line to read_u64 and remove this line:
let tag: u64 = d.decode().next().unwrap().unwrap(); println!("===============> {:?}", tag);
then everything works.

I feel i am getting close but not quite.


#8

You probably shouldn’t be re-creating the decode iterator. Try let mut it = d.decode() and then use it.next().

If that still doesn’t work, please provide code that I can run that will reproduce the problem.


#9

Ok here’s a minimum code where you can show me what you mean:

/// A
pub struct A {
    some_type: usize,
}

impl Encodable for A {
    fn encode<E: Encoder>(&self, e: &mut E) -> Result<(), E::Error> {
        CborTagEncode::new(5483_002, &(&self.some_type)).encode(e)
    }
}

impl Decodable for A {
    fn decode<D: Decoder>(d: &mut D) -> Result<A, D::Error> {
        let some_type = try!(Decodable::decode(d));
        let a = A {
            some_type: some_type,
        };
        Ok(a)
    }
}

//------------------------------


   fn main() {
        let a0 = A { some_type: 20, };
        let mut e = Encoder::from_memory();
        e.encode(&[&a0]).unwrap();

        let mut d = Decoder::from_bytes(e.as_bytes());
            let mut it = d.decode();
            let tag: u64 = it.next().unwrap().unwrap();
            assert_eq!(5483_002, tag);
            // Now decode the rest of it:
            // let a1: A = it.next().unwrap().unwrap(); // Can't uncomment this - It will fail build
}

So if you can get this working it will be of big help.