How to use rust-cbor Encoder/Decoder

This is more of a question to @BurntSushi i guess :smile: but anyone can help of-course.

The thing is I have several types, say Type0, ... , Type6. So I implement the usual traits for them to encode their content. All these types come in with separate tags of their own which are u64. Now at the deserialization end I do not know what i received. So what do you think is the best way:

<1> Try decoding it to all types Type0 - Type6 one by one till it succeeds. Is this slow or does it use some llvm name-mangler to quickly determine the discrepancy and abandon it giving error so that I can try decoding it to the next type etc ? Or is this going to be slow ? Or is this possible at all in the sense that if layouts of TypeX and TypeY are same (identical members) then serialized content of one can be deserialized into other ?

<2> I can put the tag i mentioned for the type separately like (pseudocode) CborTagEncode::new(tag_u64, &(&self.name, &self.owner, &self.value)).encode(e). But now at the decoding end i want to extract tag_u64 thingy separately before i decide what to decode the rest of the stream into. Is this possible ? If yes (which will be of tremendous help) how to do it ?

<3> This is obvious if the above two fail: So i serialize TypeX, get Vec then serialize this along with the tag. So while decoding i deserialise into tag and Vec and then depending on tag, deserialise Vec into the concrete type. This will and does work but i want to avoid this dual deserialization. Is it possible using <1> <2> or any other method ?

1 Like

First and foremost, have you studied the example for decoding/encoding custom tags? I think it should at least answer #2: at the decoding end, you must read the tag number (read_u64), and then you can decode your actual data structure: cbor - Rust --- There should be tests that demonstrate this too.

For (1), there's no name mangling going on. If all your types have the same representation, then the decoder/encoder won't care. I think the ideal situation is to go with (2): decode the tag number, then decode into the appropriate Rust type.

I don't think you need to resort to (3).

Finally, be warned, it is not fast. You should check the benchmarks. rustc-serialize essentially forces the implementation to decode into an intermediate data structure before decoding into your types. There is a "direct" decoder that is purposefully incomplete that is much faster. But it may eat your laundry. Hopefully we can all move to serde soon...

Ah sorry, that seemed to be inside the impl for MyDataStructure in that example so I don't know what you mean.
I want to do it outside. Once into that function it's too late. If I could do:

let d = Decoder::from_bytes(.....);
let tag = d.read_u64();
match tag {
123 => { let concretetype: Type0 = d.decode().next().unwrap().unwrap(); }
 .....
}

then it would be useful.

Well, did you try it? :slight_smile: It seems like it should work.

Why? e.g.,

enum MyTypes {
    Type0(Type0),
    Type1(Type1),
    ...
}

impl Decodable for MyTypes {
    fn decode<D: Decoder>(d: &mut D) -> Result<MyTypes, D::Error> {
        Ok(match try!(d.read_u64()) {
            567 => MyTypes::Type0(try!(Decodable::decode(d))),
            568 => MyTypes::Type1(try!(Decodable::decode(d))),
            ...
        })
    }
}

Now you can let val: MyTypes = d.decode().next().unwrap().unwrap().

spandan@spandan-Lenovo-Y50-70:~/virtualization/coding/rust-maidsafe/maidsafe-types/maidsafe_types$ cargo test  data::structured_data::test::serialisation_structured_data
   Compiling maidsafe_types v0.2.0 (file:///home/spandan/virtualization/coding/rust-maidsafe/maidsafe-types/maidsafe_types)
src/data/structured_data.rs:177:21: 177:31 error: no method named `read_u64` found for type `cbor::decoder::Decoder<std::io::cursor::Cursor<collections::vec::Vec<u8>>>` in the current scope
src/data/structured_data.rs:177         let val = d.read_u64();
                                                    ^~~~~~~~~~

still dont understand u here - must be missing something silly but dont know what

Also:

i dont want that extra thing .. Now i nned a swich case again

Try let tag: u64 = d.decode().next().unwrap().unwrap().

OK, but it should still solve your problem.

Ok so that worked but subsequent decode fails.

/// StructuredData
#[derive(Clone, PartialEq, Debug)]
pub struct StructuredData {
    type_tag: StructuredDataTypeTag,
    name: NameType,
    owner: NameType,
    value: Vec<NameType>,
}

impl Encodable for StructuredData {
    fn encode<E: Encoder>(&self, e: &mut E) -> Result<(), E::Error> {
        CborTagEncode::new(5483_002, &(&self.name, &self.owner, &self.value)).encode(e)
    }
}

impl Decodable for StructuredData {
    fn decode<D: Decoder>(d: &mut D) -> Result<StructuredData, D::Error> {
        // try!(d.read_u64());
        let (name, owner, value) = try!(Decodable::decode(d));
        let structured = StructuredData {
            type_tag: StructuredDataTypeTag,
            name: name,
            owner: owner,
            value: value
        };
        Ok(structured)
    }
}

    #[test]
    fn serialisation_structured_data() {
        let _obj_before = StructuredData::generate_random();
        let _obj_before_clone = _obj_before.clone();
        let _obj_before1 = StructuredData::generate_random();

        let mut e = Encoder::from_memory();
        e.encode(&[&_obj_before]).unwrap();

        let mut d = Decoder::from_bytes(e.as_bytes());
        let tag: u64 = d.decode().next().unwrap().unwrap(); println!("===============> {:?}", tag);
        let _obj_after: StructuredData = d.decode().next().unwrap().unwrap(); // This Panics
       /*
        assert_eq!(obj_before, obj_after);
        assert!(!(obj_before != obj_before_clone));
        assert!(obj_before != obj_before1);
        */
    }

So this fails
let _obj_after: StructuredData = d.decode().next().unwrap().unwrap(); // This Panics

thread 'data::structured_data::test::serialisation_structured_data' panicked at 'called `Option::unwrap()` on a `None` value', ../src/libcore/option.rs:362

but if i uncomment the line to read_u64 and remove this line:
let tag: u64 = d.decode().next().unwrap().unwrap(); println!("===============> {:?}", tag);
then everything works.

I feel i am getting close but not quite.

You probably shouldn't be re-creating the decode iterator. Try let mut it = d.decode() and then use it.next().

If that still doesn't work, please provide code that I can run that will reproduce the problem.

Ok here's a minimum code where you can show me what you mean:

/// A
pub struct A {
    some_type: usize,
}

impl Encodable for A {
    fn encode<E: Encoder>(&self, e: &mut E) -> Result<(), E::Error> {
        CborTagEncode::new(5483_002, &(&self.some_type)).encode(e)
    }
}

impl Decodable for A {
    fn decode<D: Decoder>(d: &mut D) -> Result<A, D::Error> {
        let some_type = try!(Decodable::decode(d));
        let a = A {
            some_type: some_type,
        };
        Ok(a)
    }
}

//------------------------------


   fn main() {
        let a0 = A { some_type: 20, };
        let mut e = Encoder::from_memory();
        e.encode(&[&a0]).unwrap();

        let mut d = Decoder::from_bytes(e.as_bytes());
            let mut it = d.decode();
            let tag: u64 = it.next().unwrap().unwrap();
            assert_eq!(5483_002, tag);
            // Now decode the rest of it:
            // let a1: A = it.next().unwrap().unwrap(); // Can't uncomment this - It will fail build
}

So if you can get this working it will be of big help.