What is the correct way to take serde_json into use?

[ZiCog]

I just wasted an hour trying to get serde_json to work in my program. Firstly I did the simple thing and followed the serde_json documentation: I put this into my module:

use serde::{Deserialize, Serialize};
use serde_json::Result;

and this my Cargo.toml

serde_json = "*"

Which results in errors: " unresolved import serde"

So I add serde to my Cargo.toml

serde = "*"

Which results in errors: " cannot find derive macro Serialize in this scope"

After an hour of googling and trying various suggestions that don't work and trying to understand the discussion of this issue on github I stumble on something on stackoverflow that works, despite being down voted there. I put this into Cargo.toml:

serde = { version = "1.0", features = ["derive"] } 
serde_json = "1.0.40" 

Fine, it works. But I don't understand what that does and I don't know if it's a good solution.

As far as I can make out there is some version conflict going on between what I want and what the nats crate I am using pulls in.

Any ideas?

[Yandros]

@ZiCog

I don't know if it's a good solution

According to serde's official documentation, that's the way of doing it:

@https://serde.rs/derive.html

Using derive

Serde provides a derive macro to generate implementations of the Serialize and Deserialize traits for data structures defined in your crate, allowing them to be represented conveniently in all of Serde's data formats.

You only need to set this up if your code is using #[derive(Serialize, Deserialize)] .

This functionality is based on Rust's #[derive] mechanism, just like what you would use to automatically derive implementations of the built-in Clone , Copy , Debug , or other traits. It is able to generate implementations for most structs and enums including ones with elaborate generic types or trait bounds. On rare occasions, for an especially convoluted type you may need to implement the traits manually.

These derives require a Rust compiler version 1.15 or newer.

• Add serde = { version = "1.0", features = ["derive"] } as a dependency in Cargo.toml.
• Ensure that all other Serde-based dependencies (for example serde_json) are on a version that is compatible with serde 1.0.
• On structs and enums that you want to serialize, import the derive macro as use serde::Serialize; within the same module and write #[derive(Serialize)] on the struct or enum.
• Similarly import use serde::Deserialize; and write #[derive(Deserialize)] on structs and enums that you want to deserialize.

Here is the Cargo.toml :

[package]
name = "my-crate"
version = "0.1.0"
authors = ["Me <user@rust-lang.org>"]

[dependencies]
serde = { version = "1.0", features = ["derive"] }

# serde_json is just for the example, not required in general
serde_json = "1.0"

Now the src/main.rs which uses Serde's custom derives:

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, Debug)]
struct Point {
    x: i32,
    y: i32,
}

fn main() {
    let point = Point { x: 1, y: 2 };

    let serialized = serde_json::to_string(&point).unwrap();
    println!("serialized = {}", serialized);

    let deserialized: Point = serde_json::from_str(&serialized).unwrap();
    println!("deserialized = {:?}", deserialized);
}

Here is the output:

$ cargo run
serialized = {"x":1,"y":2}
deserialized = Point { x: 1, y: 2 }

@ZiCog

I don't understand what that does

::serde is a serialization framework, meaning that it aims to be as general as possible. We could say that it acts as a proxy / middle-man between:

• a "user", i.e., a person that is defining data structures that they want to be [de]serialized.

  • That's you

  • ⇒ [De]Serialize traits

• a "serialization format" (a library defining how to [de]serialize "root elements" and how to [de]combine serialized elements).

  • One such format is JSON (available in ::serde_json)

  • ⇒ [De]Serializer traits

---

Now we should be able to answer your three-ish (inferred) questions:

1. Why do I need to depend on serde?

   • Because you want to make you own custom structs [de]serializable,
     i.e., you want them to be impl [De]Serialize

2. Ok then, why do I need serde_json?

   • Because you seem to need to use JSON as the [de]serialization format.

3. What about that features = ["derive"] thingy?

   • You tell the framework (serde) how to [de]serialize your structs by implementing the [De]Serialize traits.

     Example

     Imagine you have

     struct Person {
         age: u8,
         name: String,
     }
     

     and you want to be able to tell the framework how to serialize Person by delegating to serializing its basic elements (struct name, and each one of its fields):

     use ::serde::{
         Serialize, // general serialization logic
         Serializer, // abstraction over specific serialization implementations / formats
     };
     
     impl Serialize for Person {
         // Imagine the SerializationFormat to be JSON
         fn serialize<SerializationFormat : Serializer> (
             self: &'_ Person,
             serialization_format: SerializationFormat,
         ) -> Result<SerializationFormat::Ok, SerializationFormat::Err>
         {
             let mut state =
                 serialization_format
                     .serialize_struct(
                         "Person", // struct name
                         2, // number of fields to serialize
                     )?
             ;
             state
                 .serialize_field("age", &self.age)?
             ;
             state
                 .serialize_field("name", &self.name)?
             ;
             state.end()
         }
     }
     

     This is quite cumbersome for something as "dumb" as just calling .serialize_field for each one of the struct fields. And worst of all, it has to be kept in sync with the struct definition!

     That's where #[derive(Serialize)] shines, since it will automagically write down all this code for you. But since such tool needs to be compiled to be available, and since that leads to longer compilation times, serde's author has chosen to make that tool unavailable unless explicitely told to, which is what the features line in the Cargo.toml does.

[ZiCog]

Yandros

Thanks for your comprehensive explanation. It's all making sense.

Your answers 1) and 2) were clear enough from the outset. I forgot to mention that I already had "#[derive(Serialize, Deserialize)]" on my structs as per the serde_json documentation.

My difficulty was finding the actual incantation in the build system to make it work!

Compounded by the fact that your suggestion was made on stackoverflow and voted down:
https://stackoverflow.com/questions/32077194/cant-export-and-use-the-trait-deserialize-from-serde
(Could y'all go and vote that up a bit)

It would have helped if the little 'features = ["derive"]' in Cargo.toml thing was mentioned on the front page of the serde_json documentation page serde_json - Rust or the front page of it's github repo: GitHub - serde-rs/json: Strongly typed JSON library for Rust

I guess after a little more soaking in Rust/Cargo lore these things will become obvious.

To be honest, I'm not used to programming in a world where macros/templates/metaprogramming writes a lot of code for you. Something I will have to get used to.

[Yandros]

@ZiCog

Compounded by the fact that your suggestion was made on stackoverflow and voted down

Looking at that question, it seems that the OP was looking for the Deserialize trait, whereas your "issue" was related to the Deserialize derive (same name because the latter generates the impl of the former; ditto for Serialize).

[ZiCog]

Sorry if my opening question is not clear. As a Rust newbie perhaps I did not know exactly what I was looking for or how to ask about it.

What I actually wanted to do was write this:

let json = serde_json::to_string(&message).unwrap();

Where message is any one of half a dozen different structs wrapped in an enum having been created from parsing stream of data in a proprietary binary format.

Google finds me the serde_json documenation serde_json - Rust so I follow the example under "Creating JSON by serializing data structures". Similar to what I see on the serde_json github page GitHub - serde-rs/json: Strongly typed JSON library for Rust.

That fails to compile with the errors noted in my first post so it's back to google where I don't find a good answer, except the one on stack overflow that was voted down!

That works, but I'm not sure if it's the best way to do it, in view of that down vote, so I end up here.

All I needed was that little 'features = ["derive"]' thing in Cargo.toml. The last piece in getting what is only my second non-trivial Rust program working.

Thanks everyone. It's working beautifully now.

By the way, this has been an interesting exercise. This code is a rewrite of a couple of thousand lines of C++ which I was never very happy with. Which in turn was based on an example parser code written in C#. Reimplementing C++ like this is probably not the best way to learn a new language but it's a job I needed to get working. It went surprisingly easily, a three day discussion with the compiler and borrow checker! It's so much nicer to converse with the error messages of Rust than anything that comes out of C++, more like iterating on the Javascript "hack - run" dev loop but without so much of the "run" part. It showed up some potential problems in the C++ version. It made it pretty easy to refactor things and be confident that did not break anything. Niko Matsakis' phrase "Hack Without Fear!" is true: C++Now 2017: Niko Matsakis "Rust: Hack Without Fear!" - YouTube