What is the correct way to take serde_json into use?

I just wasted an hour trying to get serde_json to work in my program. Firstly I did the simple thing and followed the serde_json documentation: I put this into my module:

use serde::{Deserialize, Serialize};
use serde_json::Result;

and this my Cargo.toml

serde_json = "*"

Which results in errors: " unresolved import serde"

So I add serde to my Cargo.toml

serde = "*"

Which results in errors: " cannot find derive macro Serialize in this scope"

After an hour of googling and trying various suggestions that don't work and trying to understand the discussion of this issue on github I stumble on something on stackoverflow that works, despite being down voted there. I put this into Cargo.toml:

serde = { version = "1.0", features = ["derive"] } 
serde_json = "1.0.40" 

Fine, it works. But I don't understand what that does and I don't know if it's a good solution.

As far as I can make out there is some version conflict going on between what I want and what the nats crate I am using pulls in.

Any ideas?

According to serde's official documentation, that's the way of doing it:

::serde is a serialization framework, meaning that it aims to be as general as possible. We could say that it acts as a proxy / middle-man between:

  • a "user", i.e., a person that is defining data structures that they want to be [de]serialized.

    • That's you

    • [De]Serialize traits

  • a "serialization format" (a library defining how to [de]serialize "root elements" and how to [de]combine serialized elements).

    • One such format is JSON (available in ::serde_json)

    • [De]Serializer traits


Now we should be able to answer your three-ish (inferred) questions:

  1. Why do I need to depend on serde?

    • Because you want to make you own custom structs [de]serializable,
      i.e., you want them to be impl [De]Serialize
  2. Ok then, why do I need serde_json?

    • Because you seem to need to use JSON as the [de]serialization format.
  3. What about that features = ["derive"] thingy?

    • You tell the framework (serde) how to [de]serialize your structs by implementing the [De]Serialize traits.

      Example

      Imagine you have

      struct Person {
          age: u8,
          name: String,
      }
      

      and you want to be able to tell the framework how to serialize Person by delegating to serializing its basic elements (struct name, and each one of its fields):

      use ::serde::{
          Serialize, // general serialization logic
          Serializer, // abstraction over specific serialization implementations / formats
      };
      
      impl Serialize for Person {
          // Imagine the SerializationFormat to be JSON
          fn serialize<SerializationFormat : Serializer> (
              self: &'_ Person,
              serialization_format: SerializationFormat,
          ) -> Result<SerializationFormat::Ok, SerializationFormat::Err>
          {
              let mut state =
                  serialization_format
                      .serialize_struct(
                          "Person", // struct name
                          2, // number of fields to serialize
                      )?
              ;
              state
                  .serialize_field("age", &self.age)?
              ;
              state
                  .serialize_field("name", &self.name)?
              ;
              state.end()
          }
      }
      

      This is quite cumbersome for something as "dumb" as just calling .serialize_field for each one of the struct fields. And worst of all, it has to be kept in sync with the struct definition!

      That's where #[derive(Serialize)] shines, since it will automagically write down all this code for you. But since such tool needs to be compiled to be available, and since that leads to longer compilation times, serde's author has chosen to make that tool unavailable unless explicitely told to, which is what the features line in the Cargo.toml does.

3 Likes

Yandros

Thanks for your comprehensive explanation. It's all making sense.

Your answers 1) and 2) were clear enough from the outset. I forgot to mention that I already had "#[derive(Serialize, Deserialize)]" on my structs as per the serde_json documentation.

My difficulty was finding the actual incantation in the build system to make it work!

Compounded by the fact that your suggestion was made on stackoverflow and voted down:


(Could y'all go and vote that up a bit)

It would have helped if the little 'features = ["derive"]' in Cargo.toml thing was mentioned on the front page of the serde_json documentation page https://docs.serde.rs/serde_json/ or the front page of it's github repo: https://github.com/serde-rs/json

I guess after a little more soaking in Rust/Cargo lore these things will become obvious.

To be honest, I'm not used to programming in a world where macros/templates/metaprogramming writes a lot of code for you. Something I will have to get used to.

1 Like

Looking at that question, it seems that the OP was looking for the Deserialize trait, whereas your "issue" was related to the Deserialize derive (same name because the latter generates the impl of the former; ditto for Serialize).

1 Like

Sorry if my opening question is not clear. As a Rust newbie perhaps I did not know exactly what I was looking for or how to ask about it.

What I actually wanted to do was write this:

let json = serde_json::to_string(&message).unwrap();

Where message is any one of half a dozen different structs wrapped in an enum having been created from parsing stream of data in a proprietary binary format.

Google finds me the serde_json documenation https://docs.serde.rs/serde_json/ so I follow the example under "Creating JSON by serializing data structures". Similar to what I see on the serde_json github page https://github.com/serde-rs/json.

That fails to compile with the errors noted in my first post so it's back to google where I don't find a good answer, except the one on stack overflow that was voted down!

That works, but I'm not sure if it's the best way to do it, in view of that down vote, so I end up here.

All I needed was that little 'features = ["derive"]' thing in Cargo.toml. The last piece in getting what is only my second non-trivial Rust program working.

Thanks everyone. It's working beautifully now.

By the way, this has been an interesting exercise. This code is a rewrite of a couple of thousand lines of C++ which I was never very happy with. Which in turn was based on an example parser code written in C#. Reimplementing C++ like this is probably not the best way to learn a new language but it's a job I needed to get working. It went surprisingly easily, a three day discussion with the compiler and borrow checker! It's so much nicer to converse with the error messages of Rust than anything that comes out of C++, more like iterating on the Javascript "hack - run" dev loop but without so much of the "run" part. It showed up some potential problems in the C++ version. It made it pretty easy to refactor things and be confident that did not break anything. Niko Matsakis' phrase "Hack Without Fear!" is true: https://www.youtube.com/watch?time_continue=12&v=lO1z-7cuRYI

4 Likes