Begginer trying to learn to parse serde_json::Value

help
1

I am a Rust beginner trying to learn Reqwest. My background is in Python and I have used its Requests library. I am trying to make a simple call to an API endpoint. Here is what I have so far:

main.rs

use std::collections::HashMap;
use std::env;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let url = concat!(
        "https://api.pinboard.in/v1/posts/get?auth_token=",
        env!("PINBOARD_TOKEN"),
        "&format=json",
    );
    let resp = reqwest::blocking::get(url)?
        .json::<HashMap<String, serde_json::Value>>()?;
    println!("{:#?}", resp);
    Ok(())
}

cargo.toml

[package]
name = "reqwest"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
reqwest = { version = "0.11", features = ["blocking", "json"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1.0"

This works, but the outpu is not ideal since the HashMap values are all of the Value enum type:
stdout

{
    "posts": Array [
        Object {
            "description": String("How to File Taxes for Free Without TurboTax — ProPublica"),
            "extended": String(""),
            "hash": String("52259488b0fb5195707da9950c80fb21"),
            "href": String("https://www.propublica.org/article/how-to-file-taxes-for-free-without-turbotax"),
            "meta": String("c657c53004e8650fedcca1fee492ab19"),
            "shared": String("yes"),
            "tags": String("taxes irs article 2023 reference free"),
            "time": String("2023-03-03T16:13:52Z"),
            "toread": String("no"),
        },
        Object {
            "description": String("The Checkered Past of the Contractor Monitoring the Air in East Palestine - The American Prospect"),
            "extended": String(""),
            "hash": String("8f549c7c8f7868a6d852c4841b2459f1"),
            "href": String("https://prospect.org/environment/2023-03-03-cteh-contractor-air-monitoring-east-palestine/"),
            "meta": String("38545640684f00cbaac06dc9badfaaa8"),
            "shared": String("yes"),
            "tags": String("environment pollution article 2023 trains"),
            "time": String("2023-03-03T16:14:25Z"),
            "toread": String("no"),
        },
        Object {
            "description": String("What Will Happen To Everyone Who is Not White, Straight, & Male If We Don't Speak Out?"),
            "extended": String(""),
            "hash": String("cb222c1535830c022c94e19937cbfdf2"),
            "href": String("https://hartmannreport.com/p/what-will-happen-to-everyone-who"),
            "meta": String("2cbb2c12d4586ba3c3b9b114d380e9c0"),
            "shared": String("yes"),
            "tags": String("blog article 2023 US government Politics Republicans"),
            "time": String("2023-03-03T20:38:22Z"),
            "toread": String("no"),
        },
        Object {
            "description": String("A letter to supporters of Israel: It is time to break your silence – Mondoweiss"),
            "extended": String(""),
            "hash": String("4c34bacbf46d26988ca519e4b9c58820"),
            "href": String("https://mondoweiss.net/2023/03/a-letter-to-supporters-of-israel-it-is-time-to-break-your-silence/"),
            "meta": String("8ef149a31e221d93e9d8a5c7839dc127"),
            "shared": String("yes"),
            "tags": String("Palestine Israel article 2023"),
            "time": String("2023-03-03T20:47:16Z"),
            "toread": String("no"),
        },
    ],
    "user": String("Steve_Z"),
    "date": String("2023-03-03T16:13:52Z"),
}

I can "solve" this by creating a struct Pinboard {} that conforms to the Response and using .json::<Pinboard>()?; but I am wondering if there is a way to directly parse the serde_json::Value into standard types:. String, etc.

I have tried serde_json::Value::into_string() and serde_json::Value::as_str() but they both raise compile errors like this:

steve@pop-os:~/Coding/rust_projects/reqwest$ cargo run
   Compiling reqwest v0.1.0 (/home/steve/Coding/rust_projects/reqwest)
error[E0214]: parenthesized type parameters may only be used with a `Fn` trait
  --> src/main.rs:11:52
   |
11 |         .json::<HashMap<String, serde_json::Value::into_string()>>()?;
   |                                                    ^^^^^^^^^^^^^ only `Fn` traits may use parentheses

error[E0599]: no variant named `into_string` found for enum `Value`
  --> src/main.rs:11:52
   |
11 |         .json::<HashMap<String, serde_json::Value::into_string()>>()?;
   |                                                    ^^^^^^^^^^^ variant not found in `Value`

Some errors have detailed explanations: E0214, E0599.
For more information about an error, try `rustc --explain E0214`.
error: could not compile `reqwest` due to 2 previous errors

I tried that based on this page: Value in serde_json - Rust > to_string

Do I need to implement the ToString trait for Value for this to work? Am I missing something (probably)? Should I just stick with using struct Pinboard?

2

I not sure if I understand, you are explicit asking to receive a HashMap of a generic json value.

If you want to have the json parsing defined on compile time (without manually matching the json:Value) you need to define your structure some how, using existing or new structs. On your case, why did you don't use :

HashMap<String, Vec<HashMap<String, String>>>

for instance

    let s = r#"{
        "posts": [
            {
                "a": "a",
                "b": "b",
                "c": "c"
            }
        ]
    }"#;

    println!(
        "{:?}",
        serde_json::from_str::<HashMap<String, Vec<HashMap<String, String>>>>(s).unwrap()
    );

prints

{"posts": [{"a": "a", "b": "b", "c": "c"}]}
2 Likes
3

Thanks. I tried what you suggested:
.json::<HashMap<String, Vec<HashMap<String, String>>>>()?;
but got this error Error: reqwest::Error { kind: Decode, source: Error("invalid type: string \"2023-03-03T16:13:52Z\", expected a sequence", line: 1, column: 30) }

My question is whether there is a direct way to parse Value enum types into their values, e.g.:
"user": String("Steve_Z") to "user": "Steve_Z"

4

This is the intended and efficient way to do it. Value is a last resort when you don't know a schema for the data; it is necessarily more awkward to work with because it has no static type information.

6 Likes
5

Well, obviously, you have to match the static type with the structure of the JSON you are parsing. The Vec<HashMap<String, String>> was just an example. If you have a map from strings to Persons, then deserialize into a HashMap<String, Person> instead.

1 Like
6

Since your data is heterogeneous, you will probably want to use derive. Using derive · Serde will give you a start, but there's a lot of flags and attributes that you can customize.

Here it would look a bit like:

#[derive(Deserialize)]
struct Response {
    user: String,
    date: String,
    posts: Vec<Post>,
}

// repeat for Post
7

Thanks to all for your replies.

@kpreid That makes sense.
@simonbuchan That is similar to what I arrived at in my "solved" version -- using struct Pinboard.