How to deserialize an array JSON into a Vec<struct> using reqwest and serd-json?

I have an API that response an array JSON in format:

[
    {
        "key1": "value1", // etc
    }
]

Now I use reqwest to get the API data and handle the response into a Vec<ResponseTags>, the struct will contains every key in the object of the object array. I encapsualate the reqwest with proxy settings in the client, it can get the response successfully.

How can I transform the JSON

[
    {
        "key1": "value1", // etc
    }
]

into Vec<ResponseTags> like

let tags: Vec<ResponseTags> = reqwest::get(url).await?.json().await?;// Something like the code to do deserialization and deseriliaze the object array JSON into this

pub struct ResponseTags {
    pub key1: String, //etc
}

?

I'm not sure I understand the question. Could you please elaborate what "get the JSON correctly into that" signifies to you? Into what exactly do you need to get the JSON into?

I just want to deserialize an object array JSON immediately into a Vec (or an array of ResponseTags) like that I use Newtonsoft.Json in C#, not clear how reqwest json() method works or serde-json handle the object arrays.

You mean how to set up the ResponseTag struct? You normally do this using the Serialize and/or Deserialize derive macros from serde like this

use serde::{Deserialize, Serialize};

#[derive(Deserialize, Serialize)]
struct ResponseTag {
    key1: String,
    // etc
}

to implement the Serialize and Deserialize traits for ResponseTag respectively. That is all you need to do in order to enable serde_json (which is used under the hood of reqwest) to deserialize some valid JSON object into a ResponseTag and vice versa.

I know that, I have set up the derive trait in the struct, but when I use reqwest::json<>() (actually I use the method like reqwest::json<Vec<ResponseTags>>() or reqwest::json<Hashmap<String, Vec<ResponseTags>>>()), the array object JSON can't be deserialized fully into the Vec<ResponseTags>.


In the code I captured in my post, the response_content variable has just the top object in the JSON not all the objects, so I can't get all the response data. How can I get all the data into this variable? Do I use reqwest in correct way? Or must do something with serde-json after reqwest?

serde_json is perfectly capable of reading a JSON array into a Vec and reqwest::Response::json reads the whole response body, so no need to do anything different on your end as far as I can tell. I assume the problem is that your model does not match the JSON you receive. If you could share the actual JSON that is problematic for you and your model (i.e. the types you parse your JSON into), we might be able to help you debug this.

Sure

#[derive(Deserialize, Debug)]
pub struct ResponseTags {
    pub id: i64,
    pub name: String,
    pub count: i64,
    #[serde(rename(deserialize = "type"))]
    pub tag_type: i64,
    pub ambiguous: bool,
}

The json is like (I get it use browser, so I'm sure it has more than one record):

[
    {"id":2061,"name":"name1","count":1069,"type":4,"ambiguous":false}, 
    {"id":149153,"name":"name2","count":859,"type":4,"ambiguous":false}
]

Actually I can get the id: 2061 object in the variable, but just this can be got, others are lost, not sure what happens.

Your model does match the JSON you are showing here. Are you sure this is what you receive though? Could you please parse the response body as text first and print that to make sure this is what you are actually receiving? I.e.:

let body = reqwest::get(url).await?.text().await?;

println!("{body:?}");

let tags: Vec<ResponseTags> = serde_json::from_str(&body)?;

Sure

I check it, it is correct

"[{\"id\":2061,\"name\":\"name1\",\"count\":1069,\"type\":4,\"ambiguous\":false},{\"id\":149153,\"name\":\"name2\",\"count\":859,\"type\":4,\"ambiguous\":false},{\"id\":237724,\"name\":\"name3\",\"count\":71,\"type\":4,\"ambiguous\":false},{\"id\":100876,\"name\":\"name4\",\"count\":21,\"type\":4,\"ambiguous\":false},{\"id\":136440,\"name\":\"name5\",\"count\":11,\"type\":1,\"ambiguous\":false},{\"id\":162524,\"name\":\"name6\",\"count\":7,\"type\":3,\"ambiguous\":false},{\"id\":165179,\"name\":\"name7\",\"count\":5,\"type\":4,\"ambiguous\":false},{\"id\":30948,\"name\":\"name8\",\"count\":3,\"type\":4,\"ambiguous\":false},{\"id\":144822,\"name\":\"name9\",\"count\":2,\"type\":4,\"ambiguous\":false}]\xab\xab\xab\xab\xab\xab\xab\xab\xab\xab\xab\xab\xab\xab\xab\xab\xfe"

Not sure where the ending characters are from


Correct, but only have the 2061 item, others are all lost after deserialized

The trailing bytes are likely the reason why your program isn't working, though I don't understand why parsing this doesn't fail, this is not valid JSON.

Yes, but I don't even know where are the trailing bytes from, and if it is unvalid JSON, it can't be deserialized but it still deserialize the top one.

And I have written a config.json for some configurations and it works fine.

Are you sure about this? You shared a screenshot from your editor which supposedly shows the behaviour you are describing? Could you please validate that your program does not work correctly by properly executing it, looking at its output on your terminal?

Checked, thanks for helping, I tried the code in main.rs with println and it is correct, so they act differently in debug mode?

No, I believe your editor does not show the data correctly.

2 Likes

I'm using VSCode and CodeLLDB in debugging mode, don't even have idea that it shows data wrong. :frowning:

Thanks a lot for helping me so long time!

2 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.