Hello there,
I'm currently working on a project that uses Actix to retrieve JSONs from a server, processes them and POST them to another server. It works just fine if the number of JSON objects is small. However, if the response body is, say, around 20,000 lines, I get an Overflow Error from Actix. Is there a way to read lines from the body and process them asynchronously? i.e. parse them and send them to the api while getting the response. Thanks in advance.
I'm using Actix Client Crate for sending and receiving.
For reference, here's my code snippet
let client = Client::new();
// Getting payload from the server
let payload = client.request(Method::GET, uri)
.basic_auth(username,
password)
.query(params)
.unwrap()
.send()
.await
.unwrap()
.body()
.await
.unwrap();
let lines = std::str::from_utf8(&payload).unwrap()
.to_string();
// a new client to post to the api
let client = Client::new();
for line in lines.lines() {
// transform the source data to api format
let parsed = serde_json::from_str(line).unwrap();
let r = client.clone()
.post("my-api-uri")
.send_json(&parsed)
.await
.unwrap();