Best way to send Rust struct over Async TcpStream?

help
#1

currently I'm using bincode to serialize struct into Vec<u8> and first send its length as u64 and then data over Async TcpStream from tokio.

Server:-

use std::{convert::TryInto, net::SocketAddr};
use tokio::{io::AsyncWriteExt, net::{TcpListener, TcpStream}};
use serde::{Serialize,Deserialize};

#[derive(Serialize,Deserialize,Debug, Default)]
struct Msg {
    f1: String,
    f2: bool,
    f3: f64
}

#[tokio::main]
async fn main() {
    let listener = TcpListener::bind("127.0.0.1:61717").await.unwrap();
    loop {
        match listener.accept().await {
            Ok((socket, addr)) => {
                tokio::spawn( async move { process_socket(socket, addr).await } );
            },
            Err(e) => { 
                eprintln!("couldn't get client: {:?}", e);
                continue
            }
        }
    }
}

async fn process_socket(mut socket : TcpStream, addr : SocketAddr) {
    println!("New connection from {}",addr);

    let m2 = Msg {
        f1 : "rust is love".into(),
        f2 : true,
        f3 : 75240.5534
    };

    let data = bincode::serialize(&m2).unwrap();
    let len :u64 = data.len().try_into().unwrap();

    socket.write_u64(len).await.unwrap();
    socket.write(&data).await.unwrap();
}

and then on client I get data like this:-

let len = socket.read_u64().await.unwrap();
let mut data : Vec<u8> = Vec::with_capacity(len.try_into().unwrap());

socket.read_to_end(&mut data).await.unwrap();
let msg : Msg = bincode::deserialize(&data).unwrap();

But I think this is not the best approach as when I serialize Msg on server, I have 2 copies of same data one in form of Msg and another in form of Vec<u8>. I'm planning to send large size of data like png files and this would not be ideal use of RAM as far as I know.

There is method in bincode to serialize directly into writer but idk how use it. is there some way to directly serialize into Async TcpStream ? or any better aproach ?

#2

This is exactly the purpose of LengthDelimitedCodec. In a matter of fact, you don't have to manually send the length of the Vec<u8> through, because it automatically handles that for you. See: tokio_util::codec::length_delimited - Rust

Whenever you get a TcpStream, wrap it using the methods shown in the doc. You can split the Framed to obtain separate handles to the output/input sink/stream

1 Like
#3

Oh, that is nice. I didn't knew about that.

but is there any way to avoid two copies of same data while sending struct ?

#4

What do you mean? When you serialize a struct to a Vec<u8> then send it through the sink, the underlying codec reads the stored bytes from the heap and eventually communicates with the OS networking layer

#5

With sync code you have bincode::serialized_size and bincode::serialize_into. Not spent time to dig into async way or some link to convert. I would expect it has the down side of two passes (to first get the size;) so quite likely worse that vec/bytes.

1 Like
#6

There is no way to get bincode to output directly into an async TcpStream. Consider reusing the vector with the data?

1 Like