I have a server that hosts a database. This database contains one billion u64 type records. The server is implementing gRPC protocol using Rust's Tonic library. I've initiated a client to connect to this server and send a query request to fetch all one billion records. This query request returns a Tonic Streaming object. However, when I call try_collect on the Streaming object, I encounter an error: 'Cannot return body with more than 4GB of data but got xxx bytes.' It seems like Tonic is executing try_collect on the server side and then attempting to fit the collected data into a single message for transmission, which triggers this error. I'm wondering why try_collect isn't executed on the client side instead. This query request invokes the DoGet interface of Arrow Flight.
This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.