Reading more than 4096 bytes from a UnixStream

Any attempt of reading bytes from a UnixStream with any of read_to_end or read (and friends) only reads 4096 bytes.
Why is that? And how can more data be read with core Rust?

Please do not recommend the use of other crates, like tokio.

Thanks for your help.


To reproduce (on a unix system), clone the unix_sockets_basics, launch the server with

cargo run --bin server

and connect to it with netcat

nc -U mysocket

Pasting data and disconnecting, for example with data taken from

tr -cd '[:alnum:]' < /dev/urandom | head -c5000

results in only 4096 bytes being received and printed.

1 Like

That's a very interesting result. I don't have a nc that can talk to unix pipes, but I get the same result when using socat - UNIX-CONNECT:./mysocket. Additionally, your server.rs program looks correct, and if you use your client.rs program to send the data, it is sent properly:

// This works fine.
fn write_request_and_shutdown(unix_stream: &mut UnixStream) -> anyhow::Result<()> {
    unix_stream
        .write_all(b"Q6M22RYUELRzD160g3wQlKx8NKfZCxMvhnZz2sYYhdvlTw8RGkpuMRp5IQ5PREheh5FJQurrATCCOPBsqm61AthUmDRkFRQmhs0phFtwLCBUatYpe1yzGvsD4AN8delo8xdHpXLZYN3WNtNdq4Od5huDXpJNjcHuZLVcIEX2K6Q2sE9zWKQG0UFBQOpKOZ16JsUmhCnqUlpVp9MVS0cyYqmZgq9I9IM8vGQGdf0rfHpa6WZTkD4d4jRZKkyma4kLz7GJzarRlGUDuo7xtg7hSe2uuaWGMXFL29O4eezjLmN5dfUer61e4Dy5Se1cUUGGE23tAvJa1fTFDZccTwYUqoBDpxROGYL20Uhn4uZF7DVhypUmRpGsl6zz392T2FJaSrgNU3QfdF3VAtdSqNeS3YkrziPpNV0TS1VVLZb7Hly5L1WkfOaAOzwKt4vJFEpu9Ts1JCU0NYCWrncxcNhYmyT2TFrdDbk61SsVCBz4gi2JPMRBOHKLpgxYNEGmmSES0ykPDpDU0jmSbMa9dVS1JInRt8Q7BQruMpowkxkuBmhWuzwlZrwAB75uLschKVf14Nq6hm6Uyx6TxF9D0KLvEmAapyYOTkXUWQlzj4DBlsniJYz2YFdjMhattbYpAn6i3QKEcV82dXqSdGJldI9ETsPIa6hiBvDfwZQ726CL9UoNbZFB8K83QZZhajvLva0h3pwKFrJLv9cVmWWXjaDCp6Qqwkd9xzminepC6MmuSNWK3LIrgRVB1G3tMIAVCfysHCz5utAJVepvQzgaIu2na2pXq9W7AVfOExSbdID8UbhPgXcjLokcKbZ0Aveuq9cG6Xh3vyuvuPYuhQ138LAbltMYn2QbZDYKGqy0ByAiR8FGJBItu9uPpy6LPa5coFdoKwvPdiA41GBxmGjnccQ4pbd1icOy3OHQCYoD9D92FqrX7hsj9rcVmqIxxpRCCTYFkXJ9Y4YhZcknSkfy4V6fn5fugAwxmRFnMD9gpw7y3vxDMrPnKNt4YOaDCk7zWzQH2dAQOuiGoJsWXiTcXaJE2mVT3JZhFhrUVqVvg3EyaraTKKhO7BtujLRbkFlgZpLTxWRai2PRKizoXAwup2ZdOMUximCejxDuMlosoajBLhXlgKXGcbt8J8HU6JpvnOGhvVJls2UpBN9lu6b6WYMia20FPJCNZbG1cDaX5DvptJDoOZqDTH3xabkx041QoSzXJdblLMqvy8PyJ36weHtIQOoMGeLeMEVfPMJM7KlTa5mSeZhkeODpeEUUDZF3eUAnnWPQIYOSXwsHG8GMURRMA4cp6UPTf8J3A5COUCGwmdiv5K7ZvE4sE09FyOFnobrsEWVaTcaqt6WucsSHornrYhlreDw2BEk8ylnn8iotMSI7SknKV0Oo9dWFJaYxb4pYrV82va7ifTSgxlscDF2zD30MBzMVDkZUs0p8x2vP4E6kee3pKLMIcVZj0bBu0oAxFe094LBbuzVjfNC5NIvZpIkb5gDz0k8eKVtXdP0csaWPQ8hGrgnFxxxKDLQ0b3XyNNMxNL7UqXrkvKWCqQKMabC332n3BWjsd1ij6pjm4WswubTDc1YRiAdfU7hFTPrEATiZhLLHRM9Qhv2HjAyDDzGviPZrAVlJ2PG5izYanmfD4b5eslVd1SzRhyNDx9mvpNjaucrL42mGYtzyM5ZBXEhYg4uxTCg6zSOxTS6zq7h0YvgVbMeJbndFm6nwU2cywWgO776T6GrOac2sDmXRZDkfbieIUoadITfUdQVojAHDGYvsUhPDKYiRX4k2bjvhBrLaYouIDvLcUo9XYUr8XLIaYnxmjPyeaB7JabbPpTcUgp7ezQgKTXgE5SUd0h4Hhsep4mOGYdLp910BspFVPlChO3lV0ZDkGMx62z0ginZ80jcI8XLJbffX8n99dJn0MeJJMgfFn3ndHtX4OwjfpAJ2fXXNVNmXTm4hjTb1rRKNFa4ueIbOBCAhV1MvxUA1gS1bUxRfm9tTfMlOcTIqbQwzLMb4ilWY7I8wDepfU3OdkOQWMKzdTt8pDshmNjrm7droqz2bsjJol92kD9pGxTeI79MXTH04c7o9BRRIsOF8wuG2WcagXP4Ylz7FaLK1WyfWHIwUSinhLwA9AgRnvHn70VQcaDPYWLsFdjTy5tDy7ZcpPRZ1IYn0mEoIDJf6e7Pv8YwHctA62iK3xudIGLXjZzq2IZouMbD42ZHgWUpIgGhx8YcsSpljzoJdkcJOIJvEhzA9l3SRgYv67Eg5RnriEZxbGhO8xTgCmgNBb2u7XC7QS1I2wN2YTsm08q9EXi8KUshUUwYtXUO0ifzTdi47S4BcvRS8imqRo9J2szBVpLXF4McjkP6sJ2uWqYdePXjJYB3owp8lp3Y09VBbXuotqWzCt7finY4h4Ppk5Z47XLEIVR0u3WbMJcxmGmkRiNlYLt0yN9tFBVErPvxNQ5m0xzg6AbGJEfAflgaQ2IxAcOZ4uUtn3o4sVTJ1Bu0zYTMV1kScZFdqr7hB5G1loi61FywJD3G2SRSvuAxvdMXKSJ5trnjzkt76eHIG02XHkG9V07pGY9HOzVTB0YYDLtkwvBNL4LNwUcMYoktfY7DXViUrTl8YeL4yTwd0jIJVklvZ3ytcVtzfKfeBDafpAdGirnWYu7VGS4nopd5xv8yCACQc8xq8rrbSBpidP59vBVqd9KkX6fS8EiEskaazU5MoTLuF6UUfmnnV3ZrmLxA2LcpKrq2YXzNFyEiryxcTfseFwkgU0UVDnP18w3399LoFzHjSNkKptjZ6Y47IrcJ6oS8zVMbvYkNi2pnJFSrMMYfBdYVTKT97UDMWwknLe6AsIXXz7z5vUXbBVIMqBG2i2Re9VQnB5UJq4Gn8GhdSjEvMsTMjreSiuS0YjXEqTsJtPyd1NQNT9JsfzJItzKF3uRVaKjY2wJYvrk8KKrNfzcVxlihNAdbVtG09z3sh3bISUiXLRTIx2osRBgmLALDsQs6vmRr60YaaJ9UsacR2Mx9SB62glxvmSLh70R4iEJAMnXyLeEoCV2zUmLxgu2t4k3387EDqvaMvNqVe5dGZKAdpFhPHLZdHZZip4GxKeqa8fkbNzfOGghck2e7EfxiMZ1pNGyb0bcFGRXwlroHW9kAgTPZj7DIeXcA72nYv5XUh5x3Z8UqIR6F8bWPsfBCC5OQwwbBZsX1jFCBayHMxtLGLIZdtvvkcEqKEvNVS5HPrZKpiXTZghn50hcHCMNIQFqLKJaKhHcJLgmqeqBALAOSXDXyaYUPqEq1i7sxHQOCN2qkVQxYu11voQ0EEiQigYHOZEueBELazLwD3TRKHiBaetOiIjiqPk199uuV6Y8LXG1OEaJCwngr38CrxKLmL3aKFcSC4MwzJdSMAIxpCWn2cwPzYW2VUTgLo8krGL8pPWsQH8QADBTggKymyMawTm2FKL1Ac9IfVkmUjpCLChXwFoC7MBKDPW2iKdRu9agMsDxO4hY8khZ2TVTJlGkakNzTClZGmiBw0YjcF2ZobudFSH2YBJgpr5yMQF8yszuounZVNDx1ViLeVaUIsLRcF43rFf8UTJnv1nSOc4NEloxAZ6RcPVMdObcFCKYyRQFpgJTlLPBFrSVIexkialMtNQkTFMQI5DhvMvrm8TbDUmnoIX2V8a2mwvl8ssbGTiJstnUKoP9G0L31b7vczuDQdy4Djx8yCMODehhMZyXjlcWuE2dQHb4MTW8hFDD2ePAIGwx5qf6dbtg0mzRdsiqlE5zHnDYQglJ25IDOT7KZcdC17R89WTKtBRQHiVKQ98EjpvSdlmeh0PcxEyLPM5BnYHv4MOTTHKIESvOx8ooyV9L6MXlS5dXgI1UhuiDf36Tk7kNR4DmRl66dYfDeLTexy6H8PrcEYpF8khfkM8JdVjAXGabrGXSufZnt5Vi29qQOh4ysSB8t1mi9eZ8VNVsS8WKg39JVysMvvM7sDe3BXTD8DKA9ZCAUNNPloYYFUoJae6s4omI7UfWcc0SQpYOpYszwue3KjB5VM9CQKNIlMCHaQN7fNAFdFn7mlOakhdwnDhvwal9WkKdDgLTydL1TltYZgEdEpN8jqLcpfFsGW2j6Tl5rg1lm0vtSaVNwhgdjlpQiXE16wj2zd8o33A7Lop90r1N1SWFg7OZgifQJ5rV43dU7i9erKdsdneZYUdgjoUEoDlVr1pq6fiD2IeJZDdRuIJiwGFXkTrw3gOdMQ7SQPDPzZ6R8z1aIqKOXW5gYy7t2CyYO6mYVm8LkskKiS6xjlt2P01MqjMbm2O506BA92O5hxEGKhK7I8GuKhph0vDvj32CMdnl8tS8GBCzcJYBbj6vzIxm2o5VoQnpUJ1P4j4Yk0AVapJqW2G9etUNxkvaAIUYKKE1ege6lX2O5s6EnVWTAqev1VPSOrt3bA6kntIdkgKZQvobpM4nx3XPChNZtn5bxP2hlHzsqBBpAgP8XsUxNNOLb9ZSZT5ul2ukdjvYpmiB9lRelC2P9mdgGeDfqvGfClvGYhXO3n4gArdUnioYqN15H21iXkOI9lumAcpK68TRZ5IRLYpd9t6gfV0FmnWAA3E61179x4kAMaFfi74r7Mcn1bfPWUbAAPXw10cHvPV3MLShcPAEO8m4iLwPB3C6ezXrCom7yPXfjltqsPeuhwCkZy3FgsBO4FI8zj98QJRfaIdPjqEdV0ByEmD49NtB243JMAJ0IMw7egn3AEBVXQRctZH2nfZW3fuMbvvplsNovLLjQEm5nH1q3F9nUSJikEOJ2mHkXVTBPATIUgS4P8RCr6y3IW57Eyd89YKJq6uksLfakxhPr1ubWJ17lQ5R6IFRzCJdjDtD8MxIxgIXGL47cruBBw6yMJid9o6XxLPCDeUZZkVcbJqOsUxWDdZaYQzMMh3DtJcPC8ncnb4qzdfTWAg2hinOFtvln6N2XJsvzvnwsmxGIxyb6tqaE1Kj5saEaH9Gae89C7o9b2AeAYHYLndnlwaPtq3pb7aL3f")
        .context("Failed at writing onto the unix stream")?;

    println!("We sent a request");
    println!("Shutting down writing on the stream, waiting for response...");

    unix_stream
        .shutdown(std::net::Shutdown::Write)
        .context("Could not shutdown writing on the stream")?;

    Ok(())
}

Similarly, when I send more than 4096 bytes with socat, but introduce line breaks so that no line is longer than 4096 bytes, then it also works.

1 Like

I came across the termios(3) manpage, which says the following:

       * The maximum line length is 4096 chars (including the
         terminating newline character); lines longer than 4096 chars
         are truncated.  After 4095 characters, input processing (e.g.,
         ISIG and ECHO* processing) continues, but any input data after
         4095 characters up to (but not including) any terminating
         newline is discarded.  This ensures that the terminal can
         always receive more input until at least one line can be read.

I'm guessing that both nc and socat use these termios methods to read input from the terminal, causing the data to get truncated at 4096 before it is sent.

4 Likes

Thanks for your help. Indeed, I see that this is a problem in the way that I interact with the rust code.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.