Reading from fd into uninitialized memory

I'm trying to understand how to read from a file descriptor without causing undefined behavior, with unsafe if needed.

The function I use to read from a fd is nix's read: read in nix::unistd - Rust

This function takes a &mut [u8] argument as the buffer.

In my application, I don't want to allocate a new buffer every time, so I'm maintaining a Vec<u8> that I can grow when needed. It's never shrunk.

Before reading, I do buf.reserve(N); buf.set_len(old_size + N);. However, a recently clippy lint started to complain about this code saying that it's probably undefined behavior: Clippy Lints

As alternative it suggests Vec::spare_capacity_mut, but as far as I understand even that will not be UB-safe, because I still need to convert &mut [MaybeUninit<u8>] to &mut [u8] without initializing the memory, which according to MaybeUninit::slice_assume_init_mut is not safe unless I actually initialize the memory.

So I'm confused about how read from a fd without initializing the buffer memory.

Any tips?

Thanks.

If you make the call using libc::read, then it takes a raw pointer and length, so it is callable without going through &mut [u8].

Besides that, there are many crates that ignore it and convert it to an &mut [u8] anyway.

So what you suggest is, instead of

let old_len = buf.len();
buf.reserve(N);
buf.set_len(old_len + N);
let bytes_read = nix::unistd::read(fd, &mut buf[old_len..]).unwrap();
buf.set_len(old_len + bytes_read);

I do

let old_len = buf.len();
buf.reserve(N);
let read_ptr = buf.as_mut_ptr().add(old_len);
let bytes_read = libc::read(fd, read_ptr, N);
assert!(bytes_read <= N);
buf.set_len(old_len + bytes_read);

Did I get this right?

You need to insert an error check, but otherwise yes.

1 Like

Strictly speaking, it's not possible. The function should have taken &mut [MaybeUninit<u8>] or something else like &mut Vec<u8>.

I don't need to use nix::unistd::read, I can use libc::read, or some other library function.

You could use FromRawFd and read_to_end.

1 Like

FWIW, this is what the

crate is for:

use ::uninit::prelude::*;

fn read (
    fd: RawFd,
    buf: Out<'_, [u8]>,
) -> ::nix::Result<usize>
{
    let len = buf.len();
    assert!(len <= isize::MAX as usize);
    let ptr = buf.as_mut_ptr();
    let count: isize = ::nix::errno::Errno::result(unsafe {
        ::libc::read(fd, ptr, len)
    })?;
    debug_assert!(count >= 0, "count overflowed isize");
    Ok(count as usize)
}

You'd then be able to both:

use ::uninit::prelude::*;

let mut buf = [0_u8; 256];
let count = read(fd, buf.as_out())?;
let read_bytes: &[u8] = &buf[.. count];

as well as:

use ::uninit::prelude::*;

let mut buf = uninit_array![u8; 256];
let count = read(fd, buf.as_out())?;
let buf: &[u8] = unsafe { buf[.. count].assume_init_by_ref() };

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.