Nix: Unexpected behavior from AF_PACKET socket

I have a bit of C code that does what I expect: print the length of all incoming packets.

int main(void) {
    int sock;
    int read;
    char buf[1024] = {0};

    if ((sock = socket(AF_PACKET, SOCK_DGRAM, htons(ETH_P_ALL))) < 0) {
        fprintf(stderr, "Failed to create socket\n");
        exit(1);
    }

    while ((read = recvfrom(sock, buf, 1024, 0, NULL, NULL)) > 0) {
        printf("Read %d bytes\n", read);
    }

    return 0;
}

I also have this bit of Rust code, that I expect to result in the same output:

fn main() {
    let sock_fd = socket::socket(
        socket::AddressFamily::Packet,
        socket::SockType::Datagram,
        socket::SockFlag::empty(),
        socket::SockProtocol::EthAll,
    )
    .expect("failed to create socket");

    let mut buf = [0u8; 1024];

    loop {
        let (read, _) = socket::recvfrom::<()>(sock_fd, &mut buf).expect("failed to recv");

        if read == 0 {
            break;
        }

        println!("Read {read} bytes");
    }
}

However when I run the Rust code above, it just sits there and blocks.

Is there something I am doing wrong/missing? Really seems like it should result in the same behavior.

I'd appreciate any pointers in the right direction. If it's a bug in the Nix crate I'd also be glad to dig deeper into it. Thanks!

EDIT: I'll also note that I'm running both as with sudo while on WSL2.

Your code does not seem to match the code in the socket crate. A link to the crate you are using would be helpful.

I was confused too, but the crate name is in the post title. I think it's this function

1 Like

The unit caught my eye.

The relevant code from nix...

let mut addr = mem::MaybeUninit::<T>::uninit();
addr.as_mut_ptr() as *mut libc::sockaddr,
&mut len as *mut socklen_t,

When T is () (unit) does addr.as_mut_ptr() return a null pointer?

When len is zero (instead of null) does recvfrom behave correctly?

Those are the only differences I can find.

(I've got to get to bed otherwise I'd test it.)

Yeah, sorry I should have been more clear that I'm using the nix crate but looks like you guys figured it out.

But interestingly, if you get rid of the NULL pointers and just call recv, it works in C, but again, not in Rust with the Nix crate.

C vs Rust plain recv calls

C that works:

read = recv(sock, buf, 1024, 0);

Rust equivalent:

 let read = socket::recv(sock_fd, &mut buf, MsgFlags::empty()).expect("failed to recv");

The nix::socket::recv has an even simpler implementation than nix::socket::recvfrom that really just calls the libc binding:

pub fn recv(sockfd: RawFd, buf: &mut [u8], flags: MsgFlags) -> Result<usize> {
    unsafe {
        let ret = libc::recv(
            sockfd,
            buf.as_ptr() as *mut c_void,
            buf.len() as size_t,
            flags.bits(),
        );

        Errno::result(ret).map(|r| r as usize)
    }
}

Which seems to be about as direct to the C code as you can get but that somehow still doesn't work. Pretty stumped.

Kinda leads me to suspect that the issue is with the socket and not the recv calls

Does strace run under WSL2? If so, you could use that to verify that the same (Linux) syscalls are being generated by the C code and the Rust code, or see what the difference is if not.

They seem to be using the same syscalls, just the Rust one blocks at the recv. In both cases recv gets translated to a recvfrom call with NULL as the last 2 args which is expected.

Used tee for the outputs and snipped some of the irrelevant bits.

Rust strace
socket(AF_PACKET, SOCK_DGRAM, htons(0 /* ETH_P_??? */)) = 3
recvfrom(3, 

Not even one full recv...

C strace

With GCC v 9.4.0:

socket(AF_PACKET, SOCK_DGRAM, htons(ETH_P_ALL)) = 3
recvfrom(3, "E\0\0\317N\315@\0@\6\355Y\177\0\0\1\177\0\0\1\221\207\277\362\251\276\314\305\366n\"\331"..., 1024, 0, NULL, NULL) = 207
fstat(1, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
brk(NULL)                               = 0x55a7b9b41000
brk(0x55a7b9b62000)                     = 0x55a7b9b62000
recvfrom(3, "E\0\0\317N\315@\0@\6\355Y\177\0\0\1\177\0\0\1\221\207\277\362\251\276\314\305\366n\"\331"..., 1024, 0, NULL, NULL) = 207
recvfrom(3, "E\0\0004\371w@\0@\6CJ\177\0\0\1\177\0\0\1\277\374\221\207L\316\256V\334\350\327A"..., 1024, 0, NULL, NULL) = 52
recvfrom(3, 

Clang 15:

socket(AF_PACKET, SOCK_DGRAM, htons(ETH_P_ALL)) = 3
recvfrom(3, "E\0\0\321P\352@\0@\6\353:\177\0\0\1\177\0\0\1\221\207\277\362\251\347l\375\366n\330\327"..., 1024, 0, NULL, NULL) = 209
fstat(1, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
brk(NULL)                               = 0x55a12203d000
brk(0x55a12205e000)                     = 0x55a12205e000
recvfrom(3, "E\0\0\321P\352@\0@\6\353:\177\0\0\1\177\0\0\1\221\207\277\362\251\347l\375\366n\330\327"..., 1024, 0, NULL, NULL) = 209
recvfrom(3, "E\0\0\314\375\334@\0@\6>M\177\0\0\1\177\0\0\1\277\374\221\207L\322\32:\334\351\354\264"..., 1024, 0, NULL, NULL) = 204
recvfrom(3, 

I supposed it could be the fstat and brk but in both cases in C, there's a successful recvfrom before those happen.

The ETH_P_ALL option is different in the rust one isn't it?

1 Like

Wow yeah it is, not sure how I missed that.

Strange though since the nix definition is EthAll = libc::ETH_P_ALL.to_be() and the lib definition is ETH_P_ALL: ::c_int = 0x0003

I believe the problem is that the nix definition elaborates as (on a target where c_int is 32 bits)

enum SockProtocol {
    // ...
    EthAll = i32::to_be(libc::ETH_P_ALL),
}

But the canonical C usage is htons(ETH_P_ALL), which in Rust terms is u16::to_be(libc::ETH_P_ALL as u16) as i32 (again, assuming 32-bit c_int). So if you're on a little-endian system, the C code ends up passing 0x0300 as the last argument of socket(2), while the Rust code passes 0x03000000. (I don't know exactly why that shows up as just 0 in the strace, probably it gets truncated to 16 bits somewhere.) This is ultimately a bug in nix -- there's an existing PR to fix it:

2 Likes

Ah thanks for the explanation, makes sense. Welp guess its just a waiting game now.

I'd suggest submitting your own PR if you don't want to wait for an indefinite amount of time on the original author.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.