Trivial rust binaries no longer run with RLIMIT_NOFILE=0 after rust 1.48

edre · April 26, 2022, 4:37pm

I'm having issues getting a static binary out of the x86_64-unknown-linux-musl target.

I had an old docker image based off of rust:1.31 that produced static binaries, and I tried updating it to 1.60 and they are no longer static. I bisected this to first failing in rust:1.46 (released in Aug 2020). I haven't found any references to this problem online in the 2 years since then so maybe this is something weird in the docker environment.

Here's my reproduction Dockerfile. Run with docker build .:

FROM rust:1.46
RUN rustup target add x86_64-unknown-linux-musl
RUN echo 'fn main(){println!("hi");}' > hi.rs
RUN rustc --target x86_64-unknown-linux-musl -o hi hi.rs
RUN file hi

Prints dynamically linked in 1.45 and earlier, and statically linked in 1.46 and later.

wesleywiser · April 26, 2022, 4:47pm

I believe file will report "dynamically linked" for any Position Independent Executable whether statically or dynamically linked and Rust 1.46 enabled PIE by default for x86_64-unknown-linux-musl.

cuviper · April 26, 2022, 5:18pm

Using readelf -hd may be more informative for the header and dynamic section.

edre · April 26, 2022, 5:44pm

Interesting. It seems I misattributed my application error to the linking status of the file. The actual regression was with rust:1.48, and trivial rust binaries after this point fail the NOFILE ulimit.

FROM rust:1.48
RUN rustup target add x86_64-unknown-linux-musl
RUN echo 'fn main(){println!("hi");}' > hi.rs; rustc --target x86_64-unknown-linux-musl -o hi hi.rs
RUN python3 -c 'import resource,subprocess;print(subprocess.run(["/hi"],preexec_fn=lambda: resource.setrlimit(resource.RLIMIT_NOFILE, (0,0))))'

I ran strace on the binary (without the ulimit) and the output looks identical to me. I don't see any open calls.

strace 1.47 (which passes the above test)

execve("./hi", ["./hi"], 0x7ffecddf67a0 /* 8 vars */) = 0
mmap(NULL, 496, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4dff07c000
arch_prctl(ARCH_SET_FS, 0x7f4dff07c100) = 0
set_tid_address(0x7f4dff081d18)         = 12
poll([{fd=0, events=0}, {fd=1, events=0}, {fd=2, events=0}], 3, 0) = 0 (Timeout)
rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f4dff0614be}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGSEGV, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RT_1 RT_2], NULL, 8) = 0
rt_sigaction(SIGSEGV, {sa_handler=0x7f4dff03f6c0, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_SIGINFO, sa_restorer=0x7f4dff0614be}, NULL, 8) = 0
rt_sigaction(SIGBUS, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGBUS, {sa_handler=0x7f4dff03f6c0, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_SIGINFO, sa_restorer=0x7f4dff0614be}, NULL, 8) = 0
sigaltstack(NULL, {ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}) = 0
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4dff022000
mprotect(0x7f4dff022000, 4096, PROT_NONE) = 0
sigaltstack({ss_sp=0x7f4dff023000, ss_flags=0, ss_size=8192}, NULL) = 0
brk(NULL)                               = 0x555556e89000
brk(0x555556e8a000)                     = 0x555556e8a000
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1 RT_2], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
write(1, "hi\n", 3hi
)                     = 3
sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=8192}, NULL) = 0
munmap(0x7f4dff022000, 12288)           = 0
exit_group(0)                           = ?

strace on 1.48 (which fails the above check):

execve("./hi", ["./hi"], 0x7ffecddf67a0 /* 8 vars */) = 0
mmap(NULL, 496, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4dff07c000
arch_prctl(ARCH_SET_FS, 0x7f4dff07c100) = 0
set_tid_address(0x7f4dff081d18)         = 12
poll([{fd=0, events=0}, {fd=1, events=0}, {fd=2, events=0}], 3, 0) = 0 (Timeout)
rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f4dff0614be}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGSEGV, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RT_1 RT_2], NULL, 8) = 0
rt_sigaction(SIGSEGV, {sa_handler=0x7f4dff03f6c0, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_SIGINFO, sa_restorer=0x7f4dff0614be}, NULL, 8) = 0
rt_sigaction(SIGBUS, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGBUS, {sa_handler=0x7f4dff03f6c0, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_SIGINFO, sa_restorer=0x7f4dff0614be}, NULL, 8) = 0
sigaltstack(NULL, {ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}) = 0
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4dff022000
mprotect(0x7f4dff022000, 4096, PROT_NONE) = 0
sigaltstack({ss_sp=0x7f4dff023000, ss_flags=0, ss_size=8192}, NULL) = 0
brk(NULL)                               = 0x555556e89000
brk(0x555556e8a000)                     = 0x555556e8a000
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1 RT_2], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
write(1, "hi\n", 3hi
)                     = 3
sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=8192}, NULL) = 0
munmap(0x7f4dff022000, 12288)           = 0
exit_group(0)                           = ?

edre · April 26, 2022, 6:31pm

Ok the behavior is different when run with the ulimit set. 1.48 binary calls poll and gets EINVAL, then kills itself. 1.47 binary never calls poll and doesn't have problems. Without NOFILE=0, both binaries call poll and get 0.

strace of 1.47 binary with NOFILE=0

[pid    16] prlimit64(0, RLIMIT_NOFILE, NULL, {rlim_cur=0, rlim_max=0}) = 0
[pid    16] close(3)                    = -1 EBADF (Bad file descriptor)
[pid    16] execve("/hi", ["/hi"], 0xb29010 /* 9 vars */ <unfinished ...>
[pid    15] <... read resumed> "", 50000) = 0
[pid    15] close(3 <unfinished ...>
[pid    16] <... execve resumed> )      = 0
[pid    15] <... close resumed> )       = 0
[pid    16] mmap(NULL, 496, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb759953000
[pid    16] arch_prctl(ARCH_SET_FS, 0x7fb759953100) = 0
[pid    16] set_tid_address(0x7fb759958cf8 <unfinished ...>
[pid    15] wait4(16,  <unfinished ...>
[pid    16] <... set_tid_address resumed> ) = 16
[pid    16] rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7fb759938e47}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
[pid    16] rt_sigaction(SIGSEGV, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
[pid    16] rt_sigprocmask(SIG_UNBLOCK, [RT_1 RT_2], NULL, 8) = 0
[pid    16] rt_sigaction(SIGSEGV, {sa_handler=0x7fb759916d60, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_SIGINFO, sa_restorer=0x7fb759938e47}, NULL, 8) = 0
[pid    16] rt_sigaction(SIGBUS, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
[pid    16] rt_sigaction(SIGBUS, {sa_handler=0x7fb759916d60, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_SIGINFO, sa_restorer=0x7fb759938e47}, NULL, 8) = 0
[pid    16] sigaltstack(NULL, {ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}) = 0
[pid    16] mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb7598f8000
[pid    16] mprotect(0x7fb7598f8000, 4096, PROT_NONE) = 0
[pid    16] sigaltstack({ss_sp=0x7fb7598f9000, ss_flags=0, ss_size=8192}, NULL) = 0
[pid    16] brk(NULL)                   = 0x555556e5b000
[pid    16] brk(0x555556e5c000)         = 0x555556e5c000
[pid    16] rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1 RT_2], [], 8) = 0
[pid    16] rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid    16] write(1, "hi\n", 3hi
)         = 3
[pid    16] sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=8192}, NULL) = 0
[pid    16] munmap(0x7fb7598f8000, 12288) = 0
[pid    16] exit_group(0)               = ?
[pid    16] +++ exited with 0 +++

strace of 1.48 binary with NOFILE=0

[pid    16] prlimit64(0, RLIMIT_NOFILE, NULL, {rlim_cur=0, rlim_max=0}) = 0
[pid    16] close(3)                    = -1 EBADF (Bad file descriptor)
[pid    16] execve("/hi", ["/hi"], 0xeac010 /* 9 vars */ <unfinished ...>
[pid    15] <... read resumed> "", 50000) = 0
[pid    15] close(3 <unfinished ...>
[pid    16] <... execve resumed> )      = 0
[pid    15] <... close resumed> )       = 0
[pid    16] mmap(NULL, 496, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ffaf8ee2000
[pid    15] wait4(16,  <unfinished ...>
[pid    16] arch_prctl(ARCH_SET_FS, 0x7ffaf8ee2100) = 0
[pid    16] set_tid_address(0x7ffaf8ee7d18) = 16
[pid    16] poll([{fd=0, events=0}, {fd=1, events=0}, {fd=2, events=0}], 3, 0) = -1 EINVAL (Invalid argument)
[pid    16] rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1 RT_2], [], 8) = 0
[pid    16] tkill(16, SIGABRT)          = 0
[pid    16] rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid    16] --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=16, si_uid=0} ---
[pid    16] +++ killed by SIGABRT (core dumped) +++

edre · April 26, 2022, 6:37pm

Renamed the thread. This has never been a guarantee of rustc that I know of so I don't expect this to be high priority.

cuviper · April 26, 2022, 6:50pm

That poll is coming from here:

github.com/rust-lang/rust

Reopen standard file descriptors when they are missing on Unix

rust-lang:master ← tmiasko:fds

opened 02:58PM - 08 Aug 20 UTC

tmiasko

+82 -2

The syscalls returning a new file descriptors generally return lowest-numbered …file descriptor not currently opened, without any exceptions for those corresponding to stdin, sdout, or stderr. Previously when any of standard file descriptors has been closed before starting the application, operations on std::io::{stderr,stdin,stdout} were likely to either succeed while being performed on unrelated file descriptor, or fail with EBADF which is silently ignored. Avoid the issue by using /dev/null as a replacement when the standard file descriptors are missing. The implementation is based on the one found in musl. It was selected among a few others on the basis of the lowest overhead in the case when all descriptors are already present (measured on GNU/Linux). Closes #57728. Closes #46981. Closes #60447. Benefits: * Makes applications robust in the absence of standard file descriptors. * Upholds IntoRawFd / FromRawFd safety contract (which was broken previously). Drawbacks: * Additional syscall during startup. * The standard descriptors might have been closed intentionally. * Requires /dev/null. Alternatives: * Check if stdin, stdout, stderr are opened and provide no-op substitutes in std::io::{stdin,stdout,stderr} without reopening them directly. * Leave the status quo, expect robust applications to reopen them manually.

quinedot · May 2, 2022, 1:13am

Submitted #96621.

system · July 31, 2022, 1:13am

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.

Topic		Replies	Views
Building a statically linked binary in Alpine Linux help	3	2145	November 27, 2020
Rust created binary dependencies help	6	606	May 12, 2020
Release binary not working in docker	6	2832	April 2, 2020
Running rust as a service	4	254	March 15, 2024
Cross-compiling rust for x86_64-unknown-linux-musl target problems help	4	831	January 12, 2023

Trivial rust binaries no longer run with RLIMIT_NOFILE=0 after rust 1.48

Related Topics