How to use Dataflow sanitizer?

help
1

Hi. I'm trying to instrument a program using LLVM/Clang's Dataflow sanitizer, but have not been successful.

Here is my setup, which is inspired by the test cases in the compiler's repo:

  1. A crate with main.rs defined as in tests/ui/sanitizer/dataflow.rs.
  2. A abilist.txt defined as in tests/ui/sanitizer/dataflow-abilist.txt
  3. RUSTFLAGS set to -Zsanitizer=dataflow -Zsanitizer-dataflow=abilist.txt -Cunsafe-allow-abi-mismatch=sanitizer.
  4. Building with cargo build -Zbuild-std --target x86_64-unknown-linux-gnu

Then the build fails in linking due to undefined symbols, which should be instrumented version of the stdlib.

  = note: rust-lld: error: undefined symbol: _Unwind_RaiseException.dfsan
          >>> referenced by gcc.rs:72 (src/gcc.rs:72)
          >>>               panic_unwind-08cd9740da1a8428.panic_unwind.f5942db226e6de1f-cgu.0.rcgu.o:(panic_unwind::imp::panic (.dfsan)) in archive ~/dfsan_test/dummy/target/x86_64-unknown-linux-gnu/debug/deps/libpanic_unwind-08cd9740da1a8428.rlib
          
          rust-lld: error: undefined symbol: _Unwind_DeleteException.dfsan
          >>> referenced by gcc.rs:89 (src/gcc.rs:89)
          >>>               panic_unwind-08cd9740da1a8428.panic_unwind.f5942db226e6de1f-cgu.0.rcgu.o:(panic_unwind::imp::cleanup (.dfsan)) in archive ~/dfsan_test/dummy/target/x86_64-unknown-linux-gnu/debug/deps/libpanic_unwind-08cd9740da1a8428.rlib
          
          rust-lld: error: undefined symbol: open64.dfsan
          >>> referenced by mod.rs:69 (src/sys/pal/unix/mod.rs:69)
          >>>               std-63b5deb51138cac5.std.5705bc538c5524d2-cgu.04.rcgu.o:(std::sys::pal::unix::init::sanitize_standard_fds::{closure#0}) in archive ~/dfsan_test/dummy/target/x86_64-unknown-linux-gnu/debug/deps/libstd-63b5deb51138cac5.rlib
          >>> referenced by unix.rs:1368 (src/sys/fs/unix.rs:1368)
          >>>               std-63b5deb51138cac5.std.5705bc538c5524d2-cgu.09.rcgu.o:(<std::sys::fs::unix::File>::open_c::{closure#0}) in archive ~/dfsan_test/dummy/target/x86_64-unknown-linux-gnu/debug/deps/libstd-63b5deb51138cac5.rlib
          
          rust-lld: error: undefined symbol: dup.dfsan
          >>> referenced by mod.rs:65 (src/sys/pal/unix/mod.rs:65)
          >>>               std-63b5deb51138cac5.std.5705bc538c5524d2-cgu.04.rcgu.o:(std::sys::pal::unix::init::sanitize_standard_fds::{closure#0}) in archive ~/dfsan_test/dummy/target/x86_64-unknown-linux-gnu/debug/deps/libstd-63b5deb51138cac5.rlib
          
          rust-lld: error: undefined symbol: abort.dfsan
          >>> referenced by mod.rs:75 (src/sys/pal/unix/mod.rs:75)
          >>>               std-63b5deb51138cac5.std.5705bc538c5524d2-cgu.04.rcgu.o:(std::sys::pal::unix::init::sanitize_standard_fds::{closure#0}) in archive ~/dfsan_test/dummy/target/x86_64-unknown-linux-gnu/debug/deps/libstd-63b5deb51138cac5.rlib
          >>> referenced by mod.rs:305 (src/sys/pal/unix/mod.rs:305)
          >>>               std-63b5deb51138cac5.std.5705bc538c5524d2-cgu.04.rcgu.o:(std::sys::pal::unix::abort_internal) in archive ~/dfsan_test/dummy/target/x86_64-unknown-linux-gnu/debug/deps/libstd-63b5deb51138cac5.rlib
          >>> referenced by mod.rs:114 (src/sys/pal/unix/mod.rs:114)
          >>>               std-63b5deb51138cac5.std.5705bc538c5524d2-cgu.04.rcgu.o:(std::sys::pal::unix::init::sanitize_standard_fds) in archive ~/dfsan_test/dummy/target/x86_64-unknown-linux-gnu/debug/deps/libstd-63b5deb51138cac5.rlib

...

I guess it's because some sanitizer library is not being linked correctly. Does anybody what is missing?

2

rustup nightly toolchains do not ship the DFsan runtime (assuming you're using a nightly build based off the flags) in the sysroot. To fix it, you can either copy clang's runtime into the rustc sysroot, or use -Zexternal-clangrt and link it manually. Either way works, just make sure you add fun:_Unwind_*=uninstrumented and fun:_Unwind_*=discard to your ablist.txt

3

To fix it, you can either copy clang's runtime

Can you be more specific? I'm seeing librustc-nightly_rt.dfsan.a available in my sysroot. Based on the symbols defined in it, it looks like it is the runtime library we are looking for.

It seems to be the libc and libcxx functions that are missing. If I try to build in no_std mode, some other errors (due to missing std functions) surface that show the compiler-rt is being linked.

error: linking with `cc` failed: exit status: 1
  |
  = note:  "cc" "-m64" "~/dfsan_test/dummy/target/x86_64-unknown-linux-gnu/debug/deps/rustc1eCGX3/symbols.o" "-Wl,-Bstatic" "-Wl,--whole-archive" "-Wl,<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a" "-Wl,--no-whole-archive" "<1 object files omitted>" "-Wl,--as-needed" "~/dfsan_test/dummy/target/x86_64-unknown-linux-gnu/debug/deps/{libcore-7ca3d0410c2f9e09,libcompiler_builtins-14acdd2268f1f692}.rlib" "-L" "~/dfsan_test/dummy/target/x86_64-unknown-linux-gnu/debug/deps/rustc1eCGX3/raw-dylibs" "-Wl,-Bdynamic" "-B<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-fuse-ld=lld" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "~/dfsan_test/dummy/target/x86_64-unknown-linux-gnu/debug/deps/dummy-1c1bfaaae5abb124" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: rust-lld: error: undefined symbol: __libc_start_main
          >>> referenced by /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/Scrt1.o:(_start)
          
          rust-lld: error: undefined symbol: __errno_location
          >>> referenced by dfsan_interceptors.cpp:134 (/rustc/llvm/src/llvm-project/compiler-rt/lib/dfsan/dfsan_interceptors.cpp:134)
          >>>               dfsan_interceptors.cpp.o:(__interceptor_mmap) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          >>> referenced by dfsan_interceptors.cpp:148 (/rustc/llvm/src/llvm-project/compiler-rt/lib/dfsan/dfsan_interceptors.cpp:148)
          >>>               dfsan_interceptors.cpp.o:(__interceptor_mmap64) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          >>> referenced by dfsan_interceptors.cpp:159 (/rustc/llvm/src/llvm-project/compiler-rt/lib/dfsan/dfsan_interceptors.cpp:159)
          >>>               dfsan_interceptors.cpp.o:(__interceptor_munmap) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          >>> referenced 9 more times
          
          rust-lld: error: undefined symbol: personality
          >>> referenced by dfsan.cpp:1232 (/rustc/llvm/src/llvm-project/compiler-rt/lib/dfsan/dfsan.cpp:1232)
          >>>               dfsan.cpp.o:(DFsanInit(int, char**, char**)) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          >>> referenced by dfsan.cpp:1242 (/rustc/llvm/src/llvm-project/compiler-rt/lib/dfsan/dfsan.cpp:1242)
          >>>               dfsan.cpp.o:(DFsanInit(int, char**, char**)) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          
          rust-lld: error: undefined symbol: dlsym
          >>> referenced by interception_linux.cpp:42 (/rustc/llvm/src/llvm-project/compiler-rt/lib/interception/interception_linux.cpp:42)
          >>>               interception_linux.cpp.o:(__interception::InterceptFunction(char const*, unsigned long*, unsigned long, unsigned long)) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          >>> referenced by interception_linux.cpp:49 (/rustc/llvm/src/llvm-project/compiler-rt/lib/interception/interception_linux.cpp:49)
          >>>               interception_linux.cpp.o:(__interception::InterceptFunction(char const*, unsigned long*, unsigned long, unsigned long)) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          >>> referenced by sanitizer_linux_libcdep.cpp:356 (/rustc/llvm/src/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:356)
          >>>               sanitizer_linux_libcdep.cpp.o:(__sanitizer::InitTlsSize()) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          >>> referenced 1 more times
          
          rust-lld: error: undefined symbol: pthread_key_create
          >>> referenced by dfsan_thread.cpp:100 (/rustc/llvm/src/llvm-project/compiler-rt/lib/dfsan/dfsan_thread.cpp:100)
          >>>               dfsan_thread.cpp.o:(__dfsan::DFsanTSDInit(void (*)(void*))) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          
          rust-lld: error: undefined symbol: pthread_setspecific
          >>> referenced by dfsan_thread.cpp:113 (/rustc/llvm/src/llvm-project/compiler-rt/lib/dfsan/dfsan_thread.cpp:113)
          >>>               dfsan_thread.cpp.o:(__dfsan::SetCurrentThread(__dfsan::DFsanThread*)) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          >>> referenced by dfsan_thread.cpp:120 (/rustc/llvm/src/llvm-project/compiler-rt/lib/dfsan/dfsan_thread.cpp:120)
          >>>               dfsan_thread.cpp.o:(__dfsan::DFsanTSDDtor(void*)) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          
          rust-lld: error: undefined symbol: sigaction
          >>> referenced by sanitizer_linux_libcdep.cpp:115 (/rustc/llvm/src/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:115)
          >>>               sanitizer_linux_libcdep.cpp.o:(__sanitizer::internal_sigaction(int, void const*, void*)) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          
          rust-lld: error: undefined symbol: getrlimit
          >>> referenced by sanitizer_linux_libcdep.cpp:130 (/rustc/llvm/src/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:130)
          >>>               sanitizer_linux_libcdep.cpp.o:(__sanitizer::GetThreadStackTopAndBottom(bool, unsigned long*, unsigned long*)) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          
          rust-lld: error: undefined symbol: pthread_attr_init
          >>> referenced by sanitizer_linux_libcdep.cpp:183 (/rustc/llvm/src/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:183)
          >>>               sanitizer_linux_libcdep.cpp.o:(__sanitizer::GetThreadStackTopAndBottom(bool, unsigned long*, unsigned long*)) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          
          rust-lld: error: undefined symbol: pthread_self
          >>> referenced by sanitizer_linux_libcdep.cpp:184 (/rustc/llvm/src/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:184)
          >>>               sanitizer_linux_libcdep.cpp.o:(__sanitizer::GetThreadStackTopAndBottom(bool, unsigned long*, unsigned long*)) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          >>> referenced by sanitizer_posix_libcdep.cpp:57 (/rustc/llvm/src/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:57)
          >>>               sanitizer_posix_libcdep.cpp.o:(__sanitizer::GetThreadSelf()) in archive /usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-nightly_rt.dfsan.a
          ...

Note that I'm currently looking for a minimal example, so I don't really mind if libc functions are not available.

In any case, I'm curious how it is managed in the test case in the compiler project. I tried building a toolchain with --enable-sanitizers passed but still no success in building my example.

4

Can you be more specific?

Add -Clink-arg=-lc -Clink-arg=-lpthread -Clink-arg=-ldl to your RUSTFLAGS.