Sharing global default tracing subscriber across FFI boundaries

help
1

Assume that I am using the following architecture:

image
image2102×1234 202 KB

I have found that after the C executable calls init_logging, events are logged (i.e. end up in a log file or printed to stdout/stderr) via log_event, but when tracing::event is used in my_func (called also from the executable), these events are not logged. It would be possible for "other crate" to bind to "logging crate" (via a .rib), and just use the log_event function, but this would lose a lot of the advantages of using tracing macros directly (spans, context, etc.).

Main Question: Is there a way to allow tracing macros to be used in "other crate", and have them share the same global default subscriber that is being created/used in logging.so?

Secondary Question: Why are tracing macro calls in "other crate" not able to use the global default subscriber created in the "logging.so" lib?

2

Every cdylib you compile ends up with a separate copy of all Rust static variables. You need to do one of these things:

  • Have at most one "cdylib" crate in this system (which could still link against "dylib" Rust dynamic libraries, if you want).
  • Write your main program in Rust, not C.
  • Have the different cdylibs communicate with each other via explicit function calls, as you already thought of.
1 Like
3

Thanks for the ideas. The diagram I included above was missing one piece: other.so links to a my_c_lib.so file generated from C code, which in turn links to and uses logging.so. This is why logging.so was broken out as a separate lib, to avoid a dependency loop: if we want to be able to use tracing from my_c_lib.so, we can't export the logging functions within other.so, since it depends on my_c_lib.so.

So, if I understand correctly, there really isn't a way to handle this situation (if we want to use tracing macros anywhere in rust code), and we would need to replace my_c_lib.so with something generated from rust code (and re-export logging.so symbols in other.so, only binding to other.so from the C executable)?

One other question: do you have any pointers on documentation that clearly explains how / why each cdylib has separate copies of static vars?

4

Unfortunately not — I haven't seen such documentation and linking behavior isn't an area I know a lot about. But I believe it's mainly the combination of

  • cdylibs are intended to be self-contained, able to be loaded into non-Rust programs trivially.
  • Trouble would arise if multiple dynamic libraries tried to define the same global symbols. Or maybe it's that the cdylib is supposed to not expose any piece of the unstable Rust ABI.

I'm not sure whether that's right.

5

Is it possible to make logger a dylib instead of a cdylib? If so you can then use it from both other.so and my_c_lib.so as dependency using use logger as _; somewhere in the source of other.so and my_c_lib.so. This will cause other.so and my_c_lib.so to take all crates statically linked into logger.so to not be copied into other.so and my_c_lib.so.

6

The issue is that my_c_lib.so is generated from C source code. I assumed it would not work here to use a dylib from C source code (i.e. bind to it with ld), or is that not correct?

7

I missed the part where you stated that my_c_lib.so is written in C rather than in Rust. A Rust dylib can be depended on from C if it exports unmangled symbols, just like with a cdylib. The main reasons to use cdylib over dylib are that the crate metadata necessary for using the dependency from rust is omitted and that all symbols only visible to rust are made private, allowing the linker to remove them if unused. Another potential option is to merge logger.so into other.so and have my_c_lib.so depend on other.so.

8

Yes, but will that work if other.so also depends on my_c_lib.so (and at the same time, my_c_lib.so depends on other.so for the functions that used to be in logger.so)?

9

It can work if you use the right linker flags to allow this cycle. Making logger.so a dylib instead of a cdylib is probably the easier option. Make sure to compile it with -Cprefer-dynamic though to dynamically link libstd.so as otherwise no rust code can link against it due to a long standing rustc bug.

10

I finally got around to trying this, and it seems that this won't work well in my case. It requires removing the logging lib from the cargo workspace it was part of, and also requires making libstd.so available. This is either a manual process, or there is also the prefer-dynamic crate that is supposed to help with this, but this appears to not play well with WSL which is being used for compiling linux targets:

error: failed to run custom build command for `prefer-dynamic v0.1.2`

Caused by:
  process didn't exit successfully: `/path/to/logging/target/linux/debug/build/prefer-dynamic-9bf2765010a7882c/build-script-build` (signal: 7, SIGBUS: access to undefined memory)
  --- stdout
  cargo:rerun-if-changed=build.rs
  cargo:rerun-if-env-changed=OUT_DIR
  cargo:rerun-if-env-changed=RUSTUP_HOME
  cargo:rerun-if-env-changed=RUSTUP_TOOLCHAIN

Overall taking this path seems like an uphill battle, and likely to make the build toolchain more brittle. I wish this were less painful to automate with Cargo.

Let me know if there are any other possible routes to consider.

11

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.