Preventing export of `log` symbols in staticlib

[jsha]

In the crustls project (an FFI wrapper around rustls), I'd like to add a log facility, so that users of the library can register to receive logs and handle them however they'd like. That will require implementing and registering a log::Log. In the Rust ecosystem, the idea is that libraries never register a Log, leaving that up to the application, so that we can be sure never to have conflicting Log registrations.

However, for this library that's not an option, because the user of the library may be another library - for instance, libcurl.

What I'd like to do is register a Log within crustls that only affects log lines generated within the library, even if the user links multiple Rust FFI staticlibs.

I think I can achieve that by ensuring that the symbols from the log crate are not exported:

$ nm lib/libcrustls.a | grep LOGGER
0000000000000000 d _ZN3log16SET_LOGGER_ERROR17h873d41736644bcfeE
0000000000000000 d _ZN3log6LOGGER17h7a1bf662a47e7714E

Is that accurate? And if so, how would I go about doing it?

[jsha]

I think I've figured it out, and the answer wasn't preventing export of symbols. It was using name mangling to ensure those symbols aren't used by other libraries. I've written it up, along with a demo, at GitHub - jsha/rust-private-loggers: A demo of making a Rust staticlib with exclusive control of its own logs. Text of the README below for convenience:

Private Loggers (and other crates) in Rust FFI Libraries

When building libraries in Rust for consumption via FFI in other languages,
there's a question of what to do about global variables in other crates.
In particular, the Rust ecosystem mostly uses the log crate for logging.
Library crates are supposed to use the log crate's global LOGGER object
to send logs; applications are supposed to register a logger
exactly once, at the beginning of main. This is important because only one
logger can ever be registered.

What about FFI libraries? The application consuming your code probably isn't in
the Rust ecosystem, so it can't register its own logger. Your library could
register a logger, and provide an interface for the application to register
interest in messages from that logger. But that runs afoul of the requirement
that only the application is supposed to register a logger. And what happens if
some application links your library and some other Rust FFI library that
registers a logger? Whichever library registers its logger first wins.

What we really want is for each FFI library to have its own copy of the log
crate, with its own version of the global log::LOGGER object.

It turns out Rust has already solved this problem once, for dependency
resolution. Thanks to @sagebind for an excellent blog post on the topic,
which helped me figure this out.
Rust can use two versions of the same crate in one library, thanks to "name
mangling" and a "disambiguator." It assigns different names to symbols from
each of the crates.

Cargo controls the disambiguator by passing -C metadata=... flag to rustc, which
can be specified multiple times. Cargo does dependency resolution to figure out
what it needs to provide for the metadata flag in order to ensure that if
multiple versions of a crate need to be linked, they don't conflict with each
other.

You can use the RUSTFLAGS environment variable to add your own -C metadata=...
flags, which Cargo passes along to its rustc invocations. The normal metadata
flags still get passed and respected, so this doesn't break the normal
functioning of the disambiguator.

RUSTFLAGS="-C metadata=something_unique" cargo build -v

When you build your library with an invocation like the above, all of your
library's dependencies have their names mangled uniquely. So even if someone
links your library, and some other Rust FFI library that has the same
dependencies, (a) the symbols won't conflict, and (b) you can interact with
globals in your dependencies without affecting the other library.

This has a bit of a downside: If an application links in a lot of Rust FFI
libraries that all use this approach, it will wind up with a bunch of
independent copies of common Rust code. Whether this tradeoff is worthwhile
depends on your intended use for the library. I think in most libraries intended
for general use, making dependencies "private" via this technique is the right
solution.

Try It Out

Clone this repo, then run make test.

This will build two Rust staticlibs, logginglib1 and logginglib2. Each one
registers its own Log impl that writes to stdout. Logginglib1 prefixes its
lines with one: ; logginglib2 prefixes its lines with two: . In a more
realistic deployment, these libraries might offer a callback-based API so that
applications or other libraries can register to receive log lines.

The Makefile builds each of logginglib1 and logginglib2 with a different
RUSTFLAGS="-C metadata=...", and then links both of them into a small C
program main.c. That program calls a function in each library that generates
a log line. The expected output is:

./main
one: hello
two: goodbye

[Michael-F-Bryan]

Alternatively, what about using a logging crate which doesn't rely on a single global logger?

A couple years back I was working on integrating several Rust modules into a legacy Delphi app and my problem was that each cdylib would get its own copy of the log crate (including log::LOGGER) so I'd need to initialize each library's logger manually. My solution was to switch to slog and pass around a slog::Logger that both could use, which - while technically unsound - worked well enough (types from one cdylib aren't guaranteed to be compatible when used in another cdylib even if you use the same compiler).

[jsha]

In general, I think having a non-global logger is great - and I proposed that in Per-session log facility · Issue #438 · rustls/rustls · GitHub. But after some discussion, the conclusion was that passing a logger through all the layers of the crate would introduce too much complexity. And, as pointed out on that thread: even if the top level crate does it, there may be dependencies that use the global logger, and you may want to capture those.

The other thing I like about this solution is that it solves more than just the log problem - it reduces the possibility of surprising interactions when you link two Rust FFI libraries into a single binary. In theory linking two Rust libraries should not behave any differently than linking a Rust library and a C library, or two C libraries. But in practice, without this mangling trick, there's potential for two Rust libraries to interact differently than, say, a Rust and a C library.

[riking]

I feel like there's something that could be done with weak symbols, such as a #[no_mangle] #[weak] fn init_rust_logger(), if it wasn't currently buggy: Tracking issue for the `linkage` feature · Issue #29603 · rust-lang/rust · GitHub