We have a Flow-based programming platform which consists of a shared object per component. Each component is also intelligent enough to pull in OS specific libs (as is the case where we wrap the Iron webserver into a component that is able to setup an iron dependency, namely openssl).
We very quickly run out of TLS initial-exec variables and it feels like we're hitting a brick wall. Could a TLS expert please comment on this, and/or provide better resources to study?
We can
recompile every OS dep passing in certain flags (whatever they might be), undesirable, but possible.
recompile libc, although recent versions of libc do not have the DTV_SURPLUS variable to alter, besides we'd like to support hundreds possibly thousands of shared libs.
I have no practical experience with this, but I read the ELF TLS extensions spec a while back and it sounds like you want to force modules which use TLS to be compiled using the "global dynamic" or "local dynamic" TLS model; e.g. for gcc the -ftls-model option would seem to fit the bill. (Are you already using -fpic ? Supposedly this implies what you need for TLS). Haven't checked yet what the magic for LLVM/Rust is if you're using Rust TLS to a significant degree.
EDIT: clang claims to support the same option. I haven't found anything for rustc, but since clang does it it could presumably be added easily if it doesn't already work.
Super informative, I'm really not sure how to pass these args through cargo to rustc, using rustc directly becomes troublesome as we'll make the components less powerful (crates.io is filling up with some really good work). Seems like a bit of plumbing work would be needed to get make this feasible.
If others know of a neat sneaky trick to get this to work from cargo we're all ears!
EDIT(2): ...
Yes it is! What seemed like a good idea (ie components setting up their own OS deps at build time) became a short lived success run straight into a TLS wall. It's all good and well compiling components with "global dynamic", do I have to recompile openssl and all other C libraries with similar flags? If this happens we'll almost certainly have to announce after we setup a Hydra CI, the compile times will drive developers insane.
One way is to lobby for Cargo to support this. So far, Cargo developers have been less than enthusiastic about adding new flags for Cargo to pass some flags to rustc.
The way that always works is that Cargo uses the environment variable RUSTC if defined, so you can let that point to a shim shell script that calls rustc --my-options "$@" and then call cargo build.
One last important piece of information, would I need to override the configurePhase of all the transitive OS deps so that the deps also support this "global dynamic" flag?
If my suggestion is going to work at all, which I don't know, then it would need to be passed during compilation of all C libraries that use thread-local storage.