Should I follow the `-sys` crate convention for wrapping non system C libraries?

The Cargo Book mentions that "packages that link to system libraries have a naming convention of having a -sys suffix" and that these packages should not provide bindings or higher-level abstractions, leaving this responsibility to other crates built on top of the -sys crate.

For non-system libraries should I follow the same convention and have two crates: one that simply links to my library and exposes declarations for its public functions and data types, and another one that offers a higher level, more Rust-like, interface? Or is it ok if I just publish a crate with bindings for my C library that also links with the C library, but hides all the low-level details?

Even if it's not a system lib, I always call bindings to C lib -sys

But I don't know if I'm right to do so.

What's the definition of system lib anyway? I understood it as "native lib that needs bindings"

You should try to follow the *-sys convention if possible. There are a bunch of reasons why you might want to do this, but these are the big ones for me:

  • Separation of concerns: it lets you keep all the unsafe declarations and build script stuff in the foo-sys crate while keeping the main foo crate as pure Rust
  • Versioning: your foo-sys crate's version number can track whatever library you are linking to, leaving your foo crate to create new releases as you make refinements to your API without being bound to the upstream library's version number

There have also been a couple times where I've wanted to use a specific native library, but the safe bindings someone has provided aren't suitable (I don't like its API, it makes assumptions that aren't valid in my situation, missing functionality, etc.). I'm comfortable with writing unsafe code so in those situations I can still use their -sys crate and write my own bindings on top of it.

Making a native library compile and link reliably on multiple platforms is a massive pain (I'm looking at you, TensorFlow Lite) so being able to reuse work that has already been done is a massive bonus.

6 Likes

These are really good arguments. I was just filling a bit silly to make a -sys crate for my library, which is not what I consider a system library.

Since this is not a library I expect people to just have on their systems I'm forced to bundle the C sources and compile them. Fortunately, it is a fairly small library (just one .c file) that is pretty easy to build.

Am I wrong to consider that this split between a -sys crate and a bindings crate pushes me towards having different repositories (at least not share the repository of the C library with the one that hosts the -sys crate)? As far as repository organization goes, I started with the idea of creating two directories in my repo: mylib-sys and mylibrs. A simplified view of the repository would look like this:

├── CMakeLists.txt // used to build the C library
├── include // C headers
├── src // C sources
├── mylib-sys
│   ├── Cargo.toml
│   └── src
├── mylibrs
│   ├── Cargo.toml
│   └── src

The problem with this is that, the way I understood it, cargo package will package files in mylib-sys, but it can't package files from the parent directory of mylib-sys (which is fair, where would it put those files in the package?). A solution to this will be to move mylib-sys/Cargo.toml to the root of the repository, but that is a bit awkward.

I wouldn't get hung up on the "system" word. Really we should have called them "native" instead because you are actually just binding to libraries compiled to native code, but I'm guessing the first libraries we were writing bindings for were system libraries (e.g. libc) and that's where we've got the naming from... The -sys suffix is nice and compact, though.

Nope. You can have multiple crates in the same repository without any issues. About 90% of my projects end up like this nowadays because I'll split different layers/components up into their own crates.

You should have a look at cargo workspaces.

What I normally do is move any native dependencies into the mylib-sys folder, typically under a vendor/ or third-party/ folder because the native code often comes from a git submodule.

├── mylib-sys/
│   ├── Cargo.toml
│   ├── build.rs
|   ├── vendor/
|   |   ├── include/
|   |   ├── src/
|   |   └── CMakeLists.txt // used to build the C library
│   └── src/
├── mylib/
│   ├── Cargo.toml
│   └── src
└── Cargo.toml

So basically different repos for the C sources and Rust sources. I was trying to have the Rust sources in the same repository as the currently existing C sources (the repo already includes bindings for other languages in this way, and I have control over it), but that makes things harder on the Rust side. Having a different repo with a git submodule to the existing C library makes things easier, even if it splits things across two repositories.

It's really up to you. The subfolder method works well for what i do because I'm normally writing bindings for external libraries or will have my Rust bindings separate from the main native library.

The cargo package command also respects symlinks, so you could symlink the native code into a folder under mylib-sys to maintain the current folder structure if you want. A colleague ran into issues when doing CI for Windows where git clone wasn't handling symlinks properly so that's something to keep in mind, although it may not be an issue for you.

2 Likes

This might actually be a solution in my case, thanks. The symlinks problems might be an issue. libgit2 seems to have support for symlinks but I don't know if all git clients available on Windows handle symlinks properly. As a final workaround, keeping Cargo.toml in the root of the repo works, but I can no longer use a workspace for both crates and I try to not use non-default configurations because the less weird things look, the better.

1 Like

As a follow-up question, is this considered to be an OK structure? I know I'm missing some fields in my .tomls (like description, documentation, etc), I'm mostly interested in the way the directories are structured and how crates reference each other.

Git for Windows 2.31.0 seems to handle them without problems.

Yeah, that looks fine to me :+1:

If the foocli crate is just to demonstrate how the foors crate works I'd move it to foors's examples/ folder.

It if was me I'd rename the crates to be foo and foo-cli or something like that, but that's mainly because project names that end in -rs, .js, or py annoy me... I already know the library is in Rust/JavaScript/Python/whatever so it's annoying to type a bunch of redundant letters every time I refer to it.

1 Like

Thanks!

I added foo-cli just so I could test cargo install --git on a clean machine (I was a bit paranoid about the build not actually working outside of my work environment).

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.