Is there a list of most-needed crates?

I may be out of date, but it has seemed to me that Rust is still relying on C libraries and middleware for a considerable amount of function, even if it is behind a Rust "unsafe" wrapper.

Is anyone maintaining a list of which of these are most critical and/or least trusted, such that we can prioritize them for porting and/or reimplantation as native Rust code?

(I spent (mumble) years of my career working on standards, middleware and related reusables, so this was a natural direction for my thoughts to wander.)

Right of my head I can only remember that the Rust toolchain depends on libc for system calls (correct ma if I'm wrong).

There are crates that wrap C-libraries or rather libraries with a C-API for FFI. Those normally have a mechanism in place to build the library from source during the build process (build.rs) so generally it is not to much pain when you have the right tools installed. Also this FFI libraries are often statically linked so this is not much of an deployment issue.

Furthermore, on Windows Rust relies on the VS Buildtools. As a consequence Rust binaries require the vcruntime140.dll (or so) to be installed on the host system. But there is also an option to link that one statically.

Looking over Kornel's list of the most popular crates, the ones I see that bind to C (or similar) are (in the order in which they were listed)

  1. winapi — system library interface for Rust // Lib.rs
  2. native-tls — Rust network library // Lib.rs
  3. zstd — Rust compression library // Lib.rs
  4. git2 — Rust dev tool // Lib.rs
  5. PyO3 — C interface for Rust // Lib.rs
  6. Rusqlite — db interface for Rust // Lib.rs
  7. Windows — Rust API Windows // Lib.rs

There is a standard pure-Rust option for TLS: Rustls — Rust network library // Lib.rs, which that list seems to suggest is more popular than native-tls.

I imagine rewriting SQLite in Rust is not something you meant to propose. :slight_smile:

One that wouldn't be listed there is C's libm, which I understand the Rust standard library uses. There is a Rust libm that can be used instead, maybe at some ergonomic cost.

1 Like

Work on a Rust library for interacting with Git repositories is in progress:

This has funding from the Rust Foundation: Rust Foundation - Community Grantee Spotlight: Sebastian Thiel.

There appears to be ruzstd — Rust utility // Lib.rs for reading ZSTD but I don't quickly see a library for writing it.

Rewriting Python in Rust is also presumably beyond the intended scope of this thread, and MS Windows surely is.

1 Like

I guess I would conclude that a ZSTD compressor is the winner for "missing crate".

1 Like

I guess the first question to ask is... do we need to rewrite everything in Rust?

Porting anything more than a couple thousand lines of C to Rust is going to be quite an endeavour, and even then you'll need to do a fair amount of new work because the languages like to structure things differently. On top of that, you've also got to make sure your code is equivalent, which means porting their test suite to your Rust code (assuming it has tests).

That's something I might do for fun or education purposes, but rewriting a project in another language is an awfully inefficient operation. You are duplicating a lot of work for the sake of RiiR.

2 Likes

Awesome Rust

I think there are a few different questions here (or at least it can be interpreted in different ways) so I'll try to answer by explaining what Rust needs. Note that I'm much more familiar with Windows than other platforms, hence I can give greater detail there.

Rust depends on libc for some maths stuff like a number of floating point operations as well as core memory things like memcpy, memcmp, etc. These are usually highly optimized for the platform. The rust libm and compiler-builtins crates aim to provide Rust implementations of these but they may not be as performant (or even complete) as the platform's C libraries.

On Windows msvc, rust needs the vcruntime library to implement panics. Rustc/llvm (that is Rust with a LLVM backend) hardcodes a dependency on it so while we could technically rewrite it in rust, it'd require reverse engineering the vcruntime implementation to provide an exact equivalent. Windows std Rust also uses a bit of startup/runtime code but that's relatively trivial to reimplement.

On platforms such as MacOS or many BSDs using libc is required for anything that calls into the kernel. Similarly on Windows at least ntdll must be used (whether directly or, more commonly, through dlls such as kernel32). On Linux direct sys calls could be used but I as I understand it Rust would still need to rewrite a bunch more libc stuff (e.g. some pthreads things).

Beyond that there are crates that wrap libraries that are very important to a particular domain, e.g. those that wrap sqlite. Rewriting in Rust would be a massive undertaking for very little (if any) gain. See Michael-F-Bryan answer for why but I'd also add the problem of maintenance. Rewriting in Rust may lock out many of the existing maintainers (who may not want to learn Rust or switch to a reimplementation) so we instantly lose a lot of experience with the code and a wealth of domain knowledge.

3 Likes

Thanks for the insights, folks.