I've been using Rust for quite a while and have made several standalone tools in the past. However now I'm trying to integrate some Rust into a large legacy codebase at work by compiling to a DLL/*.so and providing a C-style header file.
I'd like to hear people's experiences doing something like this, and compile a small list of useful resources (if one doesn't already exist) to help others use Rust as part of larger projects.
Some interesting questions which may need to be solved:
Making sure your header file is kept in sync with the symbols your crate exports (I ended up making my own tool for this)
How do you provide good documentation for the exported bindings?
What pain points are there exposing Rust to C? (e.g. ensuring correctness and memory safety with unsafe FFI bindings, integrating into an existing build system, etc)
And just in general, if you've done something like this before, how did it go? If not, can you think of any issues (both technical and cultural) which I might encounter in the long term?
I originally looked into using rusty-cheddar too. The thing which stopped me was that it doesn't look like it's being actively worked on and the rusty-binder project doesn't look like it's ready.
That said, it wasn't overly difficult to use syn to parse a chunk of code, then walk the parse tree storing any relevant function and struct definitions so they can be rendered at a later stage.
What's rust-cpython like for making Python bindings? Is there much benefit with using that over something like cffi and loading the *.so manually?
It's not perfect by any means (the PyO3 fork will be significantly better once the features it requires are supported on stable Rust), but it's definitely much more comfortable than using cffi.
It's type-safe.
It does a pretty decent job of providing comfortable type conversions. (See the list of impls for ToPyObject and FromPyObject.)
With setuptools-rust, it can be easily integrated into the setup.py build process.
Take a look at the src/lib.rs code example in the Usage section of the rust-cpython README. (It can be used to to write Python modules in Rust or to embed a Python runtime in a Rust program. The first example you see when scrolling down is the "embed Python" case while the src/lib.rs example is the "extend Python" one.)
For my purposes, I don't need to know a thing about C or the Python C APIs as long as I'm using rust-cpython.
I also shared this link on Reddit and koheant raised a very good point.
The best tip I can offer from past experience is to keep c-integration squarely in mind when designing and implementing your library. Many Rust concepts won't translate well to C and will require some sort of translation, heap allocations (and freeing functions), or both. You have to make sure that data can be cheaply converted from/to C.
If you're going to target C codebases as first-class consumers of your library alongside Rust, it might make sense to store all your data in a C-native format.
I wrote a few things about this here https://unhandledexpression.com/2017/07/12/how-to-rewrite-you-project-in-rust/
The most important parts IMO are integrating with the build system (you must get that right as soon as possible in the project) and being careful with coupling. Sometimes it's better to work on smaller parts of the code before attacking the big chunk.
I'm lucky in that the thing I'm working on is going to be its own distinct module and it's a fairly small (only 2 people) project, so I'll be able to get away with giving the compiled DLL and a header file to my colleague and that can then be linked in like normal.
Did you experience much friction trying to get cargo to work nicely with other build tools or the IDEs which may use them?
I integrated rust with java (android and oracle jvm (two separate projects)).
But in fact of course it was C API (JNI).
At first, I just put #[no_mangle] and works with C API directly,
and it was the first time when I see crash of rust application (I convert Box to 64 word, and when convert it back mix types).
So I wrote tool to automates this binding
managment. It just take rust code as input and generate rust + java code as output.
And cargo care about approapriate updates.
In future I'm going to use this tool to automate C++ binding generation.
Because of my rust project (workspace) was part of other project, I integrate call of cargo to java build system, it calls cargo in pre-build rules and then take care of resulting .so library.
CFFI is the Python-side interface. The CFFI-based equivalent to rust-cpython or PyO3 would be Snaek. (And, judging by the README, it's much more primitive.)
That said, I've already evaluated that approach and, as far as I can tell, the only PyPy-compatible options aren't type-safe.
(I've never trusted my ability to code in C and, if I had to give up rust-cpython for anything other than PyO3, I'd just switch back to writing projects that don't mix Rust and Python in the same process.)
...plus, I dread the day I have to write something that exposes a C-friendly API. My use of Rust so far has been as a replacement for Python with stronger compile-time guarantees, which means letting rust-cpython hide the "translate through a C ABI" step of writing an API that operates where Rust and Python concepts overlap.
EDIT: Honestly, I'd be more amenable to taking responsibility for providing multiple high-level bindings through projects like Neon (The Node.JS equivalent to rust-cpython), Helix (The Ruby equivalent to rust-cpython) and the in-development Rust-GObject integration support rather than having to deal with the mental burden of designing and maintaining a properly-written C-compatible API.
Type safety is a small lie we tell ourselves to keep ourselves sane. It stops a lot of stuff at compile time, but it ends there. It doesn't exist in the linker so you're reliant on the FFI library writer finding all the bugs in the mapping. And if you only have a small interface surface then it's not that hard to write your own #[no_mangle] unsafe code. Basically -sys crates are doing this.
True, but, to me, that sounds like the comments by people who are trying to dismiss the accomplishments of formally-proven microkernels by implying that formal proofs are worthless as long as the code is running on hardware capable of real-world flaws.
I'm a life-long GCed-language user who only considers Rust a viable option so long as my productivity is preserved and I don't have to stress out over my "the only thing I know is how little I know" relationship with C and C++.
If you were to successfully convince me to not use rust-cpython, it wouldn't prompt me to write my own FFI code, it'd either prompt me to port everything back to pure Python or drive me to the next option which provides a similarly high-level API and ability to let me shirk responsibility for memory safety... which would probably be the Python frontend launching a Rust subprocess and controlling it via either D-Bus or JSON-RPC.
Even if I trusted myself to write unsafe code, which I don't except for the most trivial of things, I've only written one unsafe block across all my projects, consisting of unsafe { access(cstr.as_ptr(), mode) == 0 } as a replacement for Python's os.access.
I spent over an hour auditing it, and it still makes me feel nervous whenever I look at it because I know that I saw a post on /r/rust about how there are ways you can write the preceding CString::new(abs_path.as_os_str().as_bytes()) conversion which will compile without maximum-warn Clippy complaining, but present a use-after-free bug.