[Proposal - Arch Linux] Port `Aura` to Rust

Hi folks. This post mainly concerns Arch Linux users.

It has been on my mind for some time that there could be some significant advantages to porting the Aura Package Manager from Haskell to Rust. I have opened an official discussion about the idea, so if you have any thoughts, please let me know there.

Note that I'm not looking for reasons why Rust is good in general (I've already drank that Kool-Aid), I'm looking for reasons why it would be good (or bad!) for Aura specifically.

Thanks in advance for any thoughts you can offer.

5 Likes

Interesting comments about the use of clap on that "official discussion" page.

Recently I have have added clap to even the most silly little programs I churn out.

Sure it pulls in a lot of dependencies and compile from scratch takes a while.

But when I'm riffing on my code it's not getting rebuilt all the time. Edit/compile/test goes around quite fast enough.

1 Like

I've been a user of Arch and Rust for a long time. I've also played with a libalpm implementation in Rust. My advice is:

  • Plan. Work out what your goals are, and how to break them down. Make it so you can get something concrete out with as little work as possible. Otherwise you will give up.
  • Talk to other people who use rust/arch. Undertakings like this are much more likely to succeed if you talk to people who can help motivate you etc.
  • Use other people's work if possible. It's tempting when programming to want to do it all yourself. But in my experience you achieve more when you build on the work of others. Even the most famous scientists use mostly other people's work in their published literature, they are just adding an extra layer on top.
  • Make sure you are willing to put in what will probably be a lot of work. Don't waste time on something that you're going to put down eventually (without achieving any goals).

Personally, I love Arch but find some of the package management code a bit :bat: scary :bat:. I'd love to have it written in high quality Rust, even if just an alternative to the official code. Good luck, whether you decide to do this or something else.

1 Like

My 2 cents:

  • If it is not too cumbersome, using some Haskell-Rust FFI interface may provide a way to perform the transition gradually (should the interface be good, it would even allow you to keep the Haskell contributors around!);

    • Better performance, smaller binaries, and static linking.

    Given how convenient adding dependencies in Rust is, and especially if you are looking for rich-featured crates such as ::clap (not saying that you shouldn't!), the binary bloat is real. It is manageable though, especially if you don't hesitate to favor dyn over impl in argument position; but I just wanted to mention that "smaller binaries" is not the main reason I'd port a project to Rust :smile: (that being said, I don't know the size of Haskell binaries).

  • I was gonna mention, to counter any Rust fanatism, that a rewrite from a functional language to Rust should only be done if performance or other technicalities are really needed, but I have to admit the points you raise in your Github issue are quite accurate, so this looks, for once, like a well-reasoned RiiR decision :slightly_smiling_face:

2 Likes

Luckily Aura is already written (in Haskell) and I'm the original author. I've been maintaining it for 10 years already, so I'm not afraid of my suddenly dropping the project. In terms of using other people's work, most of it would be mine :slight_smile: But the fact that the libalpm bindings for Rust are already written is a big attractor for me. I had tried more than once in the past to get Haskell bindings to libalpm working, but each time the dev workflow was too cumbersome.

5 Likes

A quick test showed clap only adding about 600kb to a final lto = true + strip'd release binary. Could be worse, honestly.

Otherwise, I've been impressed by Rust's binary sizes. I've been writing Haskell long enough to know the tricks there too, and Aura currently sits around 9mb there. I'm confident the Rust variant can be smaller.

2 Likes

Cool, in that case most of what I said is irrelevant. I'm definitely interested in the project and might be able to do some drive-by contribution.

A request: could you split the project into the binary and a library that provides the functionality. With LTO you should get the same machine code at the end, and it would allow people to play with TUI, GUI, scripting, web ec. interfaces. :slight_smile:. The cargo codebase is a good example of this.

1 Like

Wow, this is so cool! Aura was my tool of choice when I was using Arch, thanks for writing it!

Given that we want to transition from Haskell to Rust, piece-wise transition (like you would do in c -> Rust direction) seems hard. So, it probably makes sense to just spend n weekends writing an aura4rs prototype, and evaluate based on that?

3 Likes

I'm curious. I know nothing much about Haskell or even the nitty-gritty of the Functional Paradigm, but of course I have heard a lot about it everywhere in the last couple of years. Especially since "Functional Programming" arriving in the Javascript world and my recent discovery Rust. Apart from that "Functional Programming" is not something I have ever seen anyone use, or even talk about, in decades of programming.

Meanwhile I gather you are a Functional Programming / Haskell old hand contemplating a switch to Rust.

As such you are the guy to ask. My question is, if it were not for the convenience of using that libalpm library from Rust would you still be contemplating a move to Rust? Or is this just a one off case, for expedience, and any new projects you undertake will still be in Haskell?

I guess I'm asking because I have had to use more languages than I ever wanted to over many years. I'm loath to tackle YAFL. I promised myself Rust would be the last. On the other hand I hate to think I might be missing out on something by not getting immersed in going fully FP/Haskell.

In short perhaps, is there enough FP in Rust?

I've been writing FOSS Haskell for 10 years, but also wrote Scala professionally for 4 years or so, and Haskell for 2. The Haskell ecosystem is wonderful for software development, and real practical Haskell can be written without much "wizardry" at all. Further, I have taught it to people who had no prior programming experience and they didn't have much trouble. Yet, Haskell still has issues with adoption, and many cite the learning curve. Four reasons why people fail to adopt a new thing in general:

  1. They're forced to use it (say at school or at work).
  2. Lack of support and learning resources.
  3. It's very different from what they're used to. Learning requires unlearning first.
  4. They're suspicious of it in general (say from reputation).

When it comes to Haskell, (1) is rare but I've heard of it, (2) has gotten much better in recent years, (3) cannot be helped, since the world has been writing imperative code for decades, and (4) can come out of (3) but is also worsened by Haskell's vocabulary of concepts (think Monad, which nobody in the FP world actually has a problem with or confusion of).

To the question at hand: it is my experience so far that Rust has successfully integrated both the "battlefield" lessons learned from the software development world (memory safety, performance, project management) and the conceptual/theoretical lessons that the FP communities figured out (chained errors, ADTs, pattern matching, traits and their autoderivation, etc.). As an analogy: humans have been building bridges for a long time, but we wouldn't dare build one today without applying solid engineering principles that carry with them scary words like "eigenvalue". And there would have been some awkward transition period between the "ad hoc + experience" approach to the principled approach. I think that the software world is in a transition period of its own, and Rust is precisely a language built to fill the role of "principled programming language".

But lifetimes are hard! This is all so tedious. I'm going to go write software in FooLang, I've never had a problem with it before.

Okay, and that's your freedom to do so. But don't be surprised when you drown in bugs and complexity and your bridge falls over. I want to port Aura to Rust because I don't want my own bridge to collapse. New projects for me will continue to be in Rust.

5 Likes

Yes, I currently do this for the Haskell version too, and plan to make it even a bit more granular for the Rust port. I'd love to have alternate frontends, and plan to write a webservice that uses Aura's PKGBUILD analysis logic to provide security warnings for all packages on the AUR.

1 Like

That's basically my plan. I just need to confirm:

  • Can clap solve my CLI arg issues? (preliminary results: yes it can)
  • Do the existing libalpm bindings work well?
  • Can I build with static linking?

Otherwise, when actually porting the thing, I can just stare at the Haskell at port the logic over.

The rust standard library is statically linked unless any of your rust dependencies (not c dependencies) is built as a dynamic library or you pass -Cprefer-dynamic to rustc. If you also want to statically link libc, you will have to use the x86_64-unknown-linux-musl target or similar to link to musl. Recent versions of glibc supposedly can also be statically linked. You can do this by passing -Ctarget-feature=+crt-static I think.

And if a lower-level C library has a number of dynamic libs, is there a way to statically jam all those into my final Rust binary?

As far as I know there is no linker that supports statically linking a dynamic library into an executable. It should theoretically be possible though. What you can do is compiling the C code as static library and link against that.

Ok, thanks. The static linking isn't a strict requirement at this point, more like a "nice to have".

I forgot to add this to my post, but it reflects my thinking too :wink:

1 Like

Just come across paru from the guy who wrote yay - might be interesting to you.

2 Likes

I did see that project, yes. He wrote the alpm bindings I'll be using.

At the end of the day though I think Aura should remain its own project, since it has slightly different goals and features than other projects, including yay.

1 Like

concurrent-downloads

(If the GIF isn't running, click to play it.)

This is a demonstration of my Rust port proof-of-concept. It:

  • makes multiple connections to libalpm via r2d2 and queries it for package information
  • downloads the corresponding package tarballs via curl and rayon
  • shows concurrent progress via indicatif
14 Likes