Mono repos in rust

help
1

Hi,
I am wondering if setting up our code base in a mono repo is a good idea or not. We develop several applications that do similar things and use similar (internal) dependencies. I think having everything in one repo could simplify some things. On the other hand I am afraid of compile times and testing times in the ci/cd pipeline.
Have you experience with mono repos in rust? Is there a good/practical way of not testing everything all the time and build and deploy only new versions of the changed crates? Like branch name conventions or any interesting tools.

2

Cargo supports a concept called workspaces, which simplify such a monorepo with multiple packages setup:

There are many OSS projects that utilise workspaces. The tokio codebase for example is set up as a workspace, plus they have a pretty extensive CI system and a test suite with long running (loom) tests. You might find inspiration there.

1 Like
3

If you regroup several applications in the same repository, you'll put them into separate crates in a workspace, but that won't change the dependencies. Each app, being a crate, will have a similar list of dependencies; the only difference is the name of the dependency.

You may also run into a few problems:

  • Your CD/CI pipeline will have to do more validations, at least it does all the unit / integration tests on each commit and in each branch. When there's a problem, the origin will be less visible. Also, it's more complicated to get a commit with a good delivery.
  • If you need to branch, you must branch everything. That also means you must probably have several clones of your repository if crate A uses as dependency a crate B that in another branch (if you need to test and develop both in parallel, which is usually the case).
  • If you want to checkout a particular version of a crate belonging to your unique repo, you must do it in another directory, otherwise your entire mega-project structure is impacted.
  • When you commit files, you must pay attention to split the commits per crate instead of simply committing the whole repository. It's easy to forget that and commit unrelated work.

Since you're several working on the same repository, it makes most of the problems above even more complicated, and merging will happen more often (which could be OK or difficult, depending on how you're organized).

Of course, it depends on the size of the apps, their dependencies, how close they are, etc... If they're tightly coupled and evolving together, it may make sense to use a unique repository.

2 Likes
4

I should have mentioned what I already know, sorry. I have read nice matklad's large-rust-workspaces and looked at rust-analyzer a bit.

5

At the moment we have 3 bigger applications and like 4 internal library crates. The things that annoy me right now:

  • on boarding: people have to clone 7 repos to get everything
  • maintaining the same things like ci/cd, git hooks etc in all of the repos
  • adding a feature in an application often needs a feature in one the internal deps. So I need to change the internal crate, create a PR and then either wait for merge or change the branch of the dependency in the application cargo.toml so that I can add the feature in the application

All of this is so much nicer in a monorepo. My biggest concerns are:

  • compile times and test times: If I understand correctly rust-analyzer and tokio build everything in all PRs, I don't think I would have to do this, we only need to test the things that changed
  • git repo size limitation
  • dev environment that has to contain everything of all applications

I am especially interested if there is a nice and simple way to only test and deploy things that changed.

6

I don't know if a tool does that. You could have a look at the output of cargo metadata.

I suppose you'll have the intra-repository dependencies declared as path, in order to avoid mixing objects coming from different versions (which isn't authorized). So that keeps things relatively simple: with a script, you should be able to determine which crates have changed from the list of modified files. Then propagate to get the entire list of crates that are impacted.

It requires some work, though. With such a small number of crates, only an extensive number of tests will justify doing that.

Maybe there's a better way of doing it.