(general) Developing v1, v2, and common crates

community
1

Not really rust specific, but how do you manage developing complex and potentially conflicting systems with components reused amongst them. I'm taking products not libraries.

For example, I have v1, and it's great. Now we want v2...great, big blank piece of paper....except some stuff from v1 should be pulled across. But v2 wants to apply lessons learned from v1 and potentially do things differently, better, but in an incompatible way with v2. Some other crates however, can be reused seamlessly between v1 and v2. I explicitly don't want my team to have to manage more versions of a thing then they have to.

Typical ways I've done this before over the years (excluding pre-git):

v1 branch and v2 branch (and v3 branch ...)

No, just no. Merge conflicts abound and the promise of "sharing code between them" falls apart really quickly.

[each coarse grained graph of components is a released and versioned product]
In the workspace of 100 crates, 10 of them are really a single coherent subsystem, so let's release that subsystem as a single crate pushed to the private repo.

Very easy to get "noisy" graphs which require "leafs" (i.e. common crates) so they need to be released. Pretty soon you end up with multiple graphs that each have different versions of common components.

single source tree with nested folders

 - common/db
 - v1/v1SpecificCrate
 - v2/v2SpecificCrate
Cargo.toml

This works beautifully - everything is kept up to date, refactoring across components is a dream, everything fails fast. Wonderful. Except that common crate now differs between v1 and v2...oh. So deep breath, create v1CommonCrate and v2CommonCrate and then read 3 chapters of Clean Code and 4 chapters of Code Complete to renew your soul.

And then the next divergence....

copy and paste v1 and v2

I mean....if v1 is essentially dead then maybe.... I'm not going to justify it in any way though.

submodules and subtrees

This is really a tactic to achieve variations of the above, but it very quickly becomes painful as you realise you've fallen into a trap of still needing a release process without the formality of such a process.

So what do you all do?

I think the choice of "which causes the least pain with the most safety" is really context dependent. If you have very little cross over then it doesn't really matter. If you have lots of divergence in common code then it's pain everywhere.

I think I'm settling on a single source tree and avoiding divergence in common modules. If divergence must happen then I think separate crates ruthlessly cut down to only the divergence (so common/NastyCrateCommon, v1/NastyCrate, and v2\NastyCrate where both the v1 and v2 crates just delegate what they can back to the common crate).

Feature gates help, as does a strict separation between my-crate-api and my-crate-lib.

So, what do you all do?

2

Could you do something similar to "semver trick", i.e. make one version depend on another and reexport everything common, but hide everything rewritten?

3 Likes
3

Yes, Cargo features would work in the same vein.

4

That's the approach used somewhere I worked at before, so that different versions of APIs could run together, and if you had a way of knowing if anyone is still consuming the old version, then you could deprecate, then later on remove it (code deletion is a relatively rare practice unless you have something telling you to).

This versioning method was really important for us because our software APIs were "promised" to be forever.

Another approach I've been considering (as an idea, not from experience) is, if you have just one version of logic, whose behaviour is backward compatible (does this conflict with your requirement?), then you could have versioned data types, each data type looks like this:

// Only add a new variant when the shape of data changes,
// and name it after the crate version that you release it in
pub enum SomeApiType {
    v1_0 { .. },
    v1_1 { .. },
    v2_0 { .. },
}

and always have a way to either:

  • have the logic work with old versions of data
  • have upgraders between old variants and the latest variant, and logic always works on latest variant

and for multi-crate, I'd actually keep the multiple versions of the same "capability" in one crate, and separate crates will be separate capabilities. i.e. my crates wouldn't be organized as:

  • product v1 with capabilities Av1, Bv1
  • product v2 with capabilities Av2, Bv1

but rather:

  • capability A, v1,v2,
  • capability B, v1
  • product v1 depends on A and B
  • product v2 depends on Av2 (which contains Av1 and Av2 code) and Bv1

and all of those would be in a mono repo -- important so we can debug by going "checkout this tag of this product", and it would also have all the right sub-crate tags. So product is essentially a "combine the capabilities" crate, without having any code of its own

2 Likes
5

I think we are on similar paths. The product is heavily based on event sourcing, in fact, (almost) every single mutation is captured in an event (for the world's best audit ;-)), and versioning events is...fun.

6

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.