Crate Interoperability and 3rd-Party-Types in Interfaces

help
#1

I have some basic questions about how crates are supposed to work together in Rust, specifically if their public interfaces contain common, but not trivial data types. This question requires a bit of context, so I will start with an example to explain the scenario. After that I will get into the more abstract questions.

Example

An example for such a common type would be an RGB color value:
Lets assume that there is a crate that I want to use that has public functions that take an RGB value, lets call this hypothetical crate draw (no connection to the actual draw crate). Because the authors of the draw did not want to re-invent the wheel, they decided to use the type rgb::RGBA from the rgb crate in their interface for functions such as set_color(...) and they publicly re-export this type in their crate as well.

Let's now suppose that I want to author a second crate that can be used together with draw but also in other scenarios. My crate (in this example called skycolor) has a public function called get_sky_color() that returns the current color of the sky where the program is run. Because I do not want to reinvent the wheel, either, and because I want interoperability with draw, I also use rgb::RGBA in my public interface (as the return type of get_sky_color()).

Now (as far as I understand), there is a big problem: Since I do not own the draw crate and other possible "consumers" of color values produced by skycolor, I cannot control that skycolor's version of the rgb crate is the same as the one used in draw. If someone wants to use skycolor together with a newer version of draw that references a newer version of rgb, the code will not compile when you pass the return value of get_skype_color() to set_color(...). Even worse: when someone wants to use skycolor together with two crates (e.g. draw and print) that each reference a different version of the rgb-crate, there is no way to have compatible interfaces!

Questions

  • How should crates handle complex/composite data types in their interfaces? Should one use specialized crates such as the rgb crate for types or is it better when each crate defines their own types and the crate user is supposed to do conversion each time a value is passed from one crate to another?
  • Should a crate, if it uses such a data-type-crate, publicly re-export the type? If yes, does this mean the actual source of the type is an implementation detail of the crate and therefore there cannot be interoperability between this crate and another one that uses the same type?
  • Is there a clever way to make it possible that all crates in a project actually use the same version of a data-type-crate such as rgb? If this is possible, then there would not be any compile time incompatibility (as long as the signature of the types themselves did not change).
#2

From what I understand, the cargo version specification for a package is actually a set of compatible versions. Cargo will pick a single version for common dependencies that is in the intersection of these sets if possible: If you specify that you need rgb version ^0.8.1, for example, and some other crate asks for version ^0.8.20, they'll be compiled against the same version (something semver compatible with 0.8.20).

1 Like
#3

Unfortunately even the most liberal version specification in my crate's Cargo.toml won't really help when any crate involved uses a pinned version.

You can find the "code" for this example on GitHub.

my-bin-crate/Cargo.toml:

[dependencies]
draw = { path = "../draw" }
skycolor = { path = "../skycolor" }

draw/Cargo.toml:

[dependencies]
rgb = "~0.4.0"

skycolor/Cargo.toml:

[dependencies]
rgb = "~0"

Even though ~0 and ~0.4.0 are Semver compatible, this does not build:

   Compiling my-bin-crate v0.1.0 (crate-version-testing/my-bin-crate)
error[E0308]: mismatched types
 --> src/main.rs:3:21
  |
3 |     draw::set_color(color);
  |                     ^^^^^ expected struct `draw::RGBA`, found struct `skycolor::RGBA`
  |
  = note: expected struct `draw::RGBA<u8>`
             found struct `skycolor::RGBA<u8>`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0308`.
error: could not compile `my-bin-crate`.

And my-bin-crate/Cargo.lock reveals that indeed two different versions of rgb are used:

...
[[package]]
name = "draw"
version = "0.1.0"
dependencies = [
 "rgb 0.4.0",
]
...
[[package]]
name = "skycolor"
version = "0.1.0"
dependencies = [
 "rgb 0.8.25",
]
#4

This sounds like a cargo version resolution bug. Manually setting the lockfile such that both dependencies use rgb:0.4.0 works as desired and doesn't cause any issues.

Within semver-compatible versions (e.g. ~1 and any constraints within), cargo will actively resolve multiple constraints to the same version, and refuse to resolve if that's impossible. It seems that in the case of a version constraint covering multiple semver-incompatible versions, it's not attempting (hard enough?) to match the wider constraints to the tighter one. Of course, in the general case (~0.4 + ~0.8 + ~0) it'd be impossible to match the wide constraint with both tighter ones, but in the case where all constraints have a solution, it'd be nice to find it. (Unfortunately, dependency hell is NP-complete (isomorphic to a SAT solver) (i.e. a formally Hard problem), so it's not so simple as just handling this case.)

I reported this issue as rust-lang/cargo#9029; we'll see how the cargo maintainers interpret this.

From a high-level standpoint, vocabulary type crates really should be 1.0 and near-forever-stable. Obviously, that's not entirely practical, but it would solve the issue, as cargo does correctly solve the version constraints to the same version within a single semver-compatible range (e.g. ~1).

Secondly, a dependency on >=0.0.0, <1.0.0 is always wrong, just like * is always wrong (and actually forbidden by crates-io), so it's mostly an artifact that the cargo version resolution doesn't work with it, as it's not actually a problem for actual (probably) accurate constraints published crates use. But a more reasonable constraint of >= 0.4.0, <0.9.0 also uses different incompatible versions, so there's definitely an issue here.

1 Like
#5

For type interoperability between crates only semver-major version has to match. Minor versions don't matter; they will be unified by Cargo.

Re-exporting and using re-exported types is a good idea. At least it saves you if you need to be compatible with only one crate.

Crates with "interoperability" types have to be conservative with their semver-major versions.

There's also:

2 Likes
#6

One other bit: not reëxporting the types can sometimes improve the error message. It doesn't seem to in this specific case, but cargo/rustc can often detect that two incompatible versions of the same package are in use, and emit an error like

error[E0308]: mismatched types
 --> src\main.rs:3:21
  |
3 |     draw::set_color(color);
  |                     ^^^^^ expected struct `rgb::RGBA`, found struct `rgb::RGBA`
  |
  = note: expected struct `rgb::RGBA<u8>`
             found struct `rgb::RGBA<u8>`
  = note: are two versions of the same crate in use?

Response of a cargo maintainer:

Cargos resolver (with in the one-per-major constraints) attempts to optimize for "getting everyone the most recent they can use" and does not optimize for "have the fewest versions".

Unfortunately for public dependencies with large semver-massive ranges, that means pulling in the most recent semver-compatible region, rather than unifying with other semver-lesser usages.

#7

Alright, so I guess when I was encountering this in the wild, I have been a bit unlucky with the versions in use (and hitting a potential cargo bug).

As far as I understand it, in an ideal world

  • A "type-crate" is stabilized and is very conservative with version bumping. All crates that use it refer to it as e.g."~1" or "^1.0.0" and as long that there is no breaking change, everyone should be happy.
  • Crates that use this "type-crate" should reexport the crate and maybe also explicitly reexport the type itself.

From @CAD97 remark about better error messages and from my experience with how proper encapsulation should look like, not renaming/exporting the type explicitly is probably better as it explicitly promotes interface calls using the base type (rgb::RGB) instead of the re-exportet type (draw::RGB).

#8

And the defaults - just specifying the verison as "1" or "1.0" will also work fine in cargo.

Now, I don't speak for cargo, but I don't think cargo has ever intentionally tried to unify dependencies in the way expected in the bug #9029, so it's more like a feature request than a bug report. Just my guess.