I'm attempting to use hdf5 0.7.1 and numpy 0.14.1. According to crates.io, hdf5 requires ndarray >=0.13 and <0.15; numpy requires ndarray >=0.13 and < 0.16.
When I build, I'm getting both ndarray 0.14.0 and ndarray 0.15.3. As best as I can read my Cargo.lock file, the 0.15.3 is getting pulled in because of numpy. This is causing problems elsewhere.
How can I tell if crates.io is correct that numpy 0.14.1 could work with 0.14.0?
If so, is there a way to tell cargo to use 0.14.0 everywhere? I've been searching the web, and don't see how I can insist on this.
I'm considering trying to back-rev numpy, but I'm suspecting I'll fork myself into a real dependency hell if I do.
This is exactly the reason why my npyz crate does not provide it's own support for ndarray, and instead the documentation provides functions you can copy into your own crate.
Highly experimental libraries like ndarray should not appear in public interfaces. This is, imo, a failing of the hdf5 and numpy crates if they use ndarray in their public signatures without an alternative.
Edit: though from the way you worded your post it sounds like this might not be the issue? What are the "problems elsewhere"? Normally it should be fine to have multiple versions of a crate if they are only used privately. (aside from a few technical problems with ffi dependencies)
EDIT 2: "This" (in the first sentence of both this post and the edit) refers to the issue where one library provides a type from some_crate 0.2 and another expects a type from some_crate 0.3, which is semver-incompatible.
Highly experimental crates should announce themselves as such! I'm using ndarray and ndarray-linalg everywhere!
The collision came about trying to hand a Python::numpy array to a procedure that uses a rust::numpy array, and then converting that to a rust:ndarray for my internal code. My code was using one version of ndarray, rust::numpy was using another.
I would argue that, when authoring a library, one should regard every public dependency as a responsibility. If a dependency affects users of a library—be it due to its types or traits appearing in pub fn signatures, or due to you implementing one of its traits for your public types, etc.—then you must be prepared to keep your library updated whenever the dependency publishes a semver-incompatible change.
I always try to look into other crates' policy towards breaking changes, and try to defend against frequently updated deps with these techniques:
You can re-export the version you depend on, so that users always have access to the version that works with your code. For instance, codespan_reporting (a crate for fancy error messages) provides a reexport for termcolor so that you can
use codespan_reporting::term::termcolor::{WriteColor as CsWriteColor};
if need be. This works especially well if it is unlikely that the user will be using the dependency for any other purposes besides interfacing with your crate.
You can use cargo features to force the user to opt into anything that publicly depends on the dependency. E.g. you can lock functions that take a rand::Rng behind #[cfg(feature = "rand")], with documentation warning the user that version-related troubles may arise in the future.
Unfortunately, many widely-used crates don't describe in their README their policy towards breaking changes. Most of the time, it's implicit in the version number. Generally speaking, a version number < 1.0 (such as ndarray's 0.15) is a strong indication that the authors are not willing yet to commit to a stable API. But of course, there are exceptions; for instance, nom is at 6.0... er, wait, there's already a 7.0.0?! Meanwhile, libc = "0.2" is probably one of the safest crates you can possibly depend on, especially given what was learned during the 0.1 to 0.2 migration.
One last point: "Highly experimental" was strong wording. I did not mean to suggest that ndarray is not production-ready (I can't speak for this); But it has gone through many changes, and will likely continue to do so, because (a) it faces some very difficult design problems, and (b) the language is continuing to introduce more features that are desirable to have in the API (const generics, existential types...).
So you're free to use ndarray, it's just that I would recommend keeping most usage internal.
(I looked at hdf5 by the way, and it appears that there are viable alternatives to the Array functions for working with a flat Vec and shape (via Container and DatasetBuilder); so my earlier criticism may have been unfounded)
Is ndarray highly experimental? I don't think so myself.
We do increment our 0.x regularly - once or twice a year, maybe! - due to mostly minor breaking changes. We do find good reasons to have some breaking changes, and if I would point out one reason, it would be that the whole numeric ecosystem in Rust is immature - and there is a lot left to build and find the best rustic expression for. For example, we need to bump our version every time blas-src or num-complex bump theirs.
One of the causes here - in this thread - is that given the situation mentioned " ndarray >=0.13 and <0.15; numpy requires ndarray >=0.13 and < 0.16" - cargo doesn't try to resolve this to a single version compatible with both. This is just how cargo is for now, so the user has to pick a version.
My conclusion is mostly that range versions don't really work well for this purpose, and should be avoided.
I'm also sorry that ndarray version bumps cause trouble - it takes time for version upgrades to propagate. And unfortunately, even if we want to avoid breaking changes, we already have breaking PRs filed against ndarray again - that are updating our public dependencies to new versions
