Convincing cargo to select a dependency in a range rather than multiple copies

I have a transitive dependency which expresses a suitable version range on a shared dependency (arrow). Unfortunately I cannot figure out a way to convince cargo to allow my expression of an arrow dependency to fit within the expressed range by the transitive dependency (delta_kernel).

[package]
name = "version-ranges-make-me-sad"
version = "0.1.0"
edition = "2021"

[dependencies]
arrow = "53.3.0"
# Specifies arrow >=53, <55
delta_kernel = { version = "0.6.1", features = ["default-engine"]}

Both my sample application and delta_kernel need to exchange symbols exported by the arrow crate, so it's imperative that they are using an identical arrow version. I am perplexed by the resolver choosing two duplicate crate versions (53.3.0, 54.0.0) rather than selecting the version which satisfies the range expressed.

Cargo unfortunately doesn't try to unify major versions and will keep making that duplication by default.

You can downgrade the dependency only for your lockfile

cargo update -p arrow@54.0.0 --precise=53.3.0

Some helpful commands:

cargo tree -d
cargo tree -i arrow
1 Like

Thanks for the info @kornel! I'm seeing that the new test crate's dependency resolves to arrow 53.3.0 but the version that delta_kernel is built with defaults to 54.0.0 and using

cargo update -p arrow@54.0.0 --precise=53.3.0

just gives

error: package ID specification `arrow@54.0.0` did not match any packages
Did you mean one of these?

  arrow@53.3.0

It looks like we (delta_kernel) might need to pursue just having feature flags for some finite set of arrow versions we want to support..

For context here's the entire example I was using which fails to build:

# cargo.toml
[package]
name = "arrow-version-test"
version = "0.1.0"
edition = "2021"

[dependencies]
arrow = "53.3.0"
# Specifies arrow >=53, <55
delta_kernel = { version = "0.6.1", features = ["default-engine"]}
// src/lib.rs
use std::sync::Arc;

use arrow::array::Int64Array;
use arrow::datatypes::{DataType, Field, Schema};
use delta_kernel::engine::arrow_data::ArrowEngineData;

pub fn does_arrow_work() {
    let record_batch = arrow::array::RecordBatch::try_new(
        Arc::new(Schema::new(vec![Field::new("a", DataType::Int64, false)])),
        vec![Arc::new(Int64Array::from(vec![]))],
    );
    let _data = ArrowEngineData::new(record_batch.unwrap());
}
1 Like

If cargo tree -i arrow doesn't show you arrow@54.0.0, then you don't have the problem any more.
If it shows a different version, like arrow@54.0.1, then use that in cargo update -p.

Ahh, this is an issue because we don't just depend on the "arrow" crate.

We have to handle all the various sub crates that are also depended upon. Kind of a mess, but for now this works to remove any trace of arrow 54:

cargo update -p parquet@54.0.0 --precise=53.3.0
cargo update -p arrow-arith@54.0.0 --precise=53.3.0
cargo update -p arrow-json@54.0.0 --precise=53.3.0
cargo update -p arrow-ord@54.0.0 --precise=53.3.0

Or in general just run cargo tree | grep 54.0 and downgrade until nothing is left.