Rust-rdkafka crate update with librdkafka 2.6.0

community
1

For my app, I'm using latest rust-rdkafka crate available on crates.io, which internally uses an older librdkafka:2.3.0. There are new versions of librdkafka e.g. 2.6.0 that fixes multiple issues related to recovery in admin leader election, fetch errors etc. I could upgrade the rust-rdkafka (which depends on rdkafka-sys -> librdkafka:2.6.0) but cannot publish to crates. How can I contribute/get necessary permissions? **
---
Uploading rdkafka-sys v4.8.0+2.6.0 (/Users/akundu/rust-rdkafka/rdkafka-sys)
error: failed to publish to registry at crates.io**
Caused by:
**the remote server responded with an error (status 400 Bad Request): A verified email address is required to publish crates to crates. **
---

I had forked the original fede1024/rust-rdkafka and I'm ok with publishing the crate into my forked repo's package repository. I tried updating my local ~/.cargo/config but it doesn't seem to pick up. Also, not sure if github supports rust crates natively.

Alternately, I can also publish the crate to my company's jfrog artifactory, that offers native rust package register. But I'm unable to configure the app's Cargo.toml to reference artifactory and crate. If I include the local crate folder without version, it does not work. If I specify version, it fetches from crates bypassing my upgraded library.
----
#rdkafka = { version = "=0.36.2", features = ["cmake-build", "ssl"] }
rdkafka = { path = "crates/rdkafka-0.36.3", features = ["cmake-build", "ssl"] }
---

Can someone provide pointers to which is the correct way and something I can setup in CI?

2

Please correct your formatting, it's hard to read all bold text.

These chapters from the Cargo book explain how to specify dependencies from different registries, git repos or local paths:

https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#specifying-dependencies-from-other-registries

What you probably want to do is override the original rdkafka-sys dependency with your updated fork. The maintainers of the package might also be interested in your changes and maybe would appreciate you making a PR in their repo.

1 Like
3

Thanks. the configuration does help but I get a 403 now and will follow up with my admins. I can manually Publish from UI but with token/API key, I'm unable to publish using cargo.

I had reached out to maintainer but he is not involved any more. I'll definitely create a fork MR.

Uploading rdkafka-sys v4.8.0+2.6.0 (/Users/akundu/RIV-06284/github-projects/xxxx/rust-rdkafka/rdkafka-sys)
error: failed to publish to registry at https://jfrog.goriv.co/artifactory/api/cargo/xxxx-cargo-local

Caused by:
  failed to get a 200 OK response, got 403
  headers:
  	HTTP/1.1 100 Continue
  	
  	HTTP/1.1 403
  	Date: Sat, 02 Nov 2024 23:16:26 GMT
  	Content-Type: application/json
  	Transfer-Encoding: chunked
  	Connection: keep-alive
  	X-JFrog-Version: Artifactory/7.49.10 74910900
  	X-Artifactory-Id: fe9f4d03882e66cd73370006daf51e32b9e7d53b
  	X-Artifactory-Node-Id: art2
  	
  body:
  Operation failed for path 'crates/rdkafka-sys/rdkafka-sys-4.8.0+2.6.0.crate'
4

Hey did you ever get this resolved? I am running into the same exact error with my company's jfrog artifactory.