Publishing a package with a vendored crate but is not listed as a dependency because the vendored crate author did not publish his crate onto crates.io lol

[jymchng]

The vendor crate is rsc. I need to vendor his crate because:

1. His GitHub repo is updated to version = "3.0.0" but he didn't publish it on crates.io.
2. He didn't implement his Num trait for isize which I need, so might be just easier to vendor and add the implementation for it myself.

My own crate is typenum-consts (note the branch is feature/computer).

In my lib.rs, I have this line:

#[path = "../vendors/rsc/src/lib.rs"]
mod rsc;

This is because, the directory looks like this:

Cargo.toml
- src
     - lib.rs
- vendors
    - rsc
        - lib.rs

I cannot use rsc; without doing the #[path = "..."] attribute.

With cargo check, cargo can find "../vendors/rsc/src/lib.rs" and everything works well.

But when I want to do cargo package, I get a compile error that "../vendors/rsc/src/lib.rs" is not found.

This is because, the build directory does not have the vendors crates copied over.

So I wrote a build.rs to copy the files.

So now it successfully copies the vendors crates at the right directory, it compiles but cargo publish gives me this error:

error: failed to verify package tarball

Caused by:
  Source directory was modified by build.rs during cargo publish. Build scripts should not modify anything outside of OUT_DIR.
  Added: /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/.git
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/.gitignore
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/CHANGELOG.md
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/Cargo.lock
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/Cargo.toml
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/LICENSE
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/README.md
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/benches
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/benches/benches.rs
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/grammar
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/src
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/src/bin
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/src/bin/main.rs
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/src/expr.rs
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/src/interpreter.rs
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/src/lib.rs
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/src/parser.rs
        /root/rust_project/typenum-consts/target/package/typenum-consts-0.1.0/vendors/rsc/src/tokenizer.rs

And cargo check now cannot find the "../vendors/rsc/src/lib.rs" because the build.rs script didn't move it correctly when check is invoked, lol.

[jymchng]

And what are the implications of having unverified tarbell?

Would it hinder client of my library from using it?

[derspiny]

Would you be open to the structure

Cargo.toml
- src/
  - lib.rs
  - rsc/
    - mod.rs [from rsc's original lib.rs]

? Or

Cargo.toml
- src/
  - lib.rs
  - vendors/
    - mod.rs
    - rsc/
      - mod.rs [ditto]

?

Crates.io doesn't accept packages unless they can be built as packaged. A package which has code outside of src can't be built that way - even if the included build.rs reconstructs src appropriately, so you can't publish your crate to crates.io if you maintain the structure you have. This is a policy decision, so there's no workaround for the requirements. The structures I'm proposing meet crates.io's needs, while also allowing you to keep rsc's code clearly separate from the code you're writing.

The other alternative, if you want your crate on crates.io but also want to keep rsc's code more separate than those ideas allow, is to publish jymchng-rsc or similar as its own crate, and depend on that.

[jymchng]

Hi @derspiny

Thank you for your prompt reply.

I am open to the second structure. My only follow-up question is: can I still keep the Cargo.toml of the original rsc crate once I place vendors/rsc under src?

Thank you.

[jymchng]

I am currently using your approach.

I had tried another approach / idea, which is to git submodule add <git-link-to-rsc-crate> to vendors/rsc, then replacing the source.[<git-link-to-rsc-crate>] with replace_with = "vendors" and adding [source.vendors] directory = "vendors/ and still when trying to do cargo package --allow-dirty, cargo attempts to find the dependency on crates.io instead of first - on git then (ideally because of [source.vendors] to be redirected) to vendors/ directory locally.

Summary of (failed) Alternative Approach

1. git submodule add <git-link-to-rsc-crate>
2. On Cargo.toml, do:
   a. Replacing the source.[<git-link-to-rsc-crate>] with replace_with = "vendors"
   b. Adding [source.vendors] directory = "vendors/
3. Try cargo package --allow-dirty
4. But it fails because it still attempts to get the dependency from crates.io.

[bjorn3]

[source] is ignored for anything but the workspace root. And crates published on crates.io are never workspace roots. Also the git key in dependency specifications is removed when uploading to crates.io as crates on crates.io are not allowed to depend on any crates outside crates.io.

[jymchng]

Hi @bjorn3

Thank you for your prompt response.

Then I guess we can conclude that the only way for a library to depend on a crate that is not on crates.io is to literally copy the codes as part of the library, i.e. as a mod in lib.rs.

(Of course, alternatively is to 'help' the other person to lodge it in crates.io )

[ogeon]

@jymchng

(Of course, alternatively is to 'help' the other person to lodge it in crates.io )

Well, it's possible the maintainer may appreciate it if you help them with the parts you found lacking. It's likely an honest mistake, unless there's a good reason for things, so maybe they aren't aware. Maybe it's even possible as a patch to the released version.

It's also not a bad idea to (politely) ask if there are any blockers left for 3.0 and maybe offer some help. You can't force them to release sooner, and shouldn't rush anyone along, but you can show interest.

[jymchng]

Hi @ogeon

Thank you for your response and feedback.

You have good advice so I raised this issue on his GitHub.