Pre RFC: We need a process for giving crates to new maintainers


#23

I would be more than happy to have everything go under that group if that’s what people wanted! I think we should create a new repository, but that should be totally doable.

Sorry I wasn’t more clear – I was suggesting like 3 solutions simultaneously, but this was one of them that was in my head. The discussion helped clarify for me that this was the path forward.


#24

According to the docs, rust-unofficial is not for abandoned projects; they need maintainers to move in there.


#25

The problem can be sidestepped and the cargo/crates community improved if crates had namespaces. com.github.seanmonster.requests is unambiguous, demonstrates the provenance, does not need to worry about ownership changes, shouldn’t be an issue for legal action and also permits forks to coexist with live projects.

Unfortunately, I don’t know how one might reverse engineer it into the existing system. Maybe all existing crates are defaulted to imply io.crates.pkgname


#26

Fortunately, cargo already has a feature by which you can host your crates at any DNS path you want, using git as the package exchange protocol. Since Github provides a free public git server for anyone to use, anyone who is unhappy with the chosen namespacing solution can distribute their crates at DNS-resolved name by hosting them on Github, or at another DNS-resolved name by setting up their own git server.


Here’s my take: namespacing crates.io gets constantly relegislated because it is a social issue, not a technical issue. And like many social issue, it doesn’t even have a clear problem statement. What exactly is the issue that namespaces are supposed to solve? The fact that every crate needs to have a unique, memorable identifier doesn’t go away at all. seanmonster/requests is still not as memorable a name as requests/requests. com.github.seanmonster.requests is not as easy to recall as com.requests.

My opinion is that people want namespaces because they want to be able to choose any name while performing the act of naming. It makes people sad or mad that they can’t name their libraries whatever they want, and instead they have to come up with a name that isn’t their favorite. This is understandable.

But crates.io is not optimized for naming your crates, its optimized for finding crates to use. Distingushing between 3 requests libraries each of which is paired with an arbitrary username is not easier than distinguishing between request, requests, and reqwests. In my opinion its far easier to remember "I want the one with the w" than it is to remember seanmonstar’s username. This would only be compounded once there were 20 or 30 request crates under different namespaces. An ever better example of this is html5ever, which is way easier to remember than the username of whoever maintains it (I don’t even know!).

Myself, I had a library for which the name I wanted was already taken by a library doing something completely unrelated. So I came up with a new name, and now I like the new name a lot more.


#27

Thanks for the prompt reply, @withoutboats. I agree that it’s largely a social problem but I think cargo.io is generally a social platform for sharing code so it’s there to solve social problems. But that’s semantics. Let’s get down to details:

I agree 100%. This solves most if not all of the technical issues of using crates with name clashes. However the solution is basically ‘don’t use cargo.io’ and I feel that it’s not very satisfying in a thread about fixing cargo.io to overcome naming problems. But, indeed WONTFIX is a valid outcome. However, there could be[1] some social signalling in hosting packages outside cargo. Also, there are some terrific jobs which crawl all/most of cargo.io and rebuilds crates to make sure they build with changes to the compiler. Allowing solutions which force crates outside cargo.io risks increasing the percentage of code not part of those tests.

Hypothetical issue 1:

I work at Gothra. We write some server side code which is kept in house. We have our own cargo mirror so we are protected from cargo availability issues and also to stop side channel information from leaking (i.e. anyone with access to the stats can’t say “golly, Gothra sure is interested in the Frobnicator and Sozzleflange packages. I wonder what they’re building.”). We write a package called hammer and it’s great. It took us a year and five engineers to build. We’d like to release it as open source on cargo, but in the past year, someone posted a package to cargo, also called hammer, which crawls the web downloading pictures of hammers.

If we want to use the collective repository without namespacing, everyone is eventually forced to choose Apache style names. Or we just take the C route and name everything gothra_hammer, gothra_saw, gothra_spiritlevel, etc. Or we humiliate ourselves and come up with a ‘cute’ misspelling of hammer.

This is not that hypothetical. I worked on a GiST implementation over three months. One week before I uploaded it to cargo, someone uploaded a github gist posting tool. So I had to rename my gist to gst. It was kind of ok since no one knows what the i in GiST stands for anyway. But it’s still a taste of the future of the current approach.

Hypothetical Issue 2:

hammer is really successful. Gothra spins the team off to be a consulting company and patches are flying in from customers and non customers. To prioritize patches, we’re basically pay to play. Since this is a terrible way to run a community, a group of people fork hammer. But now they can’t use cargo.io to host their crate unless they rename everything. But if they rename everything then they can’t merge back in later. Or maybe they can if they do it judiciously. Or they are relegated to using git paths for everything, and all the social signalling that entails.

Or maybe the committer is just not committed to fixing anything (@seanmonstar’s concern). In this case, using namespacing, people can just switch to use his version and ignore the abandonware. This means that using namespacing is an elegant solution since we didn’t need to make a Rust subcommittee for managing the abandoning of crates and blah blah blah. The community just started using the maintained one. Job done.

Memorability of names is not a goal of cargo projects, afaict. If it were then namespacing is the only solution since wanting org.mozilla.requests vs com.dropbox.requests is more memorable than requests vs reqwests. Namespacing also has the advantage of not humiliating everyone with stupid misspellings of names. It also prevents shadowing and typo squatting as an attack vector. e.g. hammer vs. hamrner vs. hamer. Which one has a build.rs which will send my credit card information to a third party? Crates.io won’t let me upload to the org.mozila... namespace so I can’t typo squat most of the packages.

I disagree. But it also doesn’t have to be with a username. Just a url/path. e.g. com.gothra.hammer. I just used seanmonster in there because that’s how one might namespace his project on github.

[1] I admit that “there could be” sounds like weasel words, dripping with FUD. But we’re trying to solve current and anticipated problems.


#28

I write Java in my day job, so I’m going to be pulling some examples from there since it does have namespaced packages. In my experience, that ecosystem does not behave in the way that your description implies it would.

  1. Apache libraries (at least the ones I’m thinking of) target the Java ecosystem, which, as mentioned above already has namespacing in its dependency system!
  2. Or you pick another name. In what way is “hammer” the ultimate name for a web crawler?

I don’t understand. In either case you have to change the name of the dependency you’re using. How is “a/foo” to “b/foo” fundamentally different from “foo” to “bar”? Can you outline a concrete example where the overhead required in the non-namespaced case is astronomically high?

Could you describe how this community transition from one crate to another is only possible if namespacing exists? I don’t see why a “committee” is required in any context. There are many languages in the world with their own dependency management systems. Do any of those have a Supreme Committee of Package Names?

People are slowly moving from time to chrono and rustc-serialize to serde without any trouble at all! (Also note that those crates are not called “tyme” and “rustc-serialise”).

This is frankly an absurd statement. Crates are consumed literally thousands or even millions of times more often than they are created. How is it not critical to adoption of a crate that people be able to talk with others about what they’re using?

“Oh yeah, I use that requests crate all the time, it’s great!” “Which one, requests with a queue you or requests with a double you?” “What?”

“Oh yeah, I use that requests crate all the time, it’s great!” “Which one, seanmoster requests, or sfackler requests?” “What?”

These are both bad conversations to force your users to have. Be creative.

How is this “humiliating”?

There are ways to name things other than changing random letters in the first name you can think of. Spring, Google, and Square all have dependency injection libraries in Java, and they are not called “org.springframework:injector”, “com.google:injector”, and “com.square:injector”, or “injector”, “injectr”, and “ingector”, but rather “Spring IoC”, “Guice”, and “Dagger”. There are at least 4 JDBC connection pool libraries that I am aware of, which are called “BoneCP”, “DBCP”, “C3P0”, and “HikariCP”, not “connection-pool”, “konnection-pool”, “connection-p00l”, and “conection-pool”.

Even when people don’t want to get creative with their library names they still name them distinct things rather than relying on the group to disambiguate - “dropwizard-guice”, “dropwizard-guicey”, and “dropwizard-guicer” are all libraries that integrate Guice injection into the Dropwizard web framework. Two of them are even made by the same company!

This what you see when you search for “Guava” on Maven central:

If I’ve heard that there’s some package named Guava that has a lot of helpful functionality, which one do I want to use?

TLDR: Naming is hard. You cannot hide from that fact by sticking a “com.github.sfackler” at the front of your packages.


#29

I confess I’m getting confused about what problems we’re trying to solve here.

I think the original problem was: A “high-quality” crate name was taken, but never used. What to do?
I think the only thing to do is email the crate owner and ask for a transfer. So there’s not much more to discuss on this point.

Then maybe the next problem was: The maintainer of a popular crate wants to stop working on it, but today has no formal way of indicating this or transferring ownership to a “holding” group
I think the proposed solution is a github org that’s dedicated to holding on to these abandoned yet important crates.

But now I’m hearing discussions about things like crate discoverability and naming conventions and namespacing, which feel like tangents to the original conversation :confused:


#30

Do names matter or not? Gothra hammer’s lead developer’s grandmother had recently passed and he has a lot of memories of building things with hammers. It was chosen to honour her. Or it was picked out of a hat. Who cares. They want the name hammer. They can choose gothra-hammer so there’s no problem as such. And that’s the C approach I described: prefix everything with your org or owning project name. This is also how yum/dnf and apt work. python-pip, texlive-fonts-extra, etc.

It’s not different but it no longer has any provenance.

Maybe I misunderstood what @vitiral was doing with his new rust-crates group, but it looks like a committee to handle the abandoning of crates. I’m certainly not arguing in favour of such a committee.

[quote]
This is frankly an absurd statement. Crates are consumed literally thousands or even millions of times more often than they are created. How is it not critical to adoption of a crate that people be able to talk with others about what they’re using?[/quote]

“Hey, stop using print statements in your server code. Use a logger. sfackler’s is my favourite. I use his log4rs package a lot. He even added some cool features to change log level at runtime. Check it out!”

“Ok! sfackler has so many cool packages. Let’s see, (typing) sfackler.log<tab>, oh there it is, sfackler.log4rs. This IDE’s integration with crates.io is great.”

OR

“Ok! Let’s see, I go to crates.io and search for “logger” but I don’t see log4rs on the three pages. Are you sure it’s a good one? If it was then it would be on the first page of results with tens of thousands of downloads, right? Ok, I see it when I search for log4rs specifically. Gosh, I wonder how I would have found this without you telling me the very specific name.”

That’s fine. At least they chose those names and didn’t have the names sniped from them.

I agree. I guess 80% or more of the value can be had by just prefixing packages with some project or organization name (apache-maven, apache-hadoop, etc). The open concerns can be typo squatting with malign intent and the possibility of IDE integration/discoverability. These aren’t required.

Thanks.


#31

This could be improved if the search on crates.io would also include (as an option) the tags and the description.

Then (using your example) every crate that has to do with logging would have the tags log, logger, logging and the actual name of the crate would not matter.


#32

crates.io search does include the contents of the description, readme, and tags.


#33

Of course they matter - they’re the string by which the people using the thing you’ve made identify it! The name hammer may have some sentimental value for you but ~none of your users will know or care.

And how Maven works as well! org.apache.commons:commons-math, org.apache.commons:commons-lang, io.dropwizard:dropwizard-core, io.dropwizard:dropwizard-metrics, ch.qos.logback:logback-core, ch.qos.logback:logback-classic, etc.

How does namespacing solve this? com.github.sfackler:foobar vs com.github.sfackle:foobar does not seem all that different from foobar vs fobar.


#34

Just to be clear, the holding group can hold the name of Any crate, not just important ones. But yes, the new RFC aims to just allow people to abandon crates so others can use them.


#35

Another thing that might be useful is that if it was generally accepted that when your crate became essential, that you would put it in a group like rustlang-nursery so that it would always have backup maintainers. I think there could be a sense of pride in this, that you have contributed something which is so important to the community.