Should we have an AI Filter?

1

What does everyone here think about a Cargo.toml option to declare the project does not use any AI? I would love to have the confidence that my dependencies were not written with AI.

Build could verify your dependencies, and a #![no_ai] macro could prevent compile if there is one.

AI use is definitely a spectrum, so it doesn't have to be binary. It could be something like

0: None,
1: Tests/Documentation, but no code itself touched by AI
2: Minimal use heavily reviewed
3: Vibe coded
4: Unknown

Obviously these would need to be more precisely defined.

What do you all think?

2

Bad idea, like the idea of not using code touched by people who don't share your own convictions.

Code, like virtually everything in our interwoven world, is literally touched and influenced by an uncountable number of people and technologies that run directly counter to your own opinions.

And from a purely technical view:

While a #![forbid(unsafe_code)] can be verified (and proven) by the compiler, a #![no_ai] is just a checkbox, which can be ticked or not. It guarantees nothing.

3

This is one of those things where I agree with the spirit of the idea, but I have a feeling that in practice it wouldn't really work.

Ignoring the fact that some people will be dishonest about their LLM use, there's also the issue of how people define where these lines are drawn.

4

Personally, I like to review my dependencies anyway on a number of points (in roughly this order until in either happy or move on) via a quick check of docs.rs, crates.io and the actual source repo:

  • author (there a few names which get an auto-pass)
  • quality of docs at first glance
  • transitive dependencies
  • reverse dependencies
  • version history & downloads
  • source code (I pretty much always take a quick look even if I feel good about the above steps)

If the source looks good then I wouldn't actually mind if AI helped. But it's usually easy to spot something that is AI-plausible but not good code. If there is an AI statement in the README then that's always a plus point for me. Likewise if it looks like AI and the relevant MD files are checked in that's also a lot more trustworthy than if they appear to be missing.

5

To @mroth's point, I do review my dependencies for the values their authors demonstrate, and in some cases do actually make decisions not to use libraries because I disagree with the authors. From that lens, in principle I would love for a way to catalogue and programmatically review those decisions.

However, I don't think a cargo metadata field accomplishes that, and I don't think the Cargo ecosystem will be improved by adding one, for a few reasons.

First, and most obviously, is that there is no widespread consensus on what "uses AI" means. The extreme positions are clearly distinguishable (someone who takes Claude's output and pushes it to Github cannot credibly say they did not use AI, while someone who only ever uses plain-text editors and doesn't even rely on autocomplete cannot be credibly said to have used AI), but the middle ground - where most real work lives - is hazy and vague. Attempting to impose a schema on it will founder on those disagreements, and the probable outcome is that only the situations close to those extremes will actually be catalogued with any degree of accuracy.

Second, why AI specifically? There are lots of beliefs and practices I might wish to know about before I use a library. AI is a hot issue at this exact moment, but you could say the same about blockchains a couple of years ago, and when the AI boom inevitably collapses there'll be something else for programmers to fight about. The metadata created to catalogue AI usage will mostly be dead data once people stop caring, while the same system will have to be expanded if people feel particularly strongly that whatever the next thing is should be tracked in Cargo. Repeat ad nauseaum, and if Rust remains successful then that spans decades of future squabbles. I don't think adding each of those arguments to Cargo accomplishes enough to justify carrying that weight.

Third, self-attestation is far less trustworthy than third party attestation. If there's a group you particularly trust to review crates for AI usage, I suspect you'll get better signals by integrating those reviews into your validation process than you will by asking each dependency author to check a box. We have prior art for that: cargo-audit reviews RustSec advisories, rather than each dependency's own bug tracker, for known vulnerabilities, and it works well for that.

Fourth and finally, some people will lie - for good reasons or bad ones - which will then undermine the utility of a #[no_ai] build gate. I cannot see a future where the crates.io team agree to review each crate's attestations for honesty.

Bottom line, I think this is a superficially-attractive idea that is not viable and will not accomplish your goals in practice. Starting up a crate AI review organization and publishing Cargo tooling to use your reviews is a better use of your time.

6

There are existing ~150,000 crates that haven't used AI, but they won't have the "no AI" marker. Even if that was a thing already, you will be waiting years for everyone to adopt it and make releases.

It's self-labelling, which means the publisher of the code can simply lie. It might even eventually get hovered up as part of boilerplate taught to LLMs and they'll print it every time they generate a Rust file.

In practice you'll probably want something more like an external anti-spam system that flags crates known to use AI, tracks authors who use AI. This will have false positives, disputes, and AI-using devs who will be upset by it.

I'd also caution against trying to automatically detect AI-generated code. There are code style smells similar to em-dash — but they're as flawed as freaking out about mdashes.

So in the end you should probably just review the source code for overall quality and decide if it's good enough for your taste.

7

With my lang hat on: I can't see us ever adding an attribute for this.

Crate metadata conventions, enforced the same way people check crate licenses today? That's at least plausible.

But we're not adding language attributes for the the same way we don't have a #![free_software_only] attribute.

8

Why?

If you're worried about the quality of the code, you want some sort of quality certification rather than judging that based on what tools (such as AI) were used to help write the code since that is a rather poor proxy for quality.

9

It's true that code quality, usefulness, lack of bugs, etc, are the most useful thing to know. But we have no tools for detecting those, and rely (as far as I know) on downloads, reputation of the author, projects the library is used in, etc.

For me, right now, given the bad experiences I've had with AI, I would like to know which projects used AI for code generation. I would add that to my list of other things about a project I'd use for evaluating it. Unfortunately I doubt that tools for identifying this are reliable. And I don't think any sort of voluntary labeling would help.

So I see this as a real problem, but not one with a solution at the moment.

10

I think there's little to add to what everyone already said.

The general idea is to have reliable libraries, and while AI is a liability, it's not the only factor. It's also possible some AI code got enough reviews and corrections to be acceptable.

The other problem is that crates not often comply with version semantics, many of them being too modest about it, and sites like crates.io don't offer filters for that criterion anyway. So there's not really another way to select libraries than by name, tags and keywords.

Bottom line, rather than relying on self-proclaimed status, I prefer to rely on a quick assessment: a look at the code, the validation code, the open bugs and pending pull requests, the doc, where it's used, a quick test, ... Sometimes it's complex for big crates.

There were initiatives in the past to classify and even improve the quality of crates, but the sheer number of them today makes that too difficult.

11

I would like to note that the core problem is slightly different to what you have described.

The primarily issue here is that people massively publish source codes on crates io, which in fact are not sources - they are purely generated content that pretend to be source artifacts. Worse, the source artifacts don't even exist unless we count LLM prompts as said sources.

This, in turn, leads to a problem of trust to the entire ecosystem. People expect that the programming codes in crate files are typically the initial source artifacts of authorship.

I understand your concerns about adding simple "no-ai" badge. It's unlikely it would change much. Moreover, once the badge receives wide recognition as the sign of quality, it would quickly degrade to noise in the Cargo metadata.

I also don't have good solution, unfortunately. But I suggest the Rust development team to think about this problem from this prospective too. Ultimately, it's a problem of the provenances and trust.

12

On my first published crate on crates.io, there is already an AI assistance notice at the very bottom of the crate description/README. Surprisingly even after the Github Repo got archived (due to framework mistake that introduced a very counterintuitive flaw, but can still be worked around), there are still some who tried my application.

I think it's only right to require AI notice for such cases like mine but the problem would be how to enforce it and how effective it will be.

13

The problem of enforcement to some extent could be resolved through the institution of trust: you simply declare certain statements, and people trust these statements because they trust you. However, this in turn assume that the author discloses their identity. Otherwise there is no one to trust.

14

Can I also have a flag "Proudly produced using AI"? I'd love that.

15

In fact, there is a risk for the opposite effect than we might expect.

In today's media agenda this might be seen as a welcoming sign to use Rust for "AI". Rust may become an umbrella realm for all Internet vibe-coders, which, I think, we don't really want. The team seems to prefer to avoid this topic. At least in public. Partially, because there is no good solution. If you don't no what to do, sometimes it's better to do nothing.

16

I agree it's a different problem, but the implications are the same. We don't add language attributes -- which need semver, which people need to wait to reach their MSRV before they can use them, etc -- unless necessary.

If people want to make tools for this that read something in https://doc.rust-lang.org/cargo/reference/manifest.html#the-metadata-table, that can all happen without anyone in the rust project needing to approve anything, so that's the default answer.

17

That's like saying that if two people have a discussion before writing code based on that discussion, the contents of the discussion must be included or else the code is not the actual source.

18

No, it's not the same thing. The code written by these two programmers is still a work of authorship of two co-authors. Maybe a part of the work but still a work.

In case of the generated content, it is not considered as a work of authorship. Such content is not protected by the copyright laws. You even don't need a license to use it.

So, in fact it's even worse than publishing compiled binaries without the code on crates io. Compiled binaries at least may have source artifact, even if such artifact is not disclosed.

19

Even if true, how is this a problem for the user of the crate?

Whether it's true or not probably depends on details like how AI was used exactly (and on the jurisdiction). If a human is actively editing code with help from AI, rather than just blindly publishing generated code, I think they have copyright everywhere.

20

Please, can you define, which group you're referring to when you say "we"?