Using AI to generate Rust code

21

People keep saying this. I don't see the "hockey stick" pace I would expect to see from new technology, just the linear increase you would expect from increased investment. Yes, GPT models are say, 1000x more impressive now than the initial versions. They also have had 1000x (or more!) money spent on training them and take 1000x as much compute to run.

If anything, it's "NPU" compute hardware (throw more transistors at it!) that's been the real MVP here.

I don't think we're going to see actually impressive results on the software side until the investment drops out and these companies have to actually innovate rather than just throwing money at the problem.

22

FWIW I wasn’t asking rhetorically or hoping for a philosophical answer.

23

Sure. And let's not forget: after asking huge investments from gullible parties, which have no chance of giving a positive ROI. :smiley:

But that'd correspond to the definition of the "irresponsible" bit I mentioned. I doubt there would be any gain, and I'm pretty sure that's not what the OP had in mind, either.

24

Problem is: What do you do when the AI has generated code that you don't understand and you can no longer fathom what is going on?

25

Development have already stalled. If you look on different models you would see that they have achieved approximately similar results… none of them have shown any breakthroughs, that fixed any flaws they had 20 years ago.

They still have the exact same weakness they always had: neural networks are often wrong but never in doubt.

Define “excitement”. Development of AI today is similar to development of the Web at the end of the last century: it would completely transform everything… but that sock puppet would become a history first.

Similarly here: there would be huge amount of excitement and failed attempts to use AI in the next few years, but to really transform the world… we would need the next breakthrough.

Internet needed WiFi, 3G, and smartphones to really “take over the world”, we don't know what AI would need, but we know it wouldn't arrive this year or next year.

Sublinear, by now. GPT-4 was a breakthrough, 2 years ago. We still have nothing that OpenAI may call GPT-5. And they are anything but cautious in the praises for their models.

Yup. Today all the money are funneled toward scaling… which is the dead end, it wouldn't work. I think only Google,

Bubble would have to burst before people would start thinking about how they would have to proceed toward AGI.

Today we have only managed to actually model the “outer parts”, the subconscious thinking. And even Neuro-symbolic AI add instruments that “subconscious neural network” is supposed to use.

This may never work, just like neural networks fake everything else they would fake the use of symbolics instruments, too! Today ChatGPT 4.5 confidently adders that it run your python program and get the results even if it has no means to do that!

Conscience is much smaller and weaker part of human brain, yet it drives the subconscience… and our AI doesn't have subconscience, yet.

P.S. The only thing that we have discovered is that AGI/ASI definitions and Turring test are all deeply flawed. They were based on the assumption that AI would develop into agent that's roughly similar to human (AGI)… and then would surpass that. But in reality we got AI that's wastly superior to human in some areas and equally deeply flawed in other areas. That means that AGI is not even “near”, it's, in fact, not possible in princple. If there would an AI that may do everything “a normal person” can do… it would also be crazy superhuman in most other areas… this is something vew few “forward thinkers” were preparing for. Even if they claimed they did.

26

Some time this year you should be able to review the Internet Computer's Caffeine AI tool that promises to do sort of what you say, but specifically within the Internet Computer and using their Motoko programming language in the first instance (Rust is their other current platform language). The Internet Computer is a decentralised serverless cloud where software contains business logic and UI only. Traditional cybersecurity and backend is abstracted away in effect, by cryptography and "orthogonal persistence."

27

Where do you see “philosophical answer”? AI, today, semi-randomly “looks” on parts of its knowledge bases and on parts of that input that you gave it. And then it does 50, 500, 5000 steps that all have very high probability of success. 99% rate or so. Similar to what branch predictors have, these days.

Except… 5000 steps with 99% probability of success gives you correct answer with probability 10⁻²² – if one mistake breaks the whole “thread of logic”.

That probability, practically speaking, is indistinguishable from zero, And that's “lots of step with high probability of success” is where all the issues with AI come from.

Thankfully out codebases are not made in a way where one mistake in reasoning may break everything, thus usually you get something useful even if AI does mistakes in these 5000 steps… but you get the idea, right?

And AI is really impressive at picking relevant parts of knowledge from their “head”, pre-trained set. They may also pick “important” parts from your large codebase. Really impressive stuff.

But it's all “vibes”, from the top to bottom and from the bottom to top. Remember how neural networks achieved very high performance on distinguishing military cars from civil ones by looking on whether car is in the middle of mudball, while civil cars are neatly moving on the roads?

That's how AI looks on your codebases. Parts that are covered by lots of tests are probably important ones, parts with comments are, most likely, a tricky ones and so on. Names of variables and functions are #1 source of truth, comments are great, looking on the code is not important.

Imagine how you, with your experince, would help someone who have program in a language with radically different design, like a programmer who have no idea how functional languages work, who coded in C for 40 years, would help someone to write program in Haskell.

You wouldn't look on the code of your program, you couldn't understand it. But you would look on shape of the project, would look on where are the things that are looking dangerous (if even judging from the comments attached to them), etc.

That's how AI acts today. Both on small codebases and on large ones.

My answer wasn't rhetorical or philosophical one. It was very practical.

If you ask whether AI may look on codebase that's many megabytes in size and impressively combine knowledge that's spread all over that codebase… it can do that, easily.

But there wouldn't any attempt to follow “the thread of logic”. Not on a tiny code and not an your large project.

If you ask if you can make it not to forget about some “important” details… it couldn't do that even with medium sized prompts and, of course, the more data you give it, the worse it performs.

That's total opposite of human where you may give it tests, lints and design docs and this would lead to deeper understanding and less errors.

With AI it's the opposite: the more you “push”, the less you get back, the more mistakes model does and the more chances of producing garbage instead of meaningful answer there is.

It would. As I have said: it's really impressive at pulling interesting facts from your codebase (and from AI training set, too). You can bounce your ideas against AI, you can ask it to show the rough shape of code that does something that you need in one toolkit or another… it's ultimate “rubber duck”.

But it's not reliable, it couldn't be trusted and thus it couldn't be used to build something on top of it.

Think about how Google search and then Stack Overflow changed the way we code. AI would have similar effect.

But these naïve assumptions of management that they would, soon, be able to fire their coders and replace with AI?

Well… “pump and dump” startups may do that. Anyone who needs reliability? Nope.

28

Okay, I am using Ai for coding for the past six(!) years.
I could tell some boring history from 2019 with the earl stuff but here is my honest opinion:

Ai for software engineering is well on track to become similar in spirit as CAD for mechanical engineering.

Why?

There are currently quite some limitations mostly around workable context. I wrote an it a while ago:

Meanwhile I moved on from Windsurf to RooCode leveraging the emerging multi agent paradigm. Things became quite seriously with the arrival of Gemini 2.5 Pro and the subsequent cheaper Flash model.

I am currently wrapping up a non Rust Project That was entirely writen by Ai, like a straight 100%. When you take a moment to read the article I have linkes you may realize that better specs leads to better code i.e. garbage in, garbage out and this makes intuitively sense. However, in the current project that is in the handover stage meaning transferring it to the client infra because all tests and acceptance steps were completed, well get this, even the requirements were written by Ai...

Before anyone pulls out the torches to start a flame war, a few important things:

  1. I was 100% transparent and upfront to the client that this is going to be an Ai experiment. The client, actually, didn't care and basically just said deliver the thing and we're good.

  2. A second security audit wraps up and, tbh, slob is less of an issue than security. Had to fix up a lot. That said, even after the security audit, the project still was well ahead of the projected timeline. Ultimately decided to add layer 7 security measures just to be double sure.

  3. In general, I see a major shift towards what I would refer to iterative requirements engineering. While is true that most users and clients are not exactly good at articulating complete requirements, usually they are very good at saying that they like A better than B or if you could add X. And again, ask the Ai to write proper specs, double check and let the code, test and DevOps agent do the rest.

For Rust ...

I think we need a bit longer to get to a similar experience. Mainly because. Rust is used in more advanced projects with higher complexity and quite often less precedent. Just take the Surreal DB project or the Iggy project as an example. There is nothing quite like these so obviously your Ai has no real precedent to lean on.

On my main Rust Project which meanwhile hovers around a 100 crates or so, Ai still use Ai selectively i.e. to implement a specific requirement document either within a crate or if it is bigger then I let the Ai write the entire crate.

Macros work remarkably well especially if you give the Ai a fictious usage example. It worked every single time for me.

Currently, Ai for Rust still falls flat on coherent architecture. Also, it makes a ton of assumptions i.e. everything networking is Tokio async so I have to be specific when it's not.

Are these deal breakers for me? No. Annoying at times,yes, but I still consistently produce about 4 to 5 times more Rust Code that is green on the CI than without Ai. Impossible to argue against that gain. However, without those specs /requirements documents feed into the Ai, I would drown in slob.

Ultimately, it boils down to your project style and requirement engineering process. I had to adapt mine and it's still learning. However, working with Roo Code and the new models especially Gemini 2.5 is so above and beyond anything I have seen in the last six years that I concluded that Ai assistance will, just like CAD, became staples and manage the bulk of all code.

Senior engineers and those with excellent training will do well. However, I am sincerely worried about self learned coders and especially full stack engineers who know a little bit about everything but nothing really in depth. It's hard.for them because no vibe coding in the world will help them to diagnose when the Ai is wrong. For that, you will always need a senior engineer with training and experience.

Hope that helps.

29

How can you say that when they are the exact same thing?

Have you done that by changing code manually or prompting AI to do that?

If there was a projected timeline then, most likely, the structure of the project and requirements were relatively clear from the day one. And you were writing yet-another-CRUD-website-with-subtle changes (or something equally mundane, done thousands of times before).

AI is really impressive at “doing the same thing in a different guise”. It does bazillion tiny mistakes here and there for human to fix, but like it may transform real photo to the Miyazaki-like picture it may transform one style CRUD code into another style CRUD code easily. If it saw enough examples of both styles.

You are right: in an area where there are no engineering challenges whatsoever, but where the core issie is an attempt to understand what the customer actually wants AI should work fine (even if they wouldn't be able to write correct code, but human may fix after them since there are no engineering challenges).

Sounds more and more like some CRUD-like project with close-to-zero engineering and lots of attempts to understand a disjointed stream of thoughts that the customer calls “requirements”. AI is pretty good when dealing with these, maybe actually better than most engineers already.

I wonder if people would learn to measure productivity in something meaningful, not in the lines of code.

If you produce 10000 lines of code with AI and I produce 1000 lines of code without AI then you are not 10 times more productive than me if we have solved the exact same task.

I would even say that if you have produced 10000 lines of code to solve something while I produced 1000 then you have added more future problems for the project.

Who wrote these specs/requeirements, then? In my experience writing spec is 90% of work, AI or no AI. And my own productivity is much higher when I work alone then when I have to direct others simply because I can skip that step.

AI is pretty good at extracting specs from written code, though (when I need to satisfy the rule that new features have to have specs).

Yes. And like with CADs it would take decades for the newcomers who use CADs to become more productive than “old guard” who knew how to do designs without them.

They would do fine if they would learn to leave companies quickly enough, before the whole house of cards that they are producing would dissolve on them.

Whether that's “responsible” use of AI or “irresponsible” one is debatable…

P.S. It doesn't mean that AI is not improving. It does! These days it's able to convince you that what it producing is worthwhile, that it had run code it wrote on Macbook and copy-pasted the result, that it made a typo when it copied text from said Macbook to the asnwer and then “lost” the proper number… it's all is really fascinating and very human-linke… only… instead of useful helper AI companies are turning AI into an ultimate fraudster… still useful, for some things, sure, but, well… is it what you want in the tool you plan to use to write code? Really?

30

I did it once. Cheating to learn Rust by converting JavaScript to Rust, now I want to improve the code. Ain't perfect. Either way humans are involved.

31

In every domain.

I saw an artist train an a LLM to draw like them, and it fell down on composition

32

I'll show you:

I am familiar with the philosophical arguments for "what AI actually is". Stochastic parrots, "Chinese rooms", et al. These ideas are uninteresting and irrelevant.

We are talking about computer programs. They are 100% deterministic [1], you have to add randomness (temperature, top-k/top-p sampling, and so on) to disrupt the inherent determinism in digital computation. In systems thinking, complexity arises from the combination of simple rules. That's enough to dismiss the philosophy and focus on the practical and pragmatic considerations.

"Keeping the thread of logic" can be misconstrued as having a philosophical undertone, I'll concede that much. My intention is to better understand what is reasonably possible for the performance of short-term memory with a longer context window. Everything else can be left aside.

But it's impractical because trying to describe how an AI does its reasoning is irrelevant. I didn't ask how it works under the hood. Having a solid common understanding of how a neural network's capabilities emerge doesn't affect my question in any way.

So, I'll reiterate my question, hopefully without getting derailed again. This is addressed to anyone, not to the person I am replying to:

Assuming a 10 million token context window is enough to hold an entire reasonably-complex Rust project (let's say, 20,000 LoC), what is the qualitative performance of such a model in practice? If anyone here has used Llama 4 Scout or a similar model with a large context window, please report your findings. I do not care about what you believe might happen. Actual experience only. Benchmark it, don't guess!

The way I see it, the funding is the only driver of true innovation. This much is clear from The Bitter Lesson [2]. What I find remarkable is that it's precisely the same point that John Carmack made about computer graphics, at least as far back as October 2004:

Steven Kent - Have you started a laundry list of new features for your next game engine?

John Carmack - There are a bunch of things that are analytically intractable problems... things like soft shadows, proper anti-aliasing, motion blur, order-independent translucency. These are things that there aren't closed-form solutions for arbitrary environments. You can do tricks to address any one of them; but I pretty strongly believe that with all of these things that are troublesome in graphics, rather than throwing really complex algorithms at them, they will eventually fall to raw processing power.

People write research papers about, "Here's a really tricky algorithm to do that," but it doesn't work in all cases. People who are synthesists and think of those complex algorithms, they really pooh-pooh that. They don't like to hear that because they want it to fall to cleverness rather than raw power; but the way things have consistently, undeniably fallen over the years is to raw power.

It would (still) be a mistake to underestimate the power of more processing capacity. And funding is what tends to make progress in that area. Said another way, scaling is the only thing that matters. It was true in the 70's during the first AI winter, it was true in 2004 after Doom III, it's true today in 2025 with AI that has phenomenal capabilities compared to what came before. And it will remain true in 2050 when we will still scarcely believe what these system can do.

Will it be ASI? No. Will it be AGI? Yeah, for some value of AGI. We already have nascent AGI, and it's objectively already useful and helpful. Just, not in all the ways that so many people hope.

This is a hard one to compare because it's quantitative, but I can't see AI being able to produce 4 or 5 times more "code that is green" than I already do. Part of it is that I don't have a velocity problem. I produce exactly as much code as needed, and it's produced "plenty fast enough." I think I am personally more interested in qualitative measurements. Does it produce good code? Not "does it produce code fast?"

This is probably a difference in perspective. I'm self-taught, and I know when to ask the AI to clarify why it made a specific decision, and I know that I can probe it to make corrections when necessary. I don't see anything wrong with letting new developers explore and learn from tools. It would be gross negligence to let them build critical infrastructure without appropriate guardrails, but that's a management problem. The ingredients necessary for a student to grow into an experienced senior engineer are acuity, curiosity, intrinsic motivation, and enough skepticism to ask questions intelligently.

And I have seen "vibe coding" work for an inexperienced hobbyist with my own eyes - a close friend of mine, in fact. They have all of the ingredients and none of the experience. I do not have the slightest worry that their project is going to be used for the flight control systems at Boeing! And if Boeing does use it, we're all doomed to die in a fireball falling to earth, anyway. The project is a mod management system for the Vintage Story game [3]!

There's an ocean between critical systems and hobby projects. All of the best engineers in the world started with a hobby. This is something to embrace and encourage! It's imperative. As a teacher, I reject shunning vibe-coding as much as I reject shaming people for learning with Scratch. These are good tools that can be used responsibly!

  1. At least when running inference on the same inputs on the same machine. We can rule out differences between floating point implementations between hardware vendors and such for the answering the question of "what it determinism, anyway?" ↩︎

  2. The original is available without HTTPS: The Bitter Lesson ↩︎

  3. Do you see what I did there? I lead with something that I have great concern for (safety, particularly in industries where small errors are unacceptable) and the punchline is that the project in question isn't that one project maintained by a random person in Nebraska. xkcd: Dependency ↩︎

33

Yes of course hardware scaling matters. We give all the credit for the current AI boom to transformers, but they're basically not that different to perceptron designs from the 70s, just now we have the hardware to run them.

The problem isn't that we're waiting for the hardware to run the thing we'll get whatever "good AI" we happen to be thinking of, it's that we have absolutely no idea what that actually looks like, in essentially any form: software design, product interface, test suites, and of course safety. It's active research to figure out what we want to figure out.

Compare to your Carmack quote, where he has the actual, working examples of offline rendering showing exactly how much compute needs to happen to do it "right". He was right: the solutions we have now to soft shadows, AA, motion blur and transparency needed a lot more hardware than they had in 2004. But also, he was quite "wrong" (if you take him too literally): we had plenty of "really complex algorithms" we throw at all of these to get good results without actually doing all the work, even before getting to real time raytracing, which is the ultimate expression of what he's alluding too here; and even that is extremely heavily cheated currently (by many orders of magnitude!).

It's an old saw nowadays that we expected real time raytracing to only be a thing in 2050 or later, but it turns out once you have both "enough" hardware power and something to start from, smart people can find tricks to skip most of the work to get what we actually want, rather than what the maths implies. In comparison to AI - we may well have the "enough hardware" but without the equivalent to the offline rendering there's no direction for all this research to go to be expecting actually dramatic results.

And that's where the "money problem" actually is: not that people get smarter when they don't have money, it's that the expectation is that we're on the hockey stick of a new technology like we've seen before, when we're not - until there's actual fundamental breakthroughs we're going to see pretty mediocre results for ever increasing costs at best, and at worst all the investors pull out, AI becomes even more toxic a marketing term than it already is, and nobody invests in AI research for decades.

34

You keep talking about hockey sticks, but that's marketing. I'm interested in real technical discussion, not hype and not philosophy.

I think you might be undervaluing that there are already goals for AI researchers to strive for. Much like Carmack's working example of offline rendering, the hard problems not yet solved with AI are things like reasoning about large code bases, hallucinations, designing sane specifications, testing accordingly. These are all concrete real-world problems. I don't see what about these are not the equivalent to offline rendering, as if there is no direction whatsoever.

35

... Ok read "sinusoidal" instead then. Same thing. An actually new technology getting investment will get superlinear results to that investment. Heck, we're
still getting superlinear results on the transistor, 70 years later!

And no, problems with an existing technology are not goals to work towards like offline rendering. Offline rendering is to real time graphics a test case, a proof of concept, reference, etc.. AI has exactly none of that. There is absolutely no upper bound on how much compute it takes to be a "good AI" outside of extrapolation from a human brain, because we have no idea what a "good AI" even is, outside of the human brain.

Even your examples are all essentially the same thing: "it doesn't do what we want" - and all for the same reason: "we don't actually know what that is"

But this is now so far off topic I won't continue the argument further, I've more than made my point.

36

We don't have a hardware to create a reproducible benchmark, sorry.

Hardware that may 100% deterministically run “big” LLMs like Llama 4 without reasonable timeframe simply doesn't exist. You would die before such experiment would be finished.

And yet it still fails to do things that human learns pretty early: it couldn't stop and say “I don't know what to do”. It can only forge forward and produce garbage when good solution is impossible.

That's the main reason we only have ChatGPT 4.5 and don't have CharGPT 5: not even OpenAI is ready to call what they have ChatGPT 5… and they are not known to be shy about promises.

By 2050 we would have another AI winter and, maybe, even revival. AI may even achieve AGI (and thus ASI) level by then.

Not guaranteed, but possible.

Yet this wouldn't happen with approach to models that we use today.

AGI is not possible. It's pointless to talk about AGI because AGI would just simply never happen.

AI, even today, may do some things that humans couldn't achieve without outside help. Yet it fails spectacularly at many things that human perceive as “simple”.

Thus AGI is a pipe dream: by the time when ASI would finally be able to solve all the tasks that “average human” may solve it would, by necessity, perform most other tasks drastically better that human.

We are not even close to that point.

Obviously not. And it would never learn to produce good code. Note how even @anon90448620 , who is extremely bullish on AI, admits:

And, well…

That's the whole selling point of AI, at least today: yes, it does mistakes, yes, code doesn't work, yes, human have to still review what was created and fix bugs… but look on how much code was generated and how quickly!

And if you change the requirements then it's all regenerated equally quickly! In areas where you don't need good code and your main problem is that you need to generate bazillion versions of code to show the customer who have no idea what they want. Again, the quote:

The goal of AI-assisted coding is not to write a good code (that's not in the cards), but to produce lots of variants that would help to glean, somehow, actual requirements for the code from the customer.

Entirely different kind of programming from what you, probably, have in mind.

IOW: everything that human have and AI doesn't have. Precisely what we are talking about here.

We are very close to this point, I suspect But much would depend on how many things far outside of AI sesearch would happen… in fact I suspect AI would soon get a huge boost, just not in a area of LLMs and/or coding but in an area of “how much intelligence can we squeeze into a tiny drone that we need to have a chance to survive coming wars”.

These are concrete real-world problems but, unlike 3D rendering, we have no idea how to address them in principle.

That's the biggest difference: when Carmack was talking about 3D rendering we already had working, if not practically usable solution: offline raytracing. That gave us benchmark to compare.

Carmack already had working, if not usable solution, when he wrote what he wrote.

With AI… no one have any ideas about how these these “hard unsolved problems with AI” may solved, even in principle. We just have a hope that if we will throw trillion dollars or, maybe, ten trillions then all these issues would magically become solved… not only we don't have working solution, we don't even know if working solution is even possible on that path that we have chosen.

THat's pretty big difference in my book: to have working solution and saying that we just need more hardware to use it and software tricks wouldn't have is one thing, to not have a working solition and still say the same… it very different thing.

37

But, we do know what we want. Write correct software. Design specifications without design errors. None of this is controversial. Humans aren't particularly good at it, and statistical methods may not be any better. But to dismiss these unsolved problems as "we don't know what we want" is either outrageously disingenuous or my perplexity comes from a miscommunication on how you define "what we want".

38

To clarify: "be able to describe in a formal manner the difference between the things we want and the things we don't want"

There exists no test that can distinguish between good code and bad code, just like there's no test to distinguish between an AI that will turn us all into paperclips and one that will make us coffee in the morning. It's entirely us looking at the output and seeing if it's "what we expect" - whatever that means.

39

Does it mean that if we would replace what you wrote with But, we do know what we want.Teleporter to transport people to USS Enterprise. And a good design for the warp drive. None of this is controversial. we may explore the galaxy in next 10 years if we would throw enough money on AI?

I kinda, sorta doubt it.

Miscommunication, at least it looks like miscommunication, to me.

Carmack's case: what we want is perfectly doable and we know at least one way to do that – just not in the required 16 milliseonds.

AI case: what we want is a nice thing to have but we have no idea if these things are any easier to achieve than working warp engine – and the only reason we believe that we may achieve it at all and the reason we hope to achieve it with with mindless scaling are “vibes” from some of the people involved.

It's dot com bubble all over again… only this time result would be much worse because so much is predicated on AI succeeding where it's not likely to succeed any time soon.

40

The entire basis of these arguments is "humans don't know what correct code is, thus there is no way to make progress toward it."

I don't know how to address that interpretation! We do know what correctness is and we do know how to construct correct systems. Formal verification is a big part of it, and it's usually just reserved for very expensive, high-risk projects and applied after-the-fact. Its use is almost completely absent outside of hardware designs and a handful of other sectors.

I accept that many people disagree with the importance of correctness, but that is not the same as generally not understanding what it is.

AI is just a tool, like your compiler. And like your compiler, it might be wrong sometimes. It doesn't matter in the slightest if these "have true intelligence". What matters is whether or not they are useful. And they are, without a doubt, objectively useful tools.

Implying that correctness is not perfectly doable and that we have no idea if it will work is absurd. It already works. The tools for it are just bad and not integrated into languages or compilers. Anything that makes the integration tighter is a win, even if it's having an AI assistant do the monotonous work of writing the assertions for verification. But if it can just write the code to the specification and prove its correctness at the same time, that's even better.