How much code written by AI is actually correct these days?
A friend of mine (he's a junior dev at a software company, who vibe codes) keeps on saying that software industry would be soon gone. Is it actualy getting tougher for junior devs to enter the market?
I actually started studying rust, concurrency and async hard so maybe one day I could separate myself from those junior devs but, sometimes I feel demotivated even when I really love this journey : )
I suppose it depends on what you consider to be “correct”. It compiles? It starts? Successful flow works without errors and panics? Most common errors handled without panics? But the more complex code becomes the more questions about what the correct handling is. IMO it’s impossible to thoroughly review generated code, it’s easier to handwrite it. But if you’re not reviewing it how can you be sure that the logic works correctly, all corner cases handled correctly? Unfortunately a lot of people still think that the code is fine if it compiles and running - and AI can do that. But it can not really understand what you need and can implement incorrect logic even without letting you know about it until you test all possible cases. And you should do it after each code change then cause AI can rewrite half of the codebase for a simple change. And I’m not even talking about code quality which is usually quite poor.
Don’t want to offence anybody but I’m agree with that screenshot.
As somebody who is using LLMs to generate code at work on a daily basis, I can say, that some models have become quite good in generating code.
I am still critical about the unsupervised use of AI, especially the so-called vibe coding.
The way I use AI is mostly as a typist or non-human junior developer whom I always have to supervise and double-check.
I disagree with your friend on their notion that the software industry will be gone, but it surely will change. I've seen my own work shift away from programming towards a more architecture and design focus and reviewing the generated code.
In my experience the advent of modern LLMs also changed the scales between knowing how to program and knowing the domain. The former becomes less important, especially for small and straightforward software projects and the focus shifts to the latter. With modern LLMs even people without any CS education or programming experience can write software that solves a specific task nowadays, given that they know the domain very well.
There's always a version of "that good" that these have been able to meet, and there's always going to be a version of "that good" that they will never meet (in a form at all similar to what they currently look like, at least)
At the moment, my general experience with specifically Claude is that it only generates code I'm actually happy with at most about half the time, even if it may be technically correct in some form most of the time.
This is where the first problem actually comes in, though: if you're not actually reviewing every line, and able to understand exactly what it is doing and why it was written that way because you could have written the same code yourself, you don't actually have any idea if it's doing the right thing , especially in edge cases where you haven't actually thought through what behavior you would want to have. In my experience, figuring out what the best way to behave in edge cases is often the majority of the work developing: writing it down as code is at most the process though which you discover more such edge cases.
The second problem is bloat: by default AI will cause you to have between 2x and 20x more code for the same functionality: either because it's duplicating logic that already exists elsewhere instead of refactoring to make it available, making code overly robust and handing cases that can never actually happen, or simply being bad at generating elegant code. This causes it to be much harder to review, feeding the first problem, and gets you to the next problem even faster.
That third problem is that AI code is much much much better in greenfield projects <10kloc than it is in large >50kloc brownfield by default. The larger the project, the less of it any session will/can read before generating code, which causes lots of problems like the above even worse. You can attempt to fight this degradation with memory and CLAUDE.md files and custom skills and a dozen other techniques, but it's something of a losing battle, since those start to full up context too, meaning it can read even less code.
But really, the actual problem is that at the end of the day, your job as a developer isn't to write code: it's to figure out how to solve people's problems. An AI only knows what you tell it and can't provide any taste or insight beyond pattern matching the examples in it's training data.
None of this is to say these tools are useless mind you - but just like when you were grabbing code off stackoverflow before, or adding a package still, the fact you didn't type the code in yourself doesn't mean the code isn't your responsibility.
TLDR: firing developers because AI exists is really dumb
Edit: since this wasn't directly addressed:
That's actually quite likely true, but based on general vibes I suspect that any effect AI has had on that has been dwarfed by companies in general over hiring during COVID and now wanting to downsize - but take that with a gain of salt, I'm no economist.
It's not just COVID. Read the picture in post #2: yes, AI doesn't have area of expertise and yes, it often couldn't do what expert can… but junior is not an expert yet!
Why would anyone hire a junior and spend a year or two teaching him (or her) while they can spend 2 months to write SKILL.md and get the same effect?
At least that the prevailing thinking, these days.
Once AI burst… calculations would be different. But as long as it haven't burst… hiring juniors is just not something companies do since there are plenty of experts to hire.
I use AI at work from time to time; it remains at the level of a hard-working intern who'll be going to a top-flight university next year. It can do a decent job of writing code, and it's great at going through technical documentation to find out what it should write, but it has no sense of maintainability or taste, and will happily "rewrite" in a way that just reshuffles the deckchairs and doesn't make progress (because, just like the intern, it mistakes effort for progress).
When you're driving an AI to do things, you need to (a) set up guard rails in the form of tests it can't modify (noting that if the AI has the ability to change the tests, and can't work out how to succeed otherwise, it'll change any test it can), and (b) spend your time on stripping down the code the AI generated - it will generate a lot of unnecessary code, and it loves to write huge expository comments that don't actually explain anything.
And that's the huge difference between an AI and a junior programmer - the junior programmer can be taught to not need the guard rails, and to write comments that explain why, not what or how, where the AI can't. Eventually, your junior programmer becomes a senior, and that's great.
In my experience it can. My experience only being using models in harnesses interactively (I've used Warp Terminal and DeepSeek Harness). I'm not ready for letting swarms of agents to run by themselves yet. Certainly can't afford it.
But the problem is what do you mean by "actually correct"? You need a pretty good idea of what you want the code to do and be able to specify that in some rigours detail. This by itself is not so easy. When you get the code you need to know it does what it should and does not do what it should not. That means you should have a good lot of tests. Better to write your own tests as AI is sneaky and will write or modify tests to pass the code they wrote. Again a lot of work.
Then we come to more woolly ideas of "correct". Does the code perform well? I have had AI generate code that worked well but seemed to be very slow. On inspection I saw that it had missed some obvious algorithmic optimisations. Making the optimisations got the run time down from minutes to a couple of seconds.
And we can talk about maintainability. Is the code written in a way that human or other AI can modify and extend it in the future without things growing into a horrible mess.
The things that worries me now, after working with AI for a year is that I now have a lot of code that I know little about. As I say I use AI interactively and try to keep an eye on what code it is creating. But I've ended up creating code using complex crates whose APIs and use I'm not familiar with. And code for application domains I'm not expert in. I'm starting to feel very disoriented and queasy as I slowly lose control of what is going on.
That worries me too, if I'm being honest.
But...keeping in mind that
Maintainability Problem + coherent code authoring within large code bases = solution:
I believe that the maintainability problem will be solved, for the simple reason that such a solution would suit well the very Big Tech corporations building such agentic coding systems in the first place.
The 2nd problem, coherent code authoring in the context of a large code base, might simply reduce to "larger context window needed". But even if it isn't, it'll be solved by big tech for the same reason as the Maintainability Problem.
The main question is that of time line, and that's difficult to predict. But the way things have been going especially for coding, it's not crazy to think that by this time next year it'll be little more than a bullet point on some "AI retrospective for 2027" podcast.
With both problems solved, it becomes more workable to not know the nitty gritty of the code base at all, and instead work at the level of product i.e. we rather than our code move up the abstraction ladder.
At the moment it's hard to predict what the future of software development will look like, but it's also hard to predict what the future of many other jobs will look like. So if you like programming, don't give up on it too easily, at least not for an alternative that could be transformed by AI just as well.
Have LLMs become good enough to replace junior devs? On the one hand, let's not sugar-coat it - they have, more or less, and that's a problem. On the other hand, I could imagine the role of developers to shift (back, maybe?) to something more well-rounded, covering more aspects of product design. So in the end it may not be so much about juniors vs. seniors, but about who can fill which part of which role at a company.
(My personal consequence is that I'm founding my own company after many years as a software dev, because to me it seems like exactly the right time to do that. But of course in reality I don't have any idea how that's going to work out. Plus, I'm relying on my product idea not being super easy to copy - after all, that's something LLMs are really good at. FWIW, I've chosen Rust for backend and data processing because I feel I can judge the quality of LLM-generated Rust code without actually looking at most of it. That's largely about familiarity, but also a bit about Rust as a language.)
Why AI generate code can be incorrect? I did a lot of declarative programming the last two decades. All code which is behind of declarations is carefully designed and polished to work correctly mostly in any situation. What's difference of AI generated code?, it's same declarative programming with not formal declarations. Nothing more. So most of AI generated code is already written by humans and quite polished. Why most? All, but sometimes its parts stitched roughly.
Predictability. Declarative code always does the same thing with the same result. Single prompt will generate you different code each time, easily with different behaviour, results and side effects.
I'll answer with another question: How long and how many iterations does it take AI generated code to become spaghettified (Spaghettification) compared to similar code written by human programmers?
In my experience, left to it's own and its token economy unchecked, AI agents will reach certifiable spaghettification levels much sooner than human developers. I think this has to do with:
So given any singular task with regular acceptance criteria, I'd say AI agents do speed-up development. But given enough time and iterations on the same code it has been printing over and over again, the value gained from AI might turn out to be minimal considering the cost of refactoring, distillation and realignment of the code to the actual concrete business requirements long term.
In other words it also depends on how you or the organisation you're in views the codebase: long term liability/asset and a strong foundation for future development or a way to turn some profit in a short amount of time.
And what is wrong with spaghetti code? We naturally think it is terrible for humans to maintain and evolve. That assumes one does want to maintain and evolve it.
In AI world this is not so clear to me because:
If things get stuck one can get the machine to do it over again with the new requirements in mind.
Whatever you make with your AI can also be recreated by someone else with their AI. Whoever has the deepest pockets and the most tokens wins.
Arguably the latter is a case for giving up creating any software ever again, it's going to be almost impossible to make a living at it. Spaghettification then becomes a non-issue.
Sometimes I think the only thing left is hand writing "poetry" in Rust for ones own amusement and perhaps a handful of interested readers. Recreational coding that is, like recreational maths.
If AI maintained, self evolving code works as it should in every iteration and at any level of complexity, for every edge case, then what you say is true.
That's not reality though. It is a fantasy.
I tend to think about this as the fact that the least valuable part of any software product is the raw software functionality: if that sounds crazy to you, remember that nearly every major successful commercial software product has a roughly equivalent open source project that on average nobody uses.
The main value of a software product is simply the name / reputation: the "nobody got fired for buying IBM" principle. Of course, as we might find out with the Adobe clones deal soon, that cuts both ways: if everyone hates you, they will be looking for alternatives... which leads to the secondary principle, "if it ain't broke" - this is really the killer for AI rewrites like you suggest, people are looking for reasons to not adopt a replacement/update for quite a long period after switching. The moment something that used to work doesn't, or even used to work one way works another, they're furious.
So, no, I do believe maintainability is going to stay pretty relevant, and that rewrites are, while potentially quicker, still not generally a good idea.
If you replace "human" with "junior" though (as in the original question), the difference may not be that strong on average.
Spaghettification is also not that hard to spot, and then you can discuss solutions with the AI, like which abstractions to introduce, or cases where AI overengineered something because it invented requirements you don't actually have. I can see how not doing these things lead to a mess in the long term, but honestly I don't understand the people who just want to go completely blind. (It does seem to be a growing trend indeed, but time will tell if it's the right approach.)
I replace human with junior and still the code accumulated over time becomes less spaghettified and more to the point. It is done once, and done right. The review was super fast because the code is concise and AI reviewer was also used to pin-point edge cases.
The junior learns in the process (both skills, the codebase and business requirements) and AI is at the same entropy in every iteration. It was seemingly aware of the business requirements previously, but completely oblivious to them this time around.
That's a good way to spend even more tokens and time. Why would I have to do that? I would use a cheaper AI as another tool on my toolbelt. A local AI perhaps that's only trained on a specific stack and our codebase. Our data stays indoors (so it's not distilled and sold to our competitors), junior is happily learning and contributing more (thanks to AI helping them learn) and the product is shipped much more stable albeit a measure slower than wipe-coded junk.
Yes, but there are psychologial impact: junior doesn't know things and can be taught, while LLM “knows” and can not be taught. As I was repeating many we are building AI “from outside in”. Best analogy is not junior, but formerly genius developer with Alzheimer… and new models and, more importantly, harnesses built around them make it more reliable, but it's not clear if it would ever be brought to the level of regular developer.
It's really easy to spot when you are reading what LLM have produced — and ask questions about WTH this or that piece of code is doing and why it added 500 lines of code that's duplicated existing functionality instead of adding 3 line tweak to it… if you allow LLM to un unchecked then you code stops being salvageable long before it becomes working. Again: it's the same with human-written code, but as human-written code becomes spaghettified it becomes harder and harder to tweak it, you notice it and do something to it, but if you are vibecoding then it's easy to push code into state where LLM couldn't do anything, and more importantly, at this point human couldn't do anything either.
What about debugging and making minimal fixes in a released product -- doesn't the code still need to be understood by a human? Even if the debugging and fix could be done by an AI someday, there is a need to review and verify the fixes manually before putting out a bug fix release. Code that is not understandable by humans, whether created by AI or humans, is a big liability.
If two codebases both pass the same tests, and one of them is bigger than the other, the smaller codebase is more likely to be correct because programming is theory-building and thus Occam’s razor applies.
