The AI is extremly good in writing Rust code already

On Youtube, there are some people that ask various AI tools again and again to write a snake game in Python. Makes not much sense, as the AI might have been trained just for this.

Today, I was in need for a really special program, splitting a Markdown chapter into sections while extracting the individual file names from section titles. Not too hard for a human. But I was hoping getting a skeleton from AI, saving me some typing. But what is the result from latest GPT-4 o1? A fully working tool in the first shot, based on my primitive task description. And I made a Google search before -- I have not the feeling that exactly such a tool in Rusts already existed. I have not studied the code yet -- it might be not the fines Rust. But works perfectly for real data.

Here is the OpenAI page with the code and explanations:

ChatGPT - Rust Markdown Chapter Splitter

1 Like

Neat. But I'd reject it at code review for a handful of reasons, if it were up to me. The program you have written is likely adequate only for the narrowly prescribed cases you have tested, and contains things that would make it unsuitable for more general use.

  1. What bugs might you encounter with this program, when given an input that contains alternate headings?
  2. What bugs might you encounter with this program, when given an input that does not pad the heading indicator with whitespace?
  3. What bugs might you encounter with this program, when given an input that skips a level of headings?
  4. What was your reason for not using a markdown library, more generally? There are several high-quality implementations in Rust that can emit events or document models, to spare you the trouble above.
  5. What approach would you use when writing tests for the parts of this program? In particular, how would you test the state machine, independently of the code needed to feed the state machine inputs from a file?

"Code that solves a small number of cases successfully" is a long ways from good code, let alone "extremely good" code, as you suggest. Writing this program to handle the full range of markdown programs requires understanding the syntax of markdown and making some decisions about how to model that syntax. ChatGPT, which is a statistical engine designed to generate probable text given its inputs and corpus, does not do this kind of reasoning (and cannot be retrofitted to do so; this is a definitional problem and not an implementation one), and so cannot be relied on to generate code that is grounded in a correct theory.

As a final point, can you enumerate the authors to whom you owe credit for their input on your work?

13 Likes
2 Likes

All your points might be true. And whenever I see some tests of new AI bots, I wonder if all the smartness is based only on knowledge already available in the Internet. But applications like protein folding or Google's silver-medal at the math Olympiad AI achieves silver-medal standard solving International Mathematical Olympiad problems - Google DeepMind looks for me like some form of true AI. In any case, these tools are extremely useful and fun.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.