Prototyping in Rust, versus other languages -- What's missing?

A thing I've read in a couple places: "I like Rust, but it limits my speed of prototyping, either because of it's strict borrow semantics, static type system, etc. so I prototype in Go/Python/blah" There's sortof two questions that pop up for me.

  1. Why shouldn't Rust have a quick_n_dirty (or non_strict, if you're feeling serious) attribute, like this thread from a couple months ago mused over?
    (Sub question as to what quick_n_dirty actually means, but I think a loosening of some invariants around the borrow checker and trait requirements wouldn't be insane)

This other thread explored some ideas for what prototyping could be like. Tossing everything on the heap, assuming clones, and auto-unwrapping results stood out to me. Developing guidelines for prototyping with Rust

  1. Most of the code I write is in the "must be correct, security and performance matter" bucket. I'm not sure I'd actually use a "quick_n_dirty" prototype mode if it existed. My brief experience using Golang suggests that it sortof approximates what I'd want in a prototyping mode in quick-compiling and simplicity, but is so restricted in language features that I'd just as rather start in Rust. I can see the argument for dynamically typed language prototyping more clearly, but Rust's never going to want to do that. So TLDR on that: If you prototype in another language and port to Rust, I'd love to know what that language does for your prototyping process.
2 Likes

I've been using a lot of Python nowadays (for the last 3 years or so), and it's perfect for demonstrating stuff, e.g. teaching basic algorithms to other people. Dynamic typing is also quite convenient in the cleaning phase of a data analysis pipeline when you have to work with data that is a mess. Also the language has a massive standard library.

I've tried to build larger-scale software with Python, though, and the lack of typing along with shared mutability of "everything is a pointer" classes made the attempt an absolute nightmare.

So my $0.02 would be: use Python for prototyping, and under no circumstances let that code escape to production :sweat_smile:

13 Likes

Language features are secondary at best here. The important thing is documentation, which translates into time from typing "how do I..." in google to finding page with an example. This requires documentation to exist and focus on providing quick access to solutions. With Rust, most common documentation form is dump of module structure, which doesn't work for this purpose at all.

Rust already has crates for almost anything you may need, but they often lack polish.

Also search on creates.io is showing way too much irrelevant things.

5 Likes

I don't know. I think we shouldn't open the can of worm if we are OK to have a mode where it's fine to just let users do UB by accident.

The fact that I don't have to find bugs during prototyping is a speed up for me. But it may also be because I'm more proficient in Rust than in eg. Python. Making the types right without the compiler support in there, having to re-run the thing multiple times before getting it correct and watching it crash again and again, compared to longer time of writing it until the compiler is happy and then just running it and… it works seems like a good trade-off for me.

But I do think that „prototyping“ Rust and „production“ Rust looks a bit different and most of the teaching materials are for the „production“ Rust. It's not about allowing dangling references in the borrow checker, but I think some kind of quick and dirty tutorial might help.

I think one part of why Rust feels slow when prototyping is compile times. If you are used to prototyping in python, having to compile (and wait for it) definitely feels slow. I've learned tricks for that too.

I can try putting the idea of Q&D tutorial to a todo list/blog post ideas list, but I don't promise to get to it soon or that an attempt to write it will be in any way successful enough for me to publish it.

4 Likes

This seems hard to achieve without very large changes to the rust compiler, such as e.g. adding a garbage collector to the language.

3 Likes

Pretty much everything I write in Rust are prototypes.

Seeing what works and what does not during the prototyping is useful, otherwise your prototype may simply not work in Rust.

9 Likes

Seeing what works and what does not during the prototyping is useful, otherwise your prototype may simply not work in Rust.

The question is, are you trying to answer „Is this possible to write in Rust“ or „Is this possible at all“?

I guess this is mostly about writing all kinds of throw-away or single-use bits of software. Nobody really cares if it's nice, maintainable or if it runs fast.

Perhaps the Rust-based-and-implemented dynamic programming language Rune would server your purposes.

3 Likes

Ever since I discovered IBM's REXX scripting language in the 1970's, I've done most of my algorithm exploration in a scripting language, preferably one with a REPL. Once I know what I should do, I rewrite it in a compiled language, preferably a strongly typed one. The exercise of expressing the algorithm in two completely different ways sharpens my understanding and exposes any fuzzy thinking I may have done.

4 Likes

I have always been kind of confused about the concept of prototyping. I haven't really ever seen the value of it. Writing something that works in one language and than porting it to another is just going to make the process take twice as long.

Can someone explain how they find prototyping helpful?

1 Like

@TomP : Does Rune have a repl ? I don't see mention of it in https://rune-rs.github.io/book/?search=REPL

@asafigan : One are this might be useful is due to library support.

Say for example you have to train some computer vision model. It is probably easiest to (1) open up a pytorch notebook (2) iterate there until you get thed right model and (3) port it to Rust, rather than build it from scratch in Rust.

One of the advantages of Rust for prototyping too is that while you'll probably spend more time writing the code to satisfy the borrow checker, in my experience you are much less likely to have runtime bugs. Oftentimes when I'm using Python I'll make a mistake and write a function like:

def foo(x = []):
    x.append(123)
    return x

forgetting that the default value is implicit global mutable state. And then it takes a while to debug because it looks innocuous to me.

I never have that kind of issue in Rust because the compiler protects me from accidentally adding global mutable state, which means I can focus more on my code and less on making sure I'm managing state appropriately.

9 Likes

Lol that's terrible

4 Likes

Fast prototyping of what exactly?

OK. It's really annoying that the Rust type system will not allow one to assign integers or floats or strings or arrays or objects to whatever variable name you have that is of some other specific type.

It's really annoying that when you do that the language does not do some automatic conversion to make it so. Including parsing strings into integers or converting integers into strings.

So this prototyping language will have to do away with Rust's type system.

It's really annoying to have to worry about life times. I mean, if anyone, anywhere has a handle on whatever it is then it should remain alive and valid. Right?

So this prototyping language will have to have a garbage collector. And do away with Rust's borrow checker.

Hmm... this "prototyping language" sounds exactly like Javascript. Where everything is totally malleable all the time. And everything lives as long as someone has a handle on it.

Hmmm... so when one wants to prototype a thing in that way, why not use Javascript? Or perhaps Python?

As far as I can tell, if one strips Rust of its type system and borrow checker then one might as well use C.

3 Likes

Actually, when I watch the likes of Jon Gjengset live coding a TCP/IP stack in Rust in a few hours: "Implementing TCP in Rust (part 1)": https://www.youtube.com/watch?v=bzja9fQWzdA&t=11629s

I wonder what all this fuss about "prototyping language" is. He just churns it out, in Rust, faster than any Javascript or Python ninja.

5 Likes

I have plenty of examples.

One of them was computing the shape of a spectral line from a rotating, pulsating star. (I used to be a professional astronomer.) There was a lot of fiddling around trying to find the best representation to use. Being able to make mistakes really fast is a big advantage in that kind of exploration. I used APL on the console for a good bit of that work. I could usually tell two or three steps in that an attempt wasn't going to work. In those days, APL was too slow to use for a research paper, so a student rewrote the APL version in Fortran.

Another example was using the Unix (not Linux in those days) bc desktop calculator to work out the algorithms for quad floating point on the Itanium. This work identified a bug in the C version of the multiply before it went into production. The REPL was key in finding the error because it made it easy for me to see every intermediate result. I was also able to test my fix right there.

4 Likes

It also depends on what scale you're prototyping something. If you're prototyping something fairly large, then that means refactoring constantly, and having a compiler that catches all your errors as you go along is a big plus.

I wouldn't say Rust is perfect here, but it's loads better than C or C++, because it has got your back. Java has a pretty good refactoring experience, though. With Rust in a breaking and then fixing up cycle, sometimes I seem to have to get through fixing a few waves of errors before discovering the unfixable borrow error which makes me have to reverse and try something else. I don't mind, though. Perhaps it's unavoidable.

1 Like

Searching lib.rs tends to produce much more relevant search results. It could be good to point people to that.

Um, wow. Sorry, Python, but that's awful.

Yeah, I think compile times are really the only blocker here for me. I feel like I'd rather write prototypes in Rust than anything else, but that's also partly because I'm very good with Rust and I love it so much. I think the biggest difference about Rust is that you need to be probably quite a bit more proficient at Rust before you can just "throw stuff together" and not have to worry a lot about the strict rules because you already feel the rules and you're used to just following them mostly and the compile errors you run into are easy to fix.

If we really want to make Rust a great prototyping language, they key isn't going to be to change Rust itself as a language, but to provide tooling to make it easy to do stuff.

  • Speeding up the compiler will always help things
  • Providing crates that make certain tasks easier will help a lot. These would probably need to have specifically functions that are designed to get stuff working as fast as possible with as little code as possible
    • Nannou is kind of an example of this as it makes it super easy to get procedural 2D animated graphics stuff to the screen.
6 Likes

The usefulness of prototyping also depends on how much planning you do in advance, and how similar it is to something you've done before. If you already know all the ins and outs of the design, translating it into code is a mostly mechanical process. If you sit down at the keyboard with a vague idea and intend to hash out the details as they come up, you're inevitably going to take some wrong turns that you'll have to back out and fix later; then it can be useful to build one to throw away.

When I look at something like the video @ZiCog linked I know that I'm not just seeing raw programming skill, but the developer's intimate knowledge of these protocols, probably years of experience working with other stacks in other languages, probably a few failed projects that taught him valuable lessons. He knows what the end result will look like before he starts.

In my professional life I write programs that nobody has ever written before to solve problems that are unique to my field. There are no reference implementations I can learn from, no RFCs to read, no specifications, no books, no experts but my coworkers. There are parallels with other kinds of programs, but the parallels are incomplete. I don't know what the end result will look like. So sometimes I write a prototype and end up throwing it out. Sometimes I go through a couple prototypes. Sometimes the prototype ends up being my v1.0. I just never know what might work.

Back to the topic: when I'm writing code like this it is handy to allow some sloppiness with ownership. Prototypes don't generally have to run for long periods, so leaks aren't a concern; they don't usually have to be super performant, so extra copying isn't a concern; but they do need to be memory safe. Garbage collection is probably the #1 feature that I associate with "prototyping". Metaprogramming is probably #2.

2 Likes