I am an experienced JavaScript/TypeScript, Python developer with some Go experience too.
I've played around with functional programming, coming to my own conclusions that designing with mostly pure functions and relegating side-effects to the periphery of the program is a good practice before I even knew what functional programming was.
I'm bored of front-end programming and solving the same problem over and over again with CRUD APIs on the backend. So I am looking for a new direction.
I started learning Rust for fun and found the functional aspects familiar and elegant and the ideas of memory management interesting, even if I didn't really understand a lot of it. I then realized that I couldn't understand exactly what pain Rust is trying to solve without teaching myself some C and understanding more about stack/heap etc, so I've been going through a C book and took a very basic embedded course, but I'm not sure I'm exactly interested enough in embedded to keep going down that course.
I guess I haven't had enough "Systems programming" experience to understand what C/C++/Rust can excel at that higher level languages can't.
I understand that mission critical, performance critical applications are "systems programming". Things like Databases, server software, OSs etc, but I'm having trouble understanding what I could work toward in small incremental steps that could put something like Rust to use, with the end goal using Rust.
What is a very simple project that I could do (we are talking a few weeks at most) that I could try in C or Rust that would demonstrate the unique use-case of such a language compared to say Python/JavaScript? If I had to guess at what area of "Systems Programming" I'm most interested in learning more about it would be either network/server related or OS related (I really like the Unix toolset).
My two cents: try making a simple shell - like really simple (can only execute commands).
You can then ramp it up by supporting different shell job control commands (fg, bg, &, fork).
In a way, yes.
The point here is to figure out how to use system libraries (nix for Linux, winapi for Windows), because that is something that is not really done in Python or Javascript (it could be, but it isn't).
For example: when you type emacs & on the terminal (using bash shell, perhaps), you see a new window of Emacs pop up, and a new prompt arrives. But how does it actually work? Can you write a program that does this? (Note that using the C system() function is cheating, since that just calls into bash, more or less)
That said, don't get too invested in making a new shell as a product - it's probably not going to be used (new shells have very low adoption rate). But you can make anything for fun!
I only have a few years of experience of web stuff, CRUD API etc, after many years of embedded and application programming in compiled languages. Given what you say there is is perhaps ironic that I am now creating "web stuff", CRUD API's, etc, in Rust. Thinks like Rocket provide great web server frameworks. Web Sockets is no problem. Database interaction a breeze, etc.
There are some very good reasons for using compiled languages:
Performance - Generating actual machine instructions out runs limping around some interpreter or JIT.
Correctness/Robustness - Compiled languages tend to have strict(er) type systems unlike the sloppy typing of Javascript, PHP etc. Rust and Ada in particular. C++ also. C not so much. That gives one great confidence in the working of ones creations.
Security - Now with Rust we get a lot more checking at compile time, thus providing correctness/robustness. This contributes greatly to to security. Even if one can still write insecure systems in a language.
I'm sure you will come up with some use of those qualities of compiled languages, especially Rust, even if it is still in web services, micro-services and so on. Who would not want performance, correctness and security?
Personally I would forget the C part. I see that it may be helpful when getting familiar with stack, heap, pointers etc. But I worry it is a lot of low level detail that hinders your focus on Rust. It may introduce "bad ways" to ones thinking that Rust won't allow.
I'm not sure I would call those other languages "higher level" than Rust. Except perhaps Haskel. A lot of Rust that I write looks pretty much like Javascript.
All good points. I totally understand the advantages of compiled languages; as far as web stuff--most I wouldn't do in Rust just because you get more bang for your buck in Python/JavaScript/Ruby. Those frameworks like Django have everything you need (e.g. db migrations) without having to paste together a bunch of other dependencies. Also in general web developers are more of a commodity than some other types of developers...and the web dev community is very entrenched in JS/Python/Ruby etc. Only a few shops would make the choice to use something like Rust for this. Of course there are exceptions, but they are very much exceptions.
The question isn't why would I use rust over python et al. I understand that rust's niche is in the area of Systems Programming, where either one or all of the following is true: memory is at a premium, safety is of utmost concern, performance really matters. I also understand rust gives us similar performance characteristics as C/C++ but without the undefined behavior, as well as what other type-safe, compiled languages give us.
The question is, for someone who has never done this kind of work before, is there a relatively simple exercise what can be done with something like Rust that can't be done with Python/JS etc, apart from embedded.
Is there a one week or less project I could do in Rust that can demonstrate its use-case, where an interpreted language like Python just can't compete?
Not really. If it is short enough to be finished within a week, I would say that Python/JS would be the thing to do. I mean, you can even write a shell in Python.
To be frank, there are a huge number of problems that don't need Rust. Probably most programs in the world can easily afford to take 100% more CPU and 100% more RAM than would be "optimal", for example -- and if you give a GC enough memory for double the working set then they tend to work great.
While Rust is mostly safety, here's a couple other places where it can be particularly good:
CLI tools where startup time matters. ripgrep is so fast that it can search the whole rust library directory in under 100 ms on my machine. Adding another few hundred milliseconds of framework startup and jitting and it's a huge perf regression -- and normally startup that fast is by running slowly, with the better-optimizing JiT only kicking in later. That's no big deal for a long-running web server, say, but the more command line things you run -- especially in a script -- the more that adds up. Not to mention that ripgrep is just a couple MB, with no need to also go install a huge runtime before you can use it.
Anything with complex parallelism. Remember that Mozilla funded Rust originally for doing parallel page layout -- and that's such a hard problem that it needed language help to get it right. If you've ever had a bug where someone used a Dictionary where it needed to be a ConcurrentDictionary, you have a place where Rust can help. Or, more generally, Rust will give compilation errors that amount to "you forgot to Mutex this", which is a complete game-changer for the freedom with which you can write parallel code. And the borrow checker is the single-threaded version of this -- if you've ever gotten a "collection was modified while enumerating" exception, that's something that Rust could have helped with.
This is a great answer. Thanks. Thus the issue...it's such a fascinating language to me that it seems like perhaps it's warranted to explore its problem space. It's not likely to come up with a toy project for this, and maybe a better idea to explore existing projects, understand the problem space and see if I can contribute. Learn by doing...
