On modern programming languages and growing hardware complexity


#1

This post aims to be a continuation of an off-topic discussion started in The best approach to learning Rust , where it was claimed that Rust’s cost-benefit proposition is only worthwhile in a relatively restricted corner of the programming landscape (almost entirely excluding user-space applications, for example), due to the fact that modern hardware is very fast and requires less careful handling from software, whereas the complexity of Rust’s efficiency-oriented lifetime and borrowing system leads to reduced programmer productivity which is on its side a very expensive commodity.

In this discussion, it was acknowledged that Rust has more to offer to programmers than a lifetime system. That its highly expressive static typing system improves self-documentation and compile-time error checking, thusly reducing the need for the tedious manual debugging that is the norm in dynamically typed programming languages, for example. Or that its ownership and borrow mechanism greatly simplifies the task of leveraging multi-core hardware parallelism. Which led to a shift of the discussion towards how Rust compares to other modern statically typed programming languages, such as Go or Haskell, aiming to simplify the programming of such hardware.

A first counter-argument that was given was that one underlying assumption of this discussion, while not inaccurate, is quite misleading. While it is true that computer hardware has, for several decades, quickly increased in performance, leading to a shift in the balance between expending programmer effort in software optimization on one side, and prioritizing manpower savings through increased hardware expenditures on the other side, this statement has stopped being accurate in the middle of the 2000s, when the race towards faster CPU clocks, memory bus performance, and instruction-level parallelism came to a near complete halt.

Modern computer hardware only increases in performance through mechanism which are, to a large extent, not transparent to the programmer, and require explicit support on the software side. These include:

  • Multi-core parallelism (and hardware multi-threading in general), which is the best-known one.
  • Vectorization, which increases the CPU throughput of a program per instruction, but can only be automatically carried out by compilers in specific cases (when memory regions are properly aligned and not mutably aliased, and control flow is not too complex, for example).
  • Increasingly deep and complex memory hierarchies, whose fastest layers are so starved for space that they are in practice as difficult to fit in as older-generation computers, with the added complexity of new cache trashing phenomena such as false sharing.
  • Specialized hardware accelerators, such as GPUs or FPGAs, whose programming model is fundamentally different from CPUs in the sense that they are much less forgiving of complex code, sophisticated control flow, and high memory footprints.

In effect, computers are arguably more difficult to program efficiently today than they have ever been, and acknowledging this complexity by giving programmers tools to fight it entails a lot more than just multi-core support.

A second point that was raised is that while it is true that programmers can waste a lot of precious time chasing imaginary performance requirements, they can just as well waste everyone else’s time and CPU power by failing to properly account for the performance requirements that they actually have. Examples of practical software performance concerns that were invoked were the electrical power consumption of data centers, monotonically degrading performance of many productivity-oriented desktop applications, and tragically bad battery life of energy-constrained mobile devices.

From this point of view, it was claimed that there is actually a real need for software performance in many applications, both user-land and elsewhere, and that Rust can be a productivity boon for programmers having efficiency needs as it is ahead of available alternatives as far as the “highly efficient, but still productive” compromise is concerned.

To both of these points, it was replied that Haskell is a better solution than Rust, due to its strong support of multi-core parallelism, which is what I am going to reply to now.

I don’t think that Haskell would be a good solution to the software efficiency crisis that we are traversing (irrespective of the many other merits of this language), because it only focuses on some aspects of modern hardware efficiency at the expense of others:

  • Multi-core is only one part of the hardware equation. If your take on leveraging it involves lots of memory copies (originating from a focus on immutability and recursion) and an indirection-rich data model, you are likely to lose in memory performance what you gain in parallelism elsewhere. This is why benchmark-winning Haskell code is usually very far from one would consider idiomatic or clean code in this language.
  • While vectorizing Haskell code is not impossible, the implementation strategy of this language does not lend itself very well to it. Lazy evaluation leads to conditionals-heavy code, which most SIMD architectures are enemies of, and indirection-heavy data structures such as lists are all but impossible to write efficient vector code for. Again, Haskell code which vectorizes well will be very far from idiomatic use of the language, making it difficult to write. Contrast with libraries like faster in Rust, which leverages the language’s support for efficient high-level abstractions in order to make SIMD code very natural to write.
  • Idiomatic use of garbage collection naturally leads to an indirection-heavy data model which trashes CPU caches by having them load plenty of useless data only to leverage a small pointer out of it. Any programming language which relies heavily on it as a native abstraction will be in trouble when trying to make the most of the limited capabilities of modern CPU’s highly stressed memory subsystems. This point also applies to Go or Java, for example.
  • All the previous discussions become even more salient when targeting hardware accelerators, such as GPUs or FPGAs, which are becoming increasingly popular in areas such as high-speed finance computations or machine learning. While Rust has not yet made very strong inroads in this area, I expect its “keep the runtime lean and compile down to highly efficient code” approach to shine in this area of computing, and am very excited about projects aiming to compile Rust down to SPIR-V GPU shaders for example.

(@donallen, please drop me a PM if you think that I misrepresented some of your points in this summary, I am ready to improve it if needed).


The best approach to learning Rust
#2

I wish I had more time to write a thorough reply :slight_smile:

Instead, I’ll through in this talk which is highly relevant to discussion:

…and which introduces a very interesting notion of “time to performance”: the amount of time a programmer needs to spend to write an adequately performing program: fast, but not necessary the fastest.

The speaker argues that languages like Java have better time to performance than more low level languages, which seams plausible to me: in Java, I write ArrayList<Integer> and it’s bad for performance because of boxing, but it is asymptotically OK. In Rust, I write Vec<i32> and then either do a lot of .clone ining, which is worse then boxing, or I resort to &[i32] and start fighting with the borrow checker.


#3

I’ve seen the “time to performance” argument before. It’s an odd angle to take on performance and is more of a “marketing” term IMO. It doesn’t quantify performance but rather assigns a qualitative “fast” or “fast enough” yardstick. That’s somewhat meaningless because any language might be “fast enough” - depends on requirements of course.

But, let’s run with that definition. The second issue is performance is multidimensional. There’s predictability of it, control over it, and things like memory footprint. Java pretty much fails all of these. There’s virtually no performance model beyond primitive types vs references. The entirety of “fast Java” depends on JIT heroics, knowing how the JIT optimizes and using “low level” Java code (abstractions cost you 99% of the time). There’s no predictability - the JVM can stall for its own reasons at virtually any point for a variety of reasons (that I can get into if anyone is interested). Memory layout control is not existent in the language and hacks are needed. I’m not even going to get into the GC aspects.

A lot of the Java vs C++, Rust, etc comes down to control - how much do you have when you need to move the needle and what’s required? Java is not very good at this.

I’ll keep this short for now but happy to expand.


#4

I wouldn’t quite say it that way. I have some experience in manipulating vectors of numbers in Java, and it was quite a struggle. Without value types, arrays of primitives (e.g. float[]) are the go-to types for a standard efficient representation, and these don’t support slicing at all. Rust (like C++), can not only provide the most efficient representation for vectors of numbers, it also provides a safe and fairly intuitive API around contiguous slices of values. Fighting the borrow checker is an intrinsic part of Rust programming which becomes less problematic with experience, and I’d say it’s definitely worth it for the benefits already stated in this thread. :slight_smile:


#5

Thanks for channeling this into its own topic @HadrienG !

I will support the general idea that Rust has to be a conscious decision. I can do with a few lines of bash/perl/groovy things that would take a day’s worth of C/Rust. For most things, I’d get acceptable performance quicker in high level languages.
But, after I’ve tried the quick, high level solution and deemed it too slow, (which happens regularly, though not often) I am down-on-my-knees grateful that I can use Rust, instead of C.

I consider myself something of a language polyglot, having programmed in Java, Python, Groovy, Bash, Perl and Rust for production use cases in bioinformatics.
Importantly: I am NOT a C/C++ developer with years of experience, and I am NOT a Functional wizard with Haskell/Lisp/Erlang experience.

I am not the worlds best programmer. I learnt mostly in dynamic languages, and I WILL mess up some obscure pointer aliasing, cache invalidation or thread safety rule if you let me.
Rust tells me when I’m about to do this, every time I hit “compile”. If I had to do it in C, probably my production server logs would tell me, on a Monday morning after the servers spent an entire weekend twiddling thumbs because my code segfaulted two minutes after I left the office.

I often get the impression that people who complain about “fights” with the borrow checker feel that it an unnecessary obstacle. I however see it as a safety net telling me I’ve missed an edge case I am too inexperienced to recognise.

Also, I can have my cake and eat it too; write most stuff in Python, Perl or Groovy, and embed some compiled rust in the hot loops (numpy style). I haven’t used it yet; but I a so pleased to have the option!

I suspect that I am not alone in being conservative with my risk. The part where programmers “waste everyone’s time and cycles” is not out of malice, but out of fear/caution to not mess it up.
I’d rather my program be slow in python than crash because I misused C-embedding.
Rust allows me to learn and experiment with native coding, confident that the compiler and\or type system will catch me.

In that respect, Rust doesn’t offer new possibilities for performance tuning and close-to-the-metal programming, but it does expand the audience of people to whom it is available.


#6

Seeing that the “time vs cost” (EDIT: “cost vs benefit”) seems to be one of the arguments of @donallen, I think I need to put down my experience (this won’t be long as I am still in school and heavily learning about programming).

As a new Rustacean, I value different aspects of the language than all all of you might. Let’s talk about my failed attempts at learning C:

To begin with, in school we learn C# and Java (when we’re not trying to do things in Ferora 14 (yes, 14, not 25) to learn about linux), with some hints of javascript and PHP. Thus, I never touched anything that didn’t have an oriented object approach. (appart from javascript, but even there I heavily use objects)
So, in my spare time, I searched for a lower level language to learn on my own. I already had done a little bit of C++, but I wanted to go lower, just to see.
Maybe I made a mistake, but I chose to write my own Window Manager using XCB. Well that wasn’t a good choice. I wanted to learn C first, and the documentation is all over the place. I can find a lot to learn, but it’s on many different sites that all do things differently… Not a good first impression; I don’t know anything about the language and I’m already lost.
I decided to still give a shot at writting that Window Manager; time to read XCB’s docum… oh wait. None, there is NONE. I couldn’t go any further. Every information that I read about people trying to build WM’s said: look at AwesomeWM’s code or i3’s code. Not easy when you don’t know a lot about the language.
I decided to drop C completely (knowing that it kind of was my fault, I didn’t begin with the right project).

I then decided to do a Minecraft launcher in C++. This also failed in about 2 to 4 hours when I realised I couldn’t compile the damn thing on Windows. So I took the easy route and did it in Java. It worked.

But then I found that weird language called Rust (searching for something on the Rust game, what a coincidence). I waited for some times to have a project I could do with it. When the idea came, I lauched myself:
I took one of my sites and converted the PHP code to Rust. Well, I completely re-did the application really.
The first thing to do was to learn Rust. Here came the 2 books and the endless ammount of help threads on this forum (I recommend anyone to read them all, I learned a lot doing this).
The second thing was to make some code. I decided to make the ftps part first and quickly stopped because the ftp crate didn’t work well.
So I experimented with SSH then, I had some lifetime issues that I finally worked out, and it worked like I wanted (even though SSH seems to be slow to connect, about 0.30s).
Seeing that the FTP crate didn’t get fixed, I got into the code of someone else for the first time and fixed it.

I did all of this in less time that I took to research to learn C and how XCB works.
For someone like me, that’s not out of school yet, learning time is more valuable than anything else. And Rust’s auto documentation is a god sent.

I’m sorry if this is long. This summarize my experience with Rust and how I see the language as someone that doesn’t already have much experience in other languages.

TL;DR: Rust is a great language to learn and fun to write with a really good community that wants you to be a better programmer by helping you and explaining to you what’s going on. Not the experience I had with C. From a learning perspective, Rust’s automatic documentation, strict naming of variables and functions, formatting with RustFMT, this will to enforce a style is amazing. This is the first time that I was able to go inside the code of someone and understand things! I am able to go inside, take a function, and search into the doc of the crate used. I feel like every time I spend on fighting the borrow checker and trying to understand lifetimes is time I don’t spend on trying to understand 3 different docs for the same API and 4 forum posts that give 4 different solutions for a problem.

EDIT: Once again, I didn’t talk about other things that I like about Rust. Like the fact that it can be a low level or a high level language depending on how you want to use it. And yes, I know that a web app is maybe not the good domain to write Rust code in for a first experience, but I wanted to see how far it could go: as far as I want. Not like PHP, where I didn’t try Sftp (FTP over SSH) because it has a weird implementation that is not complete. And debugging PHP is, well, not a pleasant experience.

EDIT2: added precision to the cost argument, my position doesn’t change since I’m not even really saying anything about this argument :smile:


#7

An impressive write-up, a good summary of your position and a fair and
mostly accurate characterization of mine.

However, I do think that you may have misunderstood (or I didn’t state it
clearly enough) what I said about Haskell (and Go). I am NOT arguing that
Haskell is equivalent to, but better than, Rust. I am saying that for many
applications that don’t have particular real-time constraints or require
ultimate performance, Haskell and Go are easier to learn and use than Rust.
For the kind of very-high-performance code, where the issues you discuss in
your write-up are important, then it’s likely that Rust is a better choice.

Everything I’ve said boils down to: choose the best tool for the job at
hand, based on real analysis of the situation. Being in love with a
technology is not real analysis.


#8

Luxed https://users.rust-lang.org/u/luxed
December 4

Seeing that the “time vs cost” seems to be one of the arguments of
@donallen https://users.rust-lang.org/u/donallen,

Not quite. It’s cost vs benefit.


#9

Yes, this is a simple analysis that needs to be done before each project: what language to use for the specific needs. Rust is just a new language in the sea of languages that we currently have.

Thank you! Added the precision to my post.

I don’t think I have more to say, but I still want to read the opinion of people and, even more, their experience.
This forum is so respectful that reading opinions is extremely enriching.


#10

@matklad @vitalyd I think that “time to performance” does have some merit as a qualitative metric, insofar as one understands it as programming languages and their ecosystem making it easy to do the right thing from a performance point of view.

Speaking from my professional homeland of scientific computing, for example, languages like MATLAB or Fortran 90 which make multidimensional arrays and linear algebra operations easy and obvious are a godsend, as they prevent people from reinventing them badly through inefficient array-of-array data layouts and barely legible hand-written loops. I personally feel that numpy is a pretty poor substitute for these two as far as numerical computing is concerned, but so long as people like it, at least they won’t be doing linear algebra in raw Python :slight_smile: I have high hopes in Julia in this respect, but these hopes are based on a purely theoretical understanding of the language at this point and may evolve into disillusions by the time I will actually have managed to find an excuse project to play with it.

At a more generic abstraction level, I think Rust’s iterators are another impressive achievement in this area. They are basically C++'s STL algorithms done right: easy to use, idiomatic, concise, and yet still every bit as performant as hand-written loops thanks to some impressive compile-time magic. The standard ones are already very good, and itertools and rayon take it to the next level. It’s one of the few times I have seen a programming model based on higher-order functions which “normal” people actually want to use, the other big success in this area that I’m aware of being Apache Spark, which managed the no small feat of repackaging Hadoop/MapReduce into a technology that I can actually envision suggesting to my less computing-focused colleagues.

I am still waiting for comparable library + language abstractions for democratizing good memory layout and vectorized code. On the vectorization front, I keep seeing my colleagues falling back to hand-written compiler intrinsics and brittle compiler directives for any non-classical task (i.e. not your average BLAS or FFT, which someone else has already written), which strikes me as a crude, brittle, and poorly maintainable approach, and I have high hopes in tools like the aforementioned “faster” crate which multiplies Rust’s iterators with vectorization for quadratic levels of awesomeness. On the memory layout front, I see various “80% solutions” out there, but they tend to either solve only a small-ish subset of the problem or involve C macro-based codegen tricks so arcane that the compiler error messages will make you cry, so I think this problem is still looking for a better solution.

This is the meaning which I personally put behind the “performance per time” motto, and for that I’m not convinced that Java is the language that comes best prepared, because as @vitalyd points out it tends to hide the machine so much that even implementing a library-based approach can be difficult (without even speaking of making it feel nice and usable). But I’m ready to be impressed!

@donallen Apology for the rushed out and oversimplified header of the Haskell part, I was at the airport and my plane was about to leave :slight_smile: Obviously, I did not understood your post as meaning that Haskell is unilaterally better than Rust, but rather that Haskell is a good solution for solution for “efficient, but not maximally so” programs, which I happen to disagree with for reasons described above. I love the functional paradigm for many things (especially anything that touches an AST, a database or a network), but I don’t think that Haskell’s idiomatic writing style based on pure functional programming combined with lazy evaluation and garbage collection is appropriate for the performance-sensitive parts of most applications. Even with the current impressive state of compiler technology, it can still takes a fair share of impurity to make software go fast, and I have the impression that Rust is much better than Haskell at this impure part.


#11

I’ve switched off of Rust in the past for a very simple reason:

Programs like bors-ng, Janitor, Discourse, and Hadoop don’t use Rust because they’re spending the vast bulk of their runtime waiting on other (“backend”) programs to run, and all of the efficiency gains are going to come from making fewer and more intelligent calls into that backend.


#12

HadrienG https://users.rust-lang.org/u/hadrieng
December 4

@matklad https://users.rust-lang.org/u/matklad @vitalyd
https://users.rust-lang.org/u/vitalyd I think the “performance per
time” does have some merit as a qualitative metric, insofar as one
understands it as programming languages and their ecosystem making it easy
to do the right thing from a performance point of view.

Speaking from my professional homeland of scientific computing, for
example, languages like MATLAB or Fortran 90 which make multidimensional
arrays and linear algebra operations easy and obvious are a godsend, as
they prevent people from reinventing them badly through inefficient
array-of-array data layouts and barely legible hand-written loops. I
personally feel that numpy is a pretty poor substitute for these two as far
as numerical computing is concerned, but so long as people like it, at
least they won’t be doing linear algebra in raw Python [image:
:slight_smile:] I have high hopes in Julia in this respect, but these
hopes are based on a purely theoretical understanding of the language at
this point and may evolve into disillusions by the time I will actually
have managed to find an excuse project to play with that language.

At a more generic abstraction level, I think Rust’s iterators are another
impressive achievement in this area. They are basically C++'s STL
algorithms done right: easy to use, idiomatic, concise, and yet still every
bit as performant as hand-written loops thanks to some impressive
compile-time magic. The standard ones are already very good, and itertools
and rayon take it to the next level. It’s one of the few times I have seen
a programming model based on higher-order functions which “normal” people
actually want to use, the other big success in this area that I’m aware of
being Apache Spark, which managed the no small feat of repackaging
Hadoop/MapReduce into a technology that the masses can actually use.

I am still waiting for comparable library + language abstractions for
democratizing good memory layout and vectorized code. On the vectorization
front, I keep seeing my colleagues falling back to hand-written compiler
intrinsics and brittle compiler directives for any non-classical task (i.e.
not your average BLAS or FFT, which someone else has already written),
which strikes me as a crude, brittle, and poorly maintainable approach, and
I have high hopes in tools like the aforementioned “faster” crate which
multiplies Rust’s iterators with vectorization for quadratic levels of
awesomeness. On the memory layout front, I see various “80% solutions” out
there, but they tend to either solve only a small-ish subset of the problem
or involved C macro-based codegen tricks which are so arcane that the
compiler error messages will make you cry, so I think this problem is still
looking for a better solution.

This is the meaning which I personally put behind the “performance per
time” motto, and for that I’m not convinced that Java is the language that
comes best prepared, because as @vitalyd
https://users.rust-lang.org/u/vitalyd points out it tends to hide the
machine so much that even implementing a library-based approach can be
difficult (without even speaking of making it feel nice and usable). But
I’m ready to be impressed!

@donallen https://users.rust-lang.org/u/donallen Apology for the rushed
out and oversimplified header of the Haskell part, I was at the airport and
my plane was about to leave [image: :slight_smile:] Obviously, I did not
understood your post as meaning that Haskell is unilaterally better than
Rust, but rather that Haskell is a good solution for solution for
“efficient, but not maximally so” programs, which I happen to disagree with
for reasons described above. I love the functional paradigm for many things
(especially anything that touches an AST, a database or a network), but I
don’t thing pure functional programming is appropriate for the
performance-sensitive parts of most applications. Even with the current
impressive state of compiler technology, it can still takes a fair share of
impurity to make software go fast…

No apologies necessary. As to “Even with the current impressive state of
compiler technology, it can still takes a fair share of impurity to make
software go fast…”, I agree generally (see the amusing comments of Simon
Peyton Jones here about the need for impurity::
https://www.youtube.com/watch?v=iSmkqocn0oQ), but my experience with
Haskell (a lot) is that it’s been plenty fast enough for my applications.
Which goes to a corollary of my central point – don’t pay for goodness
that you don’t need. We all obey this corollary more or less. When was the
last time you wrote assembly language code?

Haskell/ghc code is faster than Python and related popular interpreted
languages, which would not have been suitable for what I have been doing
with Haskell. So again, if tight performance constraints require ultimate
performance, Haskell is most likely not the answer. But for a lot of
mainstream applications that do not fall into that tight performance corner
(the vast majority, in my opinion, especially on today’s incredible
hardware), Haskell is an excellent choice – statically typed (shifting
many errors from run-time to compile-time when compared to dynamically
typed languages, a virtue shared by Rust, Go, and even C, with the right
compiler settings), very good performance (the ghc compiler is really a
work of art) and far easier memory management than Rust. The same could be
said for Go. In addition to Rust, I would also not choose C or C++ (a
blight on the programming language landscape, in my opinion) for such
applications, assuming equal familiarity with the languages.


#13

It will be nice if we can limit ourselves to shorter and discrete paragraphs.


#14

It would be great, but I don’t think short paragraphs are enough to say what needs to be said.
I like to take my time and read what everyone has to say :stuck_out_tongue:


#15

That’s what I meant by “discrete”. If you write a lengthy post that requires a TL:DR; then delete everything and keep the TL:DR; section :stuck_out_tongue:


#16

I think another interesting topic of discussion would be “On modern programming languages and growing software complexity” :slight_smile:

(I see a few comments in this thread about how people appreciate how rust can help them with software complexity. I think this thread wanted to stick to performance and hardware. But maybe it’s impossible to fully tease apart hardware and software)


#17

I’m giving the choice to people who maybe don’t care about my life and just want a short resume of what I’m saying. I could also replace the TL;DR by “In conclusion” :slight_smile:

I think I quickly said somewhere that a quality that I really like in Rust is that it is a high and low level language at the same time. I just did the last chapter in the Rust book. And then I used Iron to do the same thing but with 10 lines of code (exageration).
Or maybe this isn’t what you really meant.

I think I read somewhere that Rust can go lower than C ?
But then, if a really low level API is done, it can be easily wrapped in a higher one for easy use in programs.

The crates.io system and version system don’t get enough praise either.


#18

We need to look at Rust’s objectives. What are the problems it is trying to solve? What type of “language” it claims it is? I see a lot of people (not in this post) who often miss the point.


#19

I think its helpful to think of the lifetime system as part of the larger type system. Sure, it’s most immediate use is for managing resource allocation, etc. But it also lets you express properties of your types and APIs that are extremely hard to capture in any other way, and don’t really have an economical equivalent even in type-rich languages like Haskell and OCaml. Not being able to pass ownership of something to a function and know that the caller no longer has access to it is effectively a case where having GC as part of the semantics and implementation makes the language less expressive.


#20

With a topic like this, I don’t think “short” is going to happen :wink:
I do agree that the occasional extra linebreak would help readability, especially for many of our non-English friends.

Agreed, having 'lifetimes as part of the type is the magic ingredient that allows Rust to do compile time “liveness checking”, which is what a GC would do during runtime normally.

As for Haskell/Erlang/Etc. being “good” choices:
To me as a self-taught programmer, they have always seemed like magical, god-like languages that I am not (yet) worthy enough to use. The mentions here, and in the telecoms thread, suggest to me that I should really check them out (again).
Does that mean that Rust is my “gateway drug” into Functional-ness? :wink:

I have heard that many of the core team that designed Rust was familiar with Lisp dialects, Haskell, Elm and a bunch of other cool languages. They seem to be awesome inspirations!

As for the topic of hardware:
one thing I have always seen written about the functional languages, is that they are far away from the metal. Their almost magical compilers are very good at teasing out parallelism from “functional” code, but Rust is the first language I have heard of that manages to effectively combine close-to-metal control (eg. repr["C"], inline asm, handwritten loops, etc.) with functional ideas (iterators, map-combinators, monads, etc.).
Is that a sign of my ignorance, or the “value-add” that Rust genuinely brings to the table?