My dream language

...until someone writes the accidental Shlemiel algorithm, like O(n^2) where O(n log n) is easily achievable, and the machine time unexpectedly blows up. How do we expect to train AI to avoid these mistakes, if we don't even spot them all ourselves? And, what's worse - how do we expect to debug and optimize these cases, when (not "if") AI falls into such trap?

3 Likes

You’re implying that algorithmic complexity is defined by specification, which should never be the case. Implementation details should be behind abstractions for this reason.

Nope. I'm implying that for the same specification it's equally possible to create both "normal" and "pessimized" code, and we should be sure that if abstractions lead us to the latter, we always can dig deeper and convert it into the former.

4 Likes

See Modular Docs - Mojo🔥 FAQ

Over time we expect to open-source core parts of Mojo, such as the standard library.

For me that means, that they intend to make most of Mojo closed source. That may be justified, as the company behind Mojo has collected millions of dollars venture capital, and they may want a revenue.

1 Like

Oh, OK. I was just going by what they'd said in a previous video, which may have been earlier than this statement.

My guess is that they will make an AI-decoupled part of it open source, so it can be used in other domains and make the rest - CUDA etc - proprietary libraries.

If that's feasible that is. I know very little about such things.

IMO the entropy of predicting what technology will look like in 40 years is just too high as that any such prediction could be taken too seriously. So any discussion will just surround the collision of peoples' imagination and I think yours is just a tad more optimistic than mine :slightly_smiling_face:. I'm just having a hard time to wrap my head around the possibility of having something like ChatGPT 42 create optimized machine code from a prompt. I feel like that'd require a paradigm shift from how LLMs or any other giant deep learning architecture works these days. And I (with my limited perspective) don't think that the recent advancements in machine learning research give any indication that such a paradigm shift is imminent. Then again, I could be totally wrong and all we need is to throw more compute and more data at these non-linear models with lots of parameters to make them smart enough to reliably produce machine code directly from a human-language-like specification language :man_shrugging:

1 Like

The development seems to be extremely fast currently. OpenAI have their Q-star algorithm, Microsoft has just published information about Orca2. All that seems to lead to self improving AI with good math capability. So AGI may be closer than we really desire. Youtube has some interesting videos about it, I am just watching https://www.youtube.com/watch?v=_uupjIhq-Qw

And that's great, while it lasts.
We just don't know how long that will be though. Technology in general tends to evolve in repeated bursts. Will this one take us all the way to AGI? Tough to say, though I will add it's hard to take any output of such models seriously for my work until those models are augmented to do proper causal reasoning, of which deductive reasoning is one part. Anything up to that point is merely stochastic parroting, and as impressive as that's gotten recently, I can't trust it to be truthful.

5 Likes

AGI wouldn't lead us to the language that started the discussion. That's just simply impossible.

Why C++ programs are fast? Why Rust programs are fast? Why JavaScript or Java are slow?

Because GC is slow? Nope. Any benchmark would show that GC is not too slow, often it's even faster than malloc/free that C++ or Rust, ultimately, use.

The problem of Java or JavaScript is different: GC is invisible and thus you couldn't easily see how much memory pressure is there in a typical program written for GC-based language.

This means that this:

is fundamentally incompatible with this:

Because you are proposing to eliminate precisely and exactly what makes these programs fast: the need to deal with complexity and the need to write your program without magic.

That quote is more apt than many realize. Once you stop understanding how technology works it becomes “magic” and starts working like “magic”: needs certain rituals that make no sense, works unlreliably, produces results different from what you want or expect, etc. Heck, just read any magic book, you'll know how many different way may go wrong with magic!

And that is why nothing, not even AGI may produce the language topicstarter wants.

AGI may eliminate the need to use programming languages, maybe. But it couldn't produce language which both have “magic” and yet works reliably and predictable.

Rust works reliably and predictably precisely because magic was squeezed out from it during development.

Certain “magical” algorithms still exist in Rust (e.g. type deduction, borrow checking, etc) and they do work as “magic” (as in: they unpredictably fail or work depending on phase of moon), but, critically, Rust is designed in a very special way where magic either works and produces optimal result or it fails and forced the human to change the requirements.

I'm not quite sure how one may have “magic” of that sort in the area of “more intelligent front-end”.

“Magic” which sprinkles clone everywhere is not hard, but I fail to see how implement it in a way that it would fail in cases where change to the program may eliminate clone.

And if you couldn't do that then addition of such “magic” would do what “magic” always does in all languages: lead to slow, resource-hungry and unreliable programs.

TL;DR: C++ and Rust programs achieve efficiency not because of their specific “magic” but because of their lack of magic.

If you push the complexity in the human's face… human often may decide to change the task that s/he wants to solve to avoid that complexity, if you hide the complexity then human have no incentive to do that and then there are no way to remove that complexity from your program

5 Likes

You're not the only one who would like a language that has most of Rust's benefits but is simpler to use. I would, and we probably all would, if it were practical. And language researchers are working on ideas for this. I think Rust's success opened up the possibilities. We now know that it is possible to have a memory safe language that performs as well as C/C++, and therefore it might be possible to do that with less complexity than Rust has.

I've looked around at some new languages in this general category and the most interesting to me are Vale and especially Hylo. However, these are research languages at early stages. They have yet to prove that they can actually reduce complexity, and perform as well as Rust, when fully implemented. To me it's a very interesting possibility, but remains to be seen. We will have to wait a while to find out.

3 Likes

What I think is still missing in this discussion, is that it will get easier for you to choose between &, &mut, Rc, Box, Arc etc. If you give yourself time to understand these concepts and run a few times into a case where you have to think about which one you need you will learn why they exist. You will in fact start to wonder why you didn't have to know this in other languages since that choice will be inherent to the actual problem you try to solve. And shortly you will wield them with confidence and you will see them as helpful in capturing the underlying complexity of the problem instead of as a hindrance to progessing.

6 Likes

I do see the word magic used regularly, in place of hand waving, to describe something the compiler will just get done. I guess the usual follow-up question is that if a mental model is indistinguishable from an artifice does that make it any less real? Should we want the compiler to feel like magic? I wouldn't think so, maybe many here would disagree.

Of course, and it is really not that difficult. But as a Rust beginner, I am still unsure how much longer it will take to develop a medium size program in Rust compared to a language with a simpler memory concept like Nim. Nim is statically typed, and differentiates between stack allocated value types and heap allocated instances called references. And it uses the var keyword to denote mutable data. In Nim we can copy paste a textbook algorithm nearly verbatim without thinking, and it is very easy to refactor code. So some regard Nim as compiled Python with type annotations. For Rust, it is not really difficult, but needs some thinking. The advantage in Rust is obviously, that data is never copied automatically, just borrowed or moved. And the borrow checker allows optimization like string slices, which can avoid allocations and copies. Compared to that, Nim would for example have to allocate new substrings if we split a line of text at whitespace into substrings. (I think they are working on fixing that, but I am not sure, as I have not any longer followed Nim in the last months). Nims memory management may have still some (fixable) issues with threading, while it seems to work very well for Rust. But it is said that Golang has very good threading support as well, while it even uses a traditional garbage collector. Well, I will continue learning Rust this winter, maybe with some experience developing in Rust is not that much slower than in other languages.

[EDIT]

not that difficult.

I should correct myself: For most code, Rust's ownership annotations seems to be not that difficult. But for data with no clear owner, like spatial graphs like a delaunay triangulation, an RTree, or just a doubly linked list, I have no idea how to do it yet. But luckily we have high quality Rust packages for that.

Let's say there is a large project where some critical object of type T is used pervasively, but it is never changed.

So the compiler deduces that the critical object can be used via a &T in thousands of functions.

Then, one day, someone somewhere in an obscure part of the code, deep down the call stack, decides to make a change that mutates the critical object.

So now the compiler decides to flip thousands of &T to &mut T.

Meanwhile, some one else was planning to make an important change that requires the critical object to be &T, not &mut T, which is now incompatible to the change made earlier on a completely different part of the project.

Maybe the compiler is smart enough to point out the two places that are incompatible - after a very long compiler time.

Or maybe it is three places? One that requires the mut and two other places that want to borrow simultaneously?

If multiple teams work on the same project, the decision-making process which change should be adopted can take a lot of work.

6 Likes

Citation? The linked wikipedia says it makes programs 10% shorter, which might be true, but is a completely different statement from something about productivity.

(For example, I wouldn't be surprised if switching from Allman to K&R brace styles would also make things about 10% shorter.)


Ironically, I think scripting languages -- like Python -- are probably the worst uses for white-space-sensitive syntax, since they do less up-front checking.

If I imagine a white-space-sensitive Wust language, it would have lots of "hey, it's really weird that you changed scope here" compiler warnings that you'd get from wargo check, to help mitigate the potential problems from the copy-paste issues. But there's no step like that in typical scripting languages.

4 Likes

Actually, I visited that Wikipedia page after posting my message and found that my citation was not quite correct :slight_smile:

But I regard his comments in the linked video and the GitHub issue tracker convincing.

Video is https://www.youtube.com/watch?v=Z0w_pITUTyU starting at 36:30 up to 45:00. I let
GPT-4 convert the slide from the video to plain text:

Optional Braces

    Is very controversial, so we added it as an experimental feature

    I have now used it heavily for more than half a year. The experience has been positive beyond all expectations: It’s the single most important way how Scala 3 improved my productivity.

    Programs become shorter (>10% reduction in line count), clearer, and more beautiful.

    Writers stay “in the flow”; no need to go back and forth inserting braces

    “Indentation is like lego-bricks, braces are like baling wires”

    There were no downsides, program changes were more reliable than with braces.

    Of course details matter - we took quite some time and several trials to get it right.

His GitHub issue was Consider syntax with significant indentation · Issue #2491 · lampepfl/dotty · GitHub

Is there really such a thing?

If nobody knows if they own it then nobody can ever dispose of it. It's a wasteland where anyone can do what they want and f' everyone else. With the ensuing chaos.

3 Likes

Ah, thanks, I'd much rather read something than sit through a video.

I'm amused that it proposed removes the }s but also adding // end f "comments" with syntactic meaning.

Definitely some interesting points, though I wasn't convinced overall. Mostly, I think the "rustfmt on save" kinds of things have greatly changed a bunch of the arguments around this, especially in a more "belt and suspenders"-appreciating language like Rust.

One downside it particularly misses is the thing that happens here all the time of someone pasting in code without using ```. With braces we can drop that in rustfmt and have it good to go again immediately, but with indentation only it's lost essential information.

I do agree that else if chains are horrible syntax. Rather than

if (cond1) {
  ...
} else if (cond2) {
  ...
} else {
  ...
}

I wish it was something more like

if {
    cond1 => ...,
    cond2 => ...,
    else => ...,
}

for lots of the same consistency and editability reasons he brings up.

Rust was right not to do that change, though, since a bunch of its attraction is in being a not-weird as possible outside of the places it really needs to be.

Rust, even from pre-1.0, also fixed a bunch of the complaints in there. For example,

The code below
exhibits a common problem:

if (condition)
  println("something")
  action()

which, yes, is a common problem in C, but not in Rust because Rust intentionally make better syntax than that, without it even needing to take more (non-whitespace) characters.

3 Likes

I do not understand some of the mockery this post has received. I have always liked this quote by GBS:

The reasonable man adapts himself to the world, the unreasonable man persists in adapting the world to himself, therefore all progress depends on the unreasonable man.

One thing I'd like to see in Rust is a macro language alternative to macro_rules / procedural macros.

macro_rules is nice. procedural_macros is nice. Sometimes, I really miss Lisp style list -> list style macros. Going a step further, I'd be interested in a "layer" where we had lisp style syntax with rust semantics. I feel with the flexibility of lisp macros, one can more easily build DSLs over Rust.

1 Like

You can almost write that with match:

match () {
    _ if n > 10 => println!("big"),
    _ if n > 5 => println!("medium"),
    _ => println!("small"),
}
4 Likes