Rust-like scripting language

I fully expect I'd get to appreciate all those features of JS when I get to understand them. But I don't understand enough of it yet, and our codebase is a really complicated thing parsing a binary protocol between RXes and a central "whisker", running on the microcontroller in the whisker's Linux. I just really struggle to get my head around it. Maybe not the fairest codebase to base my opinion about JS on, but hey!

Ah, there is a thing. We have a bunch of proprietary binary protocols to deal with. Coming from various hardware devices. Lots of messy byte and bit twiddling. JS is really not up to that performance wise and it's not pleasant to deal with. So there was our C++.

Rust is great for that kind of thing. And as it turns out for us we can create all the kind of stuff we used node.js for almost as easily in Rust as well. Thanks to Cargo and crates providing the ecosystem of NPM and node modules. Thanks to the likes of Rocket and serde for WEB stuff and all the other useful crates available.

It's so much nicer to have only one language all the way through.

4 Likes

Hey, well, if you like Rust and Python, then what could be better than making Python modules in Rust with Pyo3!

I love it. I think Python's advantage is its ease and availability for sure. Rust is by far my favorite, but I think Python is a great fit for a customization language whenever you want to allow some extra user-logic.

2 Likes

I'll look at that when I get a little more experienced for sure! Unfortunately I can't use Rust at work, but from the languages we use at work, Python is definitely my favourite.

1 Like

You've just got to sneak it in their all stealthy like. :male_detective:

After all it is a Python module, it was just written in Rust! :stuck_out_tongue_winking_eye:


OK, OK, maybe don't break the rules I guess. :roll_eyes:

1 Like

Sneaky :slight_smile:

I'd rather not... Maybe in a year or two. You see, I'm the junior guy with no formal education except high school. The two other programmers are electronics engineers. They wanted someone who doesn't necessarily have any skills yet, but who they can train, to help out. I wrote I know a little Rust in my CV. During the interview, the software specialist engineer said "There's nothing Rust can do that Python can't". I didn't correct him, but it gives you an idea of how much he knows about it, and I'm still too junior and stupid to try to throw my weight around.

I answered in the interview that I don't think Rust is mature enough for the company's uses yet. I still stand with it. STM32's C HAL libraries are good enough, and we already know it. It's a small company. The cost of porting everything to Rust isn't small. And I believe the official STM32 HAL is better supported and more complete than the Rust open source HALs. And Python is good enough for our IoT servers. We don't have that high throughput on them. And again, we already have some infrastructure written in Python and my colleague already knows Python and its libraries. There's no need to spend time, and thus money, for him to learn Rust. If he gets interested and learns it in his spare time, it'd be awesome. I can hope...

4 Likes

For one so young you are very wise.

Everything you have said there is spot on. Learn everything you can from those guys. When you have earned your stripes you can rock the boat a bit.

When it comes to introducing Rust, or anything really, it might have to be a little proof of concept side project. You might have to get it started on your own time.

7 Likes

I'm a bit late to the party, so I'm not sure if this has been mentioned yet, but you can get similar behaviour through the Hindley-Milner type system. It essentially tries to determine the flow of types through programs to determine the most 'general' type for the whole program. It's most notably used in ML-descended languages, such as OCaml (and Rust![1]).

AFAIK, static typing isn't used in scripting languages mainly due to compile time. When running a scripting language, a quick start up is expected—type-checking, and other similar 'expensive' compile time transformations get in the way of that.

Additionally, dynamic scripting languages allow for greater flexibility while computing. For instance, Python's duck typing allows the creation of new objects that can be used in the place of others. Although this is similar to traits in Rust, the ad-hoc nature of the system increases the efficiency of quickly implementing such objects.

[1] Rust still requires function type signatures, among other things, so one may argue that it doesn't use the whole system.

I have no idea of the details but my understanding from various presentations is that modern Javascript engines like V8 tackle this problem by not optimizing code for a given type during start up. But if they notice that a particular function is used repeatedly with the same type parameters at run time then they will then JIT the function for use with those types so that it runs faster the next time it is called. Similarly they optimize structure access if they do not change their shape.

Then if the program ever calls the functions with different type parameters or modifies a structure shape those JIT optimizations are thrown away and it reverts to regular interpreting.

My experiments with V8 bear out this suggestion. I have seen JS programs start out slow on the first few iterations and then get into their stride.

It all sounds very hairy scary to me!

Being able to modify objects any how at run time certainly allows for quick and dirty hacking. But it does mean that in a large program it's hard to know what you are dealing with at any point in the source code.

JIT compilation provides various optimizations while benefitting from a quick start up. The main issue with JIT (if there even is one) is implementing one in an efficient manner, viz. one that compiles 'hot' code without interrupting execution. Although a JIT compiler can be easy to implement, writing a good one is difficult—naive implementations slow short programs down rather than speed them up.

IMO, elegant solutions arising from 'quick and dirty hacking' are the raison d'ĂȘtre for most scripting languages. When working on larger software projects written with a scripting language, I totally agree that static approaches should be taken.

In the context of this thread, I think that this is why @L0uisc suggested that Python complemented Rust. Although they differ in syntax and semantics, the underlying 'theme' of both languages is strikingly similar. They both straddle the line between imperative, functional, and object oriented. They both allow for rapid development within an expressive paradigm. However, because neither are used in exactly the same way for the exact same things, it's clear what the best language for a particular problem is. I think this similar-yet-complementary dynamic is why Rust is such a great systems language for Python developers, and why Python is such a great scripting language for Rust programmers.

1 Like

Yep.

That is why my current preferred languages are Rust and Javascript.

Mind you, I'm don't see how "elegant solutions" and "quick and dirty hacking" fit together.

From the point of view of that underlying theme I feel that Javascript is far more Rust like than Python.

But hey, what does it matter? Getting some people who only know Python or Javascript or whatever to explore something else is a good thing. Rust with it's cargo and crates and other comforts might be just the thing.

1 Like

For example, let's say that you want to trace what methods are called on an object. In Python, you'd just write something like:

class Logger:
    def __init__(self, inner):
        self.__inner = inner

    def __getattr__(self, attr):
        print(f"Calling {attr}")
        return getattr(self.__inner, attr)

    def __repr__(self):
        return f"Logger({self.__inner})"

x = Logger([4, 1, 2])
x.append(3)
x.sort()
print(x)

# Output:
# Calling append
# Calling sort
# Logger([1, 2, 3, 4])

Although this is quick and dirty, it's pretty elegant - it just logs and calls what's inside it. By default, it's generic across types, and doesn't require much work on the part of the programmer to implement. It's pretty easy to stumble upon short succinct solutions to complex problems in Python.

4 Likes

On another note, I wonder what a small scripting language analog to Rust would look like. Starting with Notes on a Smaller Rust, I think a language with similar constructs to Rust but with a looser type system and no lifetimes (garbage collected, or compile time gc'd like Micro-Mitten) would be a joy to use. It'd have structs, traits, enums, and everything else there's to love about Rust, but with more dynamic typing, more general built-in types (I'm looking at you &str) and so on.

2 Likes

You might want to check out some of the projects listed here.

It's feasible in Moss too:

class Logger = {
   function get(attr)
      print("Calling {}" % [attr])
      inner = record(self)["inner"]
      m = inner.(attr)
      if m: Function
         return |*argv| m(inner; *argv)
      else
         return m
      end
   end,
   function string()
      "Logger({})" % [record(self)["inner"]]
   end
}

function logger(inner)
   table Logger{inner = inner}
end

x = logger([4, 1, 2])
x.push(3)
x.sort()
print(x)

It's feasible in many scripting languages. I was just pointing out that Python (and scripting languages in general) have dynamic metaprogramming and introspection features that allow for a number of cool techniques. This is a Rust forum, so I wouldn't want to stray too far away from the topic at hand, but most scripting languages have tools available to get things done quick. For example, to parse a 2D comma-separated array from a string, one might do:

def array(string):
    return [[int(x) for x in line.split(",")] for line in string.split("\n")]

In a scripting language, whereas in Business Grade Javaℱ you'd have to do something like:

ArrayList<ArrayList<int>> array2DFromString(string: String) {
    ArrayList<ArrayList<int>> array2D = new ArrayList<>();
    String[] rows = string.split("\n");

    for (int i = 0; i <= rows.length; i++) {
        ArrayList<int> row = new ArrayList<>();
        String[] items = rows[i].split(",");

        for (int j = 0; j <= items.length; j++) {
            int number = Integer.parseInt(items[j]);
            row.add(number);
        }
        array2D.add(row);
    }
    return array2D;
}

Note: rust stacks up suprisingly well:

pub fn array(string: &str) -> Option<Vec<Vec<usize>>> {
    string.split("\n").map(|row| row.split(",").map(|item| item.parse::<usize>()?).collect()).collect()
}
2 Likes
# Note: split is a monkey patching module, not a function.
use string.split

# A weak spot, currently there is no "auto trimming".
# Let's fix that for the moment.
_int_ = int
int = |s| _int_(s.trim() if s: String else s)

# Here we go:
array = |s| s.split("\n").map(|line| line.split(",").map(int))

# Alternatively:
array = |s| list(list(int(x)
   for x in line.split(","))
      for line in s.split("\n"))

You know, what the heck. Let's design one. Rust has about three key concepts:

  • Algebraic data types
  • Lifetimes
  • multiple immutable XOR single mutable

I feel that for a scripting language, lifetimes aren't the most appropriate as scripting languages don't need to provide strong memory guarantees, so we'll scratch that. Let's start with algebraic data types.

In terms of 'atomic' data types (using the lispy definition of 'atomic'), we've got the following:

  • Bytes (xFF, x00)
  • Integers (10, -420)
  • Floating point numbers (-20.20)
  • Strings ("Hello, world!")
  • Booleans (True, False)

Vecs and HashMapss are so ubiquitous at this point they deserve to be their own thing. In terms of combinatory data types, we've got:

Structs, a set of identifiers mapping to data types:

struct <Name> {
    <field>: <Type>,
    ...,
}

Enums, a set of variants with an optional data type:

enum <Name> {
    <Variant> <Type>,
   ...,
}

Tuples, a sized list of different data types

(<item of Type 1>, ...)

Vecs, a dynamically sized collection of the same type:

[<item of Type>, ...]

Maps, a dynamically sized collection mapping one type to another:

{
    <item of Type 1>: <item of Type 2>,
    ...,
}

I don't think that a Rust scripting language would forgo static type checking, rather, it would use a flexible Hindley-Milner type inference system. This would make it possible to forgo type annotations in function definitions. Additionally, with the work recently done on compile-time garbage collection, it's totally feasible that a new Rust scripting language could do that too.

One final key feature of Rust is impl, which allows traits to be implemented for specific types. I think types should be able to have impl statements, though traits need not be defined. rather something similar to Go's interfaces or Python's duck typing should be used. Finally, semicolons need not apply.

This is all a fairly pointless what-if exercise, so here's a sieve:

fn sieve(limit) {
    primes = [2]
    for n in 3..limit {
        for prime in primes {
            if n % prime == 0   { break                 }
            if n >= sqrt(prime) { primes.push(n); break }
        }
    }
    return primes
}

Wow little did I know, but I should have guessed, that my suggestion for determining data types at run time in a Rust like scripting language here: Rust-like scripting language - #5 by ZiCog, was invented in the late 1950's and has a name.

Mind you, had I ever read anything about Hindley-Milner that started like Wikipedia "A Hindley–Milner (HM) type system is a classical type system for the lambda calculus with parametric polymorphism." I would never have guessed it was anything to do with determining data types in a scripting language.

What you have nicely outlined there is just what I had in mind. If only I had the chops to build such a thing.

2 Likes