Rust for enterprise grade systems?

Not really. “OOP best-practices” preach that you shouldn't even think about these complexities before writing code.

Technically objects are not supposed to organized in some random hierarchy, and technically you should think about what you are doing but the “best practices” say that time for that would arrive later, when you would use DI system to create a call graph.

Kinda like “dynamic typing”, just with call graphs. Opposite of the structured programming.

Yes, but the answer is well-known: without OOP it's hard to write code in that “first program, then start thinking” fashion. Yet I'm not entirely sure neither if that's needed nor if it's even desirable.

It's true that “doman knowledge” is hard and trying to pull it from non-IT workers is even harder. I'm still not sure it's good idea to rush coding and start doing things while you don't possess it.

Tight coupling vs loose coupling. Static typing vs dynamic typing. And so on. What you lose in “flexibility” you win in stability.

Most likely enterprise software as Wikipedia defines it.

From the use of “enterprise grade” it just feels as if we are talking about something of “high grade” (like Windows: Home, Pro, Enterprise), but in practice enterprise software is the most awful piece of shit you may ever see or touch. Understandable: Android is made for billions of people, but there are never even million of users of enterprise software, often there are only hundred or thousand of users! One couldn't afford to have good software engineers to develop it, math simply wouldn't work.

Yet Rust is viable, these days — as long as someone keeps price of LLM bots low. You can not develop “typical OOP pile of classes” with it, but you can change Rust code quickly when requirements are discovered.

I really wonder what that is supposed to mean. Rust has been perfectly viable for me the over six years I have been using it.

That wikipedia definition of "enterprise software" talks about the application areas, it says nothing about the languages or any language qualities or tools used to create it.

As Java is on the table here we know that "Enterprise" was a marketing term for the purposes of selling Sun's Java and collection of libraries and frameworks.

Yes — and that's the important part.

You just need to apply the logic to see what it would become.

  1. The users of said software couldn't avoid it, it's imposed on them.
  2. More often than not this software, by itself, doesn't bring anyone money, means it's pure waste, from accounting purposes.
  3. The desire to make it cheap and the lack of the need to fight for the users means the #1 goal becomes fight for the cheapest possible implementation. This means it's made by some poor people somewhere in Poland or India who have zero idea what they are doing or why.
  4. Nonetheless bugs in it may lead to huge losses which means there are usually some core (usually an SQL server if not some 50 years old COBOL program) that's actually developed by competent people which keeps the whole thing from falling apart under its weight. And there are usually some highly-paid “enterprise architects” who build a core that have to be pretty robust from components written by idiots.

This makes Rust a bad fit: you couldn't write code without understanding what you program needs to do with Rust only to radically change the design when someone looks on what have you done and tells you that you have built the skyscraper instead of airplane.

Yes, but it worked. Rememeber?

So this week, we're introducing a general-purpose tool-building factory factory factory, so that all of your different tool factory factories can be produced by a single, unified factory. The factory factory factory will produce only the tool factory factories that you actually need, and each of those factory factories will produce a single factory based on your custom tool specifications. The final set of tools that emerge from this process will be the ideal tools for your particular project. You'll have exactly the hammer you need, and exactly the right tape measure for your task, all at the press of a button (though you may also have to deploy a few configuration files to make it all work according to your expectations).

The goal here to write code without known what that code is actually supposed to do, because in “Enterprise software” you just never know, they only way is to present the code and then hear workers cries of anguish, which may help you to understand where exactly you have screwed up.

You no longer need factory factory factory approach. Just write code for the app from the spec, try it, find out spec was someone's braindump under drugs, write another code for entirely different app. Eventually you have a chance of converging on something small and userful and working without all the baggage that usual “enterprise development” leaves behind.

Holy shit this video was so much fun!

I did research for that matter, Linus Torvalds uses OOP and improvised vtable by means of plain C struct and function pointer. Looks like no way to escape from that. Java just made it part of the language. Same about C++.

This factory-factory made me laugh and cry at the some time. Some people just building something without understanding why they ever building it.

I found out, that best approach to build some good framework, is to focus on some real problem needed for business, at some point, you just figuring out, that without putting some reusable code together into framework, you just need to keep adding lot's of repetitive boiler plate code.

But the problem is, when you try to implement something too generic, you'll find out soon, that you generic controversially doesn't fir into some very specific requirements of current business.

Some frameworks really can do a great job to be something that can fit any size, like Spring Framework, but if you dig deeper there let's take Spring MVC as example, you'll realize they made so many customization inside and so much work done, to provide support for every possible feature that anyone may need to build an HTTP server.

And add backward and forward compatibility into that, which makes everything even more fun.

I hardly imagine how Rust can do it without good OOP, or something same powerful as OOP.

How much exposure to functional programming and Rust programming do you have? It kind of seems like you're trying to write Java in Rust

Argument ad-hominem, against community standards and common sense.

Show the code instead, how you would do it with functional programming.

If you have good solution, I would appreciate it.

I withdraw from this thread, apologies if I upset you. :slight_smile:

I don't know what you mean by OOP exactly. You are talking about Java but "Object Oriented Programming is not well defined, indeed the originator of the term specifically said he did not mean it to be like C++/Java.

We see lot's of huge projects written in Rust so it's not clear to me that OOP is required.

I really like how this topic moved from a particual problem (which I still struggle to understand) to a lovely "Rust and OOP" discussion.

The problem with OOP is that it doesn't have a strict definition.

If we accept that object is just a blackbox and we do not access its internal structures directly, but only call its methods (send it messages in Smalltalk terminology) then Linux drivers and VFS are, indeed, excellent examples. And it isn't Linux-specific thing: every OS (NT included) do pretty much the same: you register a structure with a function pointer that must called. The system builds a tree of these structures.

The good news is Rust does support it: Not only you can convert a plain old structure to an object by hiding its state, but also create a polymorphic, vtable-based code by means of dyn traits

N.B.: this is not awlays the right thing to do. In Java, for example, they implement every single structure as an object, and you can't really reasoning about the performance. I mean, I know perfectly how much does it cost to access name field in a structure, but have no idea what does getFoo do. It might send a spaceship to the moon.

But let's get back to Rust and OOP: the only thing Rust doesn't support is implementation inheritance.

And implementation inheritance considered a bad practice by lots of enterprise developers, because it makes systems really hard to understand and almost always leads to inconsistency.

J. Bloch in his "Effective Java" book wrote that inheritance (unlike interface implementation of course) must be prohibited by default, unless carefully designed, so all classess must be final by default.

Kotlin took this idea further and made everything final unless opened explicitly.

M.Fowler also critisized implementation inheritance and suggested to replace it with composition.

So it is common knowledge in OOP world, that pure abstract classes (known as interfaces in Java/Kotlin/C#) and final classes with implementation are better than implementation inheritance.

There are some other things Java can and Rust can't, but most of them are anit-patterns.
My favorite example is a runtime type check which is always a sign of Liskov Substitution Principle violation: one has an interface Foo and one of its implementations doesn't really implement its contract (methods do exist, but do wrong things) and one spreads (if (foo instanceof Foo)) all over the code. Rust doesn't have it and I really do not miss it.

Rather than trolling for a Rust-vs-OOP discussion, I think it is up to you to show that Rust cannot do it. You have a lot of experience with Rust and OOP, so I'm sure you could do this as well as anyone.

Let's look at the largest Rust system I could find - Cloudflare's DNS, codenamed "Big Pineapple".

Surely it has to support many different specifications, have a trait abstracting over the stored DNS records, etc?..

Looking inside... Just a enum.

pub enum RecordData {
    // Small and common variants are stored inline
    A(Ipv4Addr),
    Aaaa(Ipv6Addr),
    // Large variants are stored on the heap
    Txt(Box<Txt>),
    Naptr(Box<Naptr>),
    Svcb(Box<Svcb>),
    // ...
}

To supply some personal experience, when I started learning Rust (after I grasped the idea of borrows requiring that values remain in their places untouched) I often had to refactor a whole chain of methods, like six at a time, to pass a new borrow through them.

I see how that could be an issue if your refactor tries to touch a function owned by another team in the corp.

Nowadays, I predict the API shape more precisely so I don't need to change as much code at once.

Don’t worry about upsetting me, because I’m not right in every situation.

But I would be worried about you if you made God upset.

You’re welcome to join the conversation. If you can provide a useful example of how you would solve a similar problem using functional programming, I would really appreciate it.

I was thinking about enums. Many people say that Rust uses enums instead of classical OOP.

And this is what I came up with.

Let's say we already have a trait Collection, so we need to define an enum that will implement that trait:

enum Collection<T, I: Iterable<T>> {
    Array(Array<T>),
    Vector(Vector<T>),
    LinkedList(LinkedList<T>),

    // Covers both Set, BTree, etc.
    Iterable(I),

    MutableSet {
        set: MutableSet<T>,
        counter: usize,
    },
}

enum CollectionIterator<'a, T, I: Iterable<T>> {
    Array(/* Array iterator type */),
    Vector(/* Vector iterator type */),
    LinkedList(/* LinkedList iterator type */),

    // Iterator associated with I
    Iterable(/* I iterator type */),

    MutableSet(/* MutableSet iterator type */),
}

impl<'a, T, I: Iterable<T>> CollectionIterator<'a, T, I> {
    fn next(&mut self) -> Option<&'a T> {
        match self {
            Self::Array(it)      => it.next(),
            Self::Vector(it)     => it.next(),
            Self::LinkedList(it) => it.next(),
            Self::Iterable(it)   => it.next(),
            Self::MutableSet(it) => it.next(),
        }
    }
}

impl<T, I: Iterable<T>> Collection<T, I> {
    fn iterator(&self) -> CollectionIterator<'_, T, I> {
        match self {
            Self::Array(v) =>
                CollectionIterator::Array(v.iterator()),

            Self::Vector(v) =>
                CollectionIterator::Vector(v.iterator()),

            Self::LinkedList(v) =>
                CollectionIterator::LinkedList(v.iterator()),

            // Works for Set, BTree, or anything else implementing Iterable
            Self::Iterable(v) =>
                CollectionIterator::Iterable(v.iterator()),

            Self::MutableSet { set, .. } =>
                CollectionIterator::MutableSet(set.iterator()),
        }
    }

    fn size(&self) -> usize {
        match self {
            Self::Array(v) =>
                v.size(),

            Self::Vector(v) =>
                v.size(),

            Self::MutableSet { counter, .. } =>
                *counter,

            // LinkedList and ANY Iterable share this implementation.
            _ => {
                let mut size = 0;
                let mut iterator = self.iterator();

                while iterator.next().is_some() {
                    size += 1;
                }

                size
            }
        }
    }
}

Something we still can't completely avoid with enums in Rust: even if, let's say, all our collections implement the same trait Iterator, we still can't write a generic match arm without using dyn Trait and dynamic dispatch, because each implementation can use a different concrete type under the hood, as workaround I've tried to introduce enum CollectionIterator to keep it generic but if feels like workaround.

Enums can be flexible still. Another limitation is extensibility: if you need other code to extend your abstraction, enums are primarily closed for extension, so you need some additional mechanism to make them extensible:

  1. A trait is open over types: somebody can define a new type later and implement Collection for it without modifying your code.

  2. An enum is closed over variants: adding another collection normally requires modifying the enum and every exhaustive match.

What I'm also curious about here is benchmarking.

Traits can naturally be monomorphized, while with an enum there is always a pattern match, and we have to hope that the compiler can optimize it away.

However, for closed systems, and with some optimization luck, perhaps we can use enums instead of OOP.

Enterprise systems are very often open systems: there is always something being added from outside.

On the other hand, in complex enterprise systems there are also many situations where we deliberately want to keep something closed inside our implementation. The less API we expose, and the more things we can keep private, the fewer problems we have.

I don't think enums can completely replace OOP.

dyn Trait is effectively an enum with arbitrarily many variants, one for each concrete type used that implements Trait. When you have a closed set, I have absolutely seen OOP code still use the equivalent of dyn Trait anyway, instead of an enum (commonly because the language the code was written in has insufficient support for enums). (And that choice is usually less performant.)

There are several approaches:

  1. You declare all possible items explicitly and choose the right logic each time you use them by means of exhaustive pattern matching. To add a new item, you would need to change all usages. It liberates your items from any logic, and moves logic to a call site. This is implemented by tagged unions in TypeScript or enums in Rust, or sealed classes in Kotlin, or a "visitor" pattern in every single classical OOP lang: Java, C#, C++, ObjC etc.
  2. You declare an interface (or a trait or an abstract class or a protocol in Objc/Swift) and work with it polymorphically by means of virtual calls. That way, you can add new implementations without touching a call site (as long as you obey Liskov Principle), but now all your items contain logic.

These two options exist in almost all languages, and they existed loong before Rust.
Techincally, the first approach is popular in Rust because it might be kinda cheaper (you do not need virtual call here, this is unlike, say, Java) and less noisy, but it can also be a wrong thing if one option takes 8 bytes and another takes 256 and almost never exists.

But let's leave performance questions alone. The question is where your logic should live. If you want external people to write plugins for your system, then the only option is 2, of course.

Okay, I'll throw in some thoughts then. :slight_smile: I won't get too in to the technical details as others in the thread have already solved this and shown how you can write the code in Rust. I saw someone on reddit replied to your thread too with an example that you said was okay, so I'm going to assume that part of the puzzle is finished.

But if we're going to try and make a Rust version of a collection it would be good to know what a valid Collection is. For instance, is it valid to make a Collection that returns -1 for size (it's infinite) and returns random numbers for each iteration? Does this fulfil the contract? Is it just whatever doesn't break existing code?

You've talked about having a Collection that proxies a database with no iterator and just size, so I imagine this is similar and requires a way to return errors too.

Also if an iterator isn't intended to provide size, why is the abstract implementation using it to calculate size? Shouldn't it just return nothing? It seems like we trust iterators on some level to provide a size.

I get the feeling the answer to all of this is that you don't want to provide concrete constraints on what interfaces do because in enterprise land things change and the consumers decide the constraints.

For example, let's say you have some logic that sums N elements of a Collection. If you have an infinite stream of random numbers, it sure would be good to re-use that logic. Instead of duplicating code or making a new stream interface, you could just model it as a Collection and get the code for free.

I think Rust land is resistant to this type of development, you would instead spend time refactoring and specifying your code and updating constraints.

It's true the Linux kernel does use OOP to some extent, but it's worth remembering that the kernel is a not an 'open' system in that you have multiple developers injecting or adding new components. For example if I wanted to change the kernel to allow higher precision clock measurements I would be expected to modify all interfaces and code that uses the clocks. I wouldn't create a new clock type that some code uses and others wouldn't.

that you said was okay

Really? There were example somewhere close, but never close enough.

I am really stopped digging more, once found how Linus Torvalds doing :slight_smile: I was literally felt stupid that Linus was able to do something in C, but I can't do same without Java, and then I've found improvised vtable on structs, so it got me back on my feet :wink:

For instance, is it valid to make a Collection that returns -1 for size.
Don't bother about such exception case. I was pushing more to not use size of iterator, so I've used that example of infinite iterators which very often. Regarding Collection, I can't remember any example of infinite collection, but it can be enormously if it is proxy collection over database.

You've talked about having a Collection that proxies a database with no iterator and just size, so I imagine this is similar and requires a way to return errors too.

Also if an iterator isn't intended to provide size, why is the abstract implementation using it.
Rust implementation of Iterator has size as property. I am not really criticizing that rust already included that property there.

But if speak about clean minimalistic design, this is what Iterator meant to be:

fn process<I: Iterator>(mut iterator: I) {
    while let Some(value) = iterator.next() {
       // process value here
    }
}

Many people tried to hack through Iterator::count and perhaps it could be a valid solution for real-life problem in rust coding, but this question more about software design, let's do it without hacks like Iterator::count, just use Iterator to iterate only, what it meant to be at first place.

I think Rust land is resistant to this type of development, you would instead spend time refactoring and specifying your code and updating constraints.

That's what I am thinking too... Initially I wanted to make that post on internal.rust-lang where people more focused on discussion how they see future of rust.

This is a little bit outside of scope for this Collection question, but here is the concrete problem that I see specifically for rust. I think you know that trait From in rust, very convenient one, and I am using a lot in my code.

Let's say I am designing similar trait MyFrom:

pub trait MyFrom<T>: Sized {
    fn from(value: T) -> Self;
}

// Obviously Blanket implementation is usefull here:
// if F can be created from &T,
// then F can also be created from T.
impl<T, F> MyFrom<T> for F
where
    F: for<'a> MyFrom<&'a T>,
{
    fn from(value: T) -> Self {
        MyFrom::from(&value)
    }
}

Now if I will try to put concrete implementation from &A -> B:

impl<'a> MyFrom<&'a MyConcreteTypeA> for MyConcreteTypeB {
    fn from(value: &'a MyConcreteTypeA) -> Self {
        MyConcreteTypeB { value: value.value }
    }
}

I will get an error:

1. error[E0119]: conflicting implementations of trait `MyFrom<&MyConcreteTypeA>` for type `MyConcreteTypeB`
     --> src/main.rs:26:1

But the point is that, even if I've already made a generic defintion that any implementation for MyFrom<&T> should make blanked implementation for MyFrom<T>, I still should be able to override that blanked implementation with my concrete implementation MyFrom<T>, if I need so.

impl<T, F> MyFrom<T> for F
where
    F: for<'a> MyFrom<&'a T> { ... }

Shouldn't go into conflict with:

impl MyFrom<MyConcreteTypeA> for MyConcreteTypeB {...}

Just most concrete implementation should be chosen, such feature is know in C++ as specialization. Java doing something very similar during method overloading. In C++ specialization is actively used together with templates, I think Rust just missing the proper implementation here, it doesn't even requires classical OOP, and I guess rust will stop resisting.

This example is just the tip of the iceberg, but I think there is a lot more to explore here.

And what's important, we don't even need to re-invent the wheel, we just need to learn how it is done in other languages.

but it's worth remembering that the kernel is a not an 'open' system

Linux kernel is monolithic but it is open system:

You can even load and unload kernel modules dynamically. I think Linux worth learning how to build complex software.

if I wanted to change the kernel to allow higher precision clock measurements.

I guess you are speaking about some very core functionality, where Linux heavily depended on that? I think there was demand to modularize that driver for that hardware, it would be possible through kernel modules, just was never necessity for that. In many places where something needs to be modularized, Linux is extensible.

I would be expected to modify all interfaces and code that uses the clocks.

Backward compatibility is hard, no-one demands that. And by the way, that's why I am saying that property Iterator::size is example of bad design, once you add that property into core API, you can't simply remove it from there, now you need to support it in every possible places.

So this java Iterator definition:

public interface Iterator<E> {
    boolean hasNext();
    E next();

    default void remove() {
        throw new UnsupportedOperationException("remove");
    }

    default void forEachRemaining(Consumer<? super E> action) {
        Objects.requireNonNull(action);
        while (hasNext())
            action.accept(next());
    }
}

And this is rust Iterator:

pub trait Iterator {
    type Item;

    fn next(&mut self) -> Option<Self::Item>;

    fn next_chunk<const N: usize>(
        &mut self,
    ) -> Result<[Self::Item; N], array::IntoIter<Self::Item, N>>
    where
        Self: Sized,
    { ... }

    fn size_hint(&self) -> (usize, Option<usize>) { ... }

    fn count(self) -> usize
    where
        Self: Sized,
    { ... }

    fn last(self) -> Option<Self::Item>
    where
        Self: Sized,
    { ... }

    fn advance_by(&mut self, n: usize) -> Result<(), NonZero<usize>> { ... }

    fn nth(&mut self, n: usize) -> Option<Self::Item> { ... }

    ...
    
    // and who knows how many more methods goes here.... 

}

If you need to make your concrete implementation for Iterator rust trait or java interface, which one would be easier to maintain?

From this conversation here, I've learned that people expecting that when they doing Iterator::count, you will provide something optimal from computation complexity, not just O(n) to iterate through entire sequence how it is done by default.

Java designed many time with a mind, that they offer you some framework, but literally for everything you can do you own implementation. If we take Iterator as example, it is literally three method, where one of that method is optional. Except next and hasNext that used for iteration. You can tell me that java offers Iterator::remove() as excess, however it is hard to argue against that without that Iterator::remove(), java collection API would be complete.