Why doesn't rust provide any decent way to extend external structs?

An issue I have been running into a lot with rust is that I need to add additional functionality (such as additional methods, traits, or related data) to structures from external dependencies.

From what I can tell, your options for doing this are:

Extension Trait: This works if you just need to add some additional methods, but you can't add your own fields, you can't add private methods, and obviously it doesn't help with adding traits (such as Eq/Ord, for an obvious example) so it's only viable for a small subset of the problem. It also results in unnecessary boilerplate since you have to stub out all your methods in the trait definition even though you obviously never want anyone writing their own impls for your trait.

Wrapper struct: You can add your own traits, methods, and fields, but you've created a new type so you can't directly use it anywhere you'd normally use the underlying type, and accessing methods and fields of the underlying type are also not ergonomic. It seems you can use Deref/DerefMut to be able to pass the wrapper type, where the underlying type is needed, but this only works with references and seems to be frowned on by the community. For accessing underlying methods you can also use macros to generate delegate functions for accessing underlying methods but that doesn't help with fields.

Neither of these options provide a complete solution. Coming from an OO background, this is incredibly frustrating since this seems like a fairly common problem and in an OO language it can usually be easily solved with inheritance. Am I missing something big here or is Rust just really lousy in this area?

1 Like

Neither. Design patterns that work in OO languages often don’t translate very well to Rust (and vice versa), but the space of problems that can be ergonomically solved seems comparable.

Supporting inheritance (and especially multiple inheritance) adds a lot of complexity to the language, making programs harder to reason about for both the compiler and programmer. Rust has chosen to spend its compexity budget elsewhere.

11 Likes

You are not missing anything.

OOP is a really lousy way to architect software. Even the C++ guys preach that today.

Have a google search for "favor composition over inheritance" to find out why.

I'm pretty sure that whatever it is you want to do is better done without inheritance.

Until proven otherwise of course.

7 Likes

All of this really sounds like you're simply trying to shoehorn rust into behaving like an OOP language, Rust isn't one. Instead of trying to fit inheritance into rust, try to rethink what you are trying to do in a non OOP way.

I don't think rust is any more lousy in this area than a ship is at being a bus, it's not supposed to be one.

5 Likes

That is an interesting analogy. Can I play with it?

Base class: A thing that floats on water.

Boat class: Can carry people over water. Derived from "A thing that floats on water" and possibly from "People"

Ship class: Can carry people over water and has a motor. Derived from Boat and likely "Motor"

Oh crap, now I need a thing that has motor and can carry people over land. OK, possibly I can make one out of a new base class and "People" and "Motor" and call it a "Bus"

Oh crap, now immigration wants to stop any Bus or Ship at the boarder.

Not because it is a ship or a bus but because it has people in it.

What to do?

2 Likes

I was implying Bus vs Ship as Rust vs OOP languages, but sure go ahead :joy: I always like to think of OOP as real world objects

abstract class CanFloat

class WaterTransport extends CanFloat implements Transport?
class Ship extends WaterTransport implements EngineDriven?

class LandTransport implements Transport?
class Bus extends LandTransport implements EngineDriven?

damn this is already getting messy :sweat_smile:

can interfaces implement other ones, like
interface Transport implements ContainsPeople? :thinking:

immigration prolly doesn't like anything that is a 'ContainsPeople'

1 Like

Even in "traditional OOP", just because a class can be subclassed doesn't mean it can be usefully extended. Shelves of books have been written on the topic of "how to make code extensible". You don't get it for free just because you're using a language that supports inheritance.

Of course, in Rust you don't get inheritance at all. So you have to use other ways. And there are a lot of other ways. Many, maybe most of the techniques in those books I mentioned actually don't rely on full-fledged inheritance. Rust has inspired other patterns that probably aren't in books yet. But yeah, there's a learning curve.

If you could give some examples of where you found something particularly troublesome, people here would be more than happy to help you with it.

3 Likes

To my mind that sums up the problem with OOP. In the "real" world there are no "objects".

Today you are "zireael9797", presumably a member of the class "Human" with properties like nationality, social security ID, and so on. Some kind of OOP object.

In a hundred years time whatever you are now will be dust in the wind. Do we have class for "Dust" ?

Such "real world objects" only exist in our minds. We define them how we like.

More seriously, when it comes to composing programs it would be nice to be able to reuse some properties of something, without having to pull in ten layers of whatever it inherited from that we don't care about.

Perhaps I have had bad experiences but OOP systems I have been involved with actively fight against code reuse. Want a part of it? You need the whole thing.

1 Like

Here's what prompted this thread: I'm currently working on a project where I need to read a pcap file and ensure the packets are in the correct order, extract some metadata about the packet that's tacked on to the end of packet data, and then mutate the packet data in a number of ways, some of which are based on the metadata. I'm currently using ext traits for Packet and Vec to mutate the data and override sort/sort_unstable since Packet doesn't implement the traits required to be sortable on its own. I'm currently loading and handling the metadata separately and just passing a reference to it to any methods that need it but that hurts code reusability as anyone who needs to use any methods that require the metadata will have to to do the same even though they logically shouldn't since that information is technically part of the packet.

I'm wondering if maybe I should instead just make my own separate Packet type and convert to it from the third party Packet after reading and then convert back into it before writing but am unclear if this has a significant performance cost.

2 Likes

This is what I'd do. The From and Into traits make it quite painless.

Regarding performance, if you implement From to just move the fields across (possibly adding your own fields with default values) it shouldn't be expensive. It gets expensive when you clone() large buffers, but if you're just moving the Vec<u8> (or however the packet's raw data is represented) then you'll just copy the data pointer and two usizes... The cost for this conversion will be comparable to passing a Packet into a function by value.

The usual caveats around performance apply... Don't just rely on the opinions of random people on the internet. If performance is important to you and you think a change could have a negative impact, create a benchmark with a typical use case and run it using both the original code and the new code.

7 Likes

Anyone correct me if I'm wrong, but if inheritance existed in rust, wouldn't that also cause performance costs? Like in the case of rust you need Box<dyn Trait> to use trait objects. Wouldn't the same rule apply to Parent and Child types?

struct Foo {}

struct Bar extends Foo {}

fn baz(val: Foo) {}// <--- This wouldn't work since children of Foo could have different size from Foo

fn baz(val: Box<dyn Foo>){} //<--- Wouldn't this be required? and doesn't this require heap allocation?

fn baz(val: impl Foo){} //<--- this would probably work here but would cause limitations later, you'd still need Vec<Box<dyn Foo>>

This would require Heap allocation and wouldn't that generally cause performance costs itself?

Yes, supporting inheritance fully makes all types unsized.

4 Likes

If you only depend on a foreign trait for your extension trait, then this is how you can easily write extension traits with nobody else being able to implement the trait for arbitrary types:

trait MyTraitExt: TheirTrait { […] }

impl<T> MyTraitExt for T
    where T: TheirTtait { […] }

If you only create a newtype to work around the restriction of not being able to implement a foreign trait for a foreign type, then marking the inner type with pub is appropriate.

Instead of implementing Deref(Mut), go with Borrow/BorrowMut.

Rust doesn't always provide an obvious/easy solution to structural problems, but there usually is a solution. Knowing about all the tools you're given is important and that's where I think this user forum and the official Rust Discord server help a lot. Sometimes, all you need is a hint in the right direction.

The biggest problem is, that people coming from OOP languages like Java (or even C++ to some extent) seem to struggle with translating the way they think about structuring programs to Rust. People will just have to keep struggling until they have gotten enough experience with Rust. Tutorials help a bit, but theory alone simply isn't enough.

4 Likes

Sure, but how do you provide a decent API when doing that? Having to call newtype.0/newtype.inner anywhere you need to access the wrapped type seems far from ideal.

This is good to know, but doesn't look like it will work for my particular case since part of the reason I looked at using a wrapper type is because the foreign type doesn't implement Eq/Ord

From what I've read in this thread, it seems like the cleanest way to handle my use case, where I need additional traits, additional methods, and an addition field for storing metadata is just to define my own type, convert the foreign type to it, do what I need to, and then convert it back to the foreign type since I can move all of the fields back and forth between the types without needing to clone. It doesn't seem likely to come up in this particular project, but my remaining concern is that if I needed to go back and forth between types multiple times, the metadata field would have to be recalculated each time and I'm not sure how to get around that.

1 Like

This does indeed rule out Borrow, unless you lock the version of your dependency, only accepting hotfix version bumps automatically. This might be undesirable for obvious reasons. Depending on what you depend on, either a PR to add the respective traits or cloning the project and editing the clone to suit your needs, might be possible.

Leaving Borrow(Mut) out, the only sensible traits left worth mentioning are AsRef and AsMut before going full conversion mode with From/Into.

2 Likes

I guess the metadata you are talking about is the pcaprec_hdr_s in LibpcapFileFormat
I think, then that this points to a solution you described as a "Wrapper struct". Which is essentially what would be neater on the first place. You have already some "Packet" concept. You have a "PcapRecordHeader" concept. You compose a "PcapEntry" with a "Packet" and a "PcapRecordHeader".
As soon as you want to mutate also the "PcapRecordHeader" (e.g. Wireshark time shift function) I dont think you want the 2 base concepts really on the same object. (telling from own experience with working with pcaps)

It seems then that you are worried about the usability afterwards. I think this is pretty much workable. For example, if you use iterators, you can use map and a closure to address the type mismatch easily.

This was a good thing to research, thanks ! Since I wasn't programming back in the OOP days I wasn't really able to explain why to choose composition over inheritance. Until now.

That is an odd thing. Something has disturbed me about the C++/Java etc style off OOP that took the world by storm two decades ago or so. Similarly I was never able to translate that disturbed feeling into a coherent argument either.

I do like OOP, but I always face palm when I see a "object" of 2000 lines of code inheriting from another "object" of 2000 lines of code. That cannot possibly be a "is a" relationship. Designing OO for me is always about composing small objects. Inheritance, only when it is logically congruent (and often is not).

I remember hearing that Java was so superior to C++ because it did not have free functions. "Everything is a object , that´s the way to go!". It seems the world is past that point, but I do get your same weird feeling, without being able to express why when I hear these days "Everything is a function, that´s the way to go!"...

Everyone seems to be focused on OOP here, which is, honestly, not relevant to the problem at hand. I believe what OP is looking for here is something akin to C#’s extension methods combined with Scala’s trait mix-ins.

Essentially, all structs in Rust are the equivalent of sealed in other languages. Which makes sense at some level, but I too often find myself wanting to extend a concrete type by making it implement a trait of my own creation. I’m sure there are Reasons™️ for not allowing users to extend structs defined in external crates, but I don’t know what they are.

2 Likes