Suggestion for making Rust's trait system more usable for generics

It is doing it, after reading quite some blog posts and rust internals as well as design discussions. People do mention how the choice of traits affects how it is used. For example, in the latest tokio's blog post (tokio 0.2) pointing to a discussion of trait design for async/await:

Trait can do a lot of stuff when talking about composition over inheritance. But what I felt is that there are a lot of complexity to get started, to know what functions exists, one probably need to go through the extensive list of traits, which is maybe a rather huge space O(n*m) compared to searching in inheritance (only need to go through parents + their troublesome states). Maybe some trait searcher or discovery can help?

For example, I did not know to_string() comes mainly from Display by just reading at to_string() function docs only, for quite some time. Even though the answer is just a page scroll above but it took me some time to find that (later I know that it is also in the book). https://doc.rust-lang.org/std/string/trait.ToString.html#tymethod.to_string

Regarding the breaking changes, I believe that is a semver issue rather than a trait issue.

1 Like

But traits are designed to be an immutable public interface, meant for communicating intent. The public interface for anything else should be allowed to grow without a breaking change to allow for new functionality, otherwise one of three things would happen: crates will regularly make major version increments, ctates will stagnate, crates will add a whole bunch of traits for the sole purpose of adding a few methods. None of these are appealing.

Adding new methods currently does not suffer any of these problems because in general, adding a method can't break downstream crates.

One more thing, how would this play with generics? Currently traits are tightly coupled with generics, which is why they are so powerful, but I can't see a way to do that with your proposal.

I was referring to @mankinskin's proposal, not traits as they are now

5 Likes

This is not how I was introduced to traits. To me traits are a common interface for multiple types. Keeping them stable is already the library's responsibility.

If a library designer is adding a new public function, they can put it in a new public trait. This is even to be encouraged, because it is according to the single responsibility principle. One trait should only describe one kind of behavior. If a new behavior is added, a new trait should be added.

When a public function signature changes, then that is already breaking anyways, so this wouldn't change anything about this.

Why should I have to add a new trait to my crate interface just to get the method syntactic sugar? I could just make a free function. Yes, it would be less ergonomic, but it would also be significantly easier to maintain and to explain. I wouldn't have to document what a conforming implementation looks like, as would be the caee for designing a new trait.


How would your proposal interact with generics? If it doesn't, then it is a useless burden on library writers.

2 Likes

The idea behind this was to force libraries to implement traits for their public interfaces of objects, so that these objects can be used in generics. The entire point of traits (the way I see it) is to use them with generics. You can apply trait bounds to generic type parameters. So the more traits, the more control we have over generic type arguments.

I don't see a reason why you shouldn't be able to. If you want to force a function to be implemented only with exactly one fixed implementation and a specific, possibly generic type, you could just write a free function. i.e.

struct MustUseType;

// classic fixed implementation for fixed type
pub fn fixed_operation(with: MustUseType) { ... }

// fixed implementation for generic type
pub fn predefined_operation<T: SomeTrait>(with: T) { ... }

However traits enable library users to make their own custom implementations:

pub trait UsableTrait {

    // may be implemented differently by importing crates
    fn custom_operation(self, other: MustUseType);

    // may be implemented differently by importing crates, with generic type
    fn generic_operation<T: SomeTrait>(self, other: T) 
}
...
// in another crate
struct MyType;

impl ext::UsableTrait for MyType {

    // custom implementation
    fn custom_operation(self, other: MustUseType) { ... }

    // custom implementation for generic type
    fn generic_operation<T: SomeTrait>(self, other: T) { ... }
}

Yes, I agree with this. Which is what led me to the question, how would you use types as bounds on generics? How would you adapt existing code to this new model? I didn't properly articulate this before.

I think you missed my point, say I have a type Foo which already has some methods on it,

struct Foo { ... }

impl Foo { ... }

Now I want to add a new method. How would I do that without breaking semver?

I could make a new trait, but if that addition only makes sense for Foo (for example, if I have an enum and I want to add some functions to check if it is in a certain variant, like with Option::is_some), why should I have to go through the trouble of designing a new trait? I could make a free function, but then I lose some nice method sugar. I can't just add a new method to the type directly because that would be a breaking change.

What you have there looks like code reuse by means of a sort of inheritance mechanism.

From watching some presentations from C++ gurus who know far more about OOP than I do I gather that:

  1. When C++ came about everyone jumped on OOP as a means of code reuse. One could grab a library and derive new classes from it that tweak or add things to it.

  2. Now a days using inheritance for code reuse is considered a really bad idea! For reasons I won't get into now.

  3. Such uses of classes and inheritance should only be used to define interfaces.

2 Likes

Good example. The Option::is_some method does not describe any vital behavior but is rather just a convenience method, and may be added a few releases down the road. However if enum Option were to implicitly implement a trait Option with all of it's public associated functions and methods, then all libraries which are using the Option trait are now also required to implement Option::is_some.

btw, I thought using these would work just like with normal traits impl std::Option for MyType {…} and fn uses_std_option<T: std::Option>(obj: T)

This is of course a fundamental problem with traits, or actually public interfaces for generics in general, where all members must be implemented for the interface to be considered implemented. In regular public type methods this problem does not arise because nothing external is depending on the exact, complete definition of the type's public methods, so adding a public function will only add the functionality for future releases.

This is probably another reason why traits are rarely used, and I have also already encountered cases where this blocked me from upgrading my dependencies, because one of my dependencies was not compatible with a new major release.

Of course implementing traits for everything will make crates more likely to be incompatible, because every addition to a trait will have to be implemented for all implementing types. The alternative could be to use free functions, but as you said, that also removes a lot of the syntactic sugar we would like to have.

The only solution that I see for this is to allow for default implementations in traits. This would also be how virtual function in C++ or abstract function in Java work. It would then however be interesting to know what exactly the bad consequences of such features are, maybe @ZiCog could elaborate on this further? I will do some research for myself too tough.

And since it's taught in Universities (sic), it must be good. :slight_smile:

Increasingly, industry good practice advises against inheritance, and it's not missed in Rust. It's a different language with different idioms, and it's easy to do things in different (most often better, more elegant, more decoupled, clearer) ways, instead of trying to use the OOP hammer to hammer in a screw.

Not sure specifically what you mean by that, but Rust is definitely a language that allows and encourages the use of high-level abstractions. You absolutely don't have to write leaky abstractions in Rust, and it's very easy to write good ones.

4 Likes

Rust has the OOP people should have learnt at university. I mean unless your university just taught C++/Java classes and not any theory but that would be a terrible university level CS course.

Not me in particular. I'm no OOP guru.

But I recommend watching the presentation by Jon Kalb “Back to Basics: Object-Oriented Programming” https://www.youtube.com/watch?v=32tDTD9UJCE for the low down.

There are some gems in there related to this discussion:

Use OOP to model "is-a" relationships, not for code-reuse.

From the book "Effective C++ 3rd Edition": "Consider alternatives to virtual functions"

From the book "C++ Coding Standards": "Consider making virtual functions nonpublic, and public functions nonvirtual"

I'm not sure I have the where with all to interpret that into "trait speak" but it seems to advise against what is being suggested.

2 Likes

By that I meant that Rust takes the low level seriously and is a lot less permissive than other languages. Which is fundamentally a good thing, but we should strive to make it as ergonomic as possible.

Well I am not exactly proclaiming inheritance, especially not single inheritance. What I was suggesting is to add better support for composition using traits to be able to reuse code through generics extensively in the Rust ecosystem, by allowing something like this:

struct MyType;

impl MyType {
    fn do_something_internally(self) {…} // private

    pub fn get_something(self) -> Something {
        Something::new()
    }
    pub fn new() -> Self { Self }
}
struct OtherType;

impl MyType for OtherType {
    fn get_something(self) -> Something {
        STATIC_SOMETHING.clone() // overriding implementation
    }
    fn new() -> Self { Self }
}

Now the problem arises that this will force depending crates to adapt to any additions in the public interfaces of all public types, because traits require all members to be implemented.

But here the solution are default implementations, which are already available for traits in Rust. So library authors could still greatly mitigate the breaking of a change by just implementing a default. In fact, types would always come with defaults for their trait. Whether these are applicable would have to be decided when implementing the trait for another type (possibly fields are missing).

So what does this have to do with OOP? It simulates a kind of abstract type with traits. It is basically nothing but syntactic sugar for

struct MyType;

impl MyType {
   fn do_something_internally(self) {…} // private
}

trait MyTypeTrait {
    fn get_something(self) -> Something {
        Something::new()
    } // default
    fn new() -> Self { Self } // default
}

impl MyTypeTrait for MyType { /* use defaults */ }

struct OtherType;

impl MyTypeTrait for OtherType {
    // etc
}

All I am wondering now is how to resolve trait member conflicts properly, but this should be a fundamental problem of composition aswell. Probably it would just require a <MyType as SomeTrait>::function syntax.

I made an RFC here.

So, it's still a breaking change

In what way? None of the automatically implemented traits would be implemented yet. Nothing about currently existing code would be affected by this, I believe.

If you mean it would be a breaking change if a library author implements a method that makes use of a field or private fn/method in self, that that implementation could generally not apply as a default for implementing types, then yes, then it would be a breaking change. No design can guarantee complete absence of breaking changes in the public interface of a library, though. Changes in a trait will generally always break the implementations of all of its implementers.

But apart from that, I didn't think Rust was finished with developing its trait system. With something like "data traits" traits could also depend on fields. This would also mean breaking changes when adding fields to that trait, but that is a fundamental problem which can't be resolved, because the public trait is part of the specification of the library. If the author decides to change those, it should be breaking. When you implement a trait, then you are joining the contract of that type class. This introduces coupling with the libraries generic type system. There is no way around this. All what this RFC proposes is to expose more public interface through traits implicitly, to be able to use it in generic type expressions. Because I don't see the point of having a public interface for a type, which can not be referenced as a trait. Because it is just like a trait. And most libraries don't delegate their public functions to traits. Not consciously, but simply because they don't use it as a trait. But library users might want to use it as a trait, because obviously it can be very useful to have a function which can be applied to multiple types.

Yes, this is what I meant.

The current design, where types don't automatically generate interfaces does avoid this pitfall

Yes, this is a well known fact, but addition to a type's public interface is not a breaking change.

This is ignoring adding fields to a type that only has public fields, or adding things to enums (which are entirely public), because that falls in the same category as traits, with their entirely public api.

There was an RFC to allow fields in traits, because that would allow more efficient dynamic dispatch and allow the borrow checker reason about fields even in an generic context.

Why though? Adding a field shouldn't be a breaking change, because it shouldn't change the behavior of downstream crates significantly more than adding a private field.

I think this is largely where we disagree. I don't think exposing such an interface implicitly is a good idea. Interfaces should be explicitly designed. This is because a properly designed interface is both composable and reuseable. An implicitly designed interface will likely not be either. And to take your words,

Implicitly designs do not help promote good designs, it helps promote convenient designs that will likely not be applicable in a more general context. It will likely even help bad designs, because it is easy to just latch onto some other type's interface, even if it isn't a good fit, or to shoehorn your way into another crate's interface.

Here is another way we seem to disagree. I think traits shouldn't be incidental. They should be properly designed, and for that to happen, they can't just fall out of another type's public interface.

Being convenient is not always a good thing, see C++ for details.

If a library doesn't allow a more general interface, then it wasn't designed with that in mind. This means that forcing it to be more general won't work out. At best, it will do something unexpected, at worst it can cause UB because a crate was using unsafe without properly considering other types.

Your proposal doesn't seem to trigger my last two points, but that's because it also would require everyone to use generics everywhere for your proposal to be useful. For example,

If I have an existing crate,

fn foo(h: HashMap<u32, f32>) -> f32 { ... }

This can't be used with your proposal, it would have to be

fn foo<H: HashMap<u32, f32>>(h: H) -> f32 { ... }

Which doesn't look like an improvement to me. It's just line noise for the hope of code reuse. There is a reason why almost entirely trait based, extremely generic apis are not common. They are really hard to understand without decent documentation. Making that documentation is also really hard because there is just so much to explain. From the contracts of the interfaces, to how those interfaces interact, there is just so much ground to cover.


edit: to mods, can we please move this topic about using the public interfaces of types as traits to it's own thread. It looks off topic here.

7 Likes

No, you seem to have misunderstood. Why would the first signature not be accepted under that RFC? The type is still there and can be used as such. It's just that it has an "associated trait" Hashmap<u32, f32>::Trait.

In general, I think you are probably closer to hitting a note for Rust with what you are saying, because this RFC probably entails a bit too much which Rust just wasn't designed for and doesn't want to. I use generics wherever I can, and it never disappointed. It's only that very few crates are generic. I will probably have to keep wrapping external types and convert Into<ExternalType> until a language with focus on generics and performance and safety is developed.

What I meant was that you can't make some other type

struct SpecialMap { ... }

impl HashMap<u32, f32> for SpecialMap { ... }

Then pass this new type to my first function,

fn foo(h: HashMap<u32, f32>) -> f32 { ... }

let spec_map  : SpecialMap = ...;

foo(spec_map);

If you want this code to compile, I'm afraid that will never happen. This is because it would break all unsafe code out there today. For example, let's say I have a type,

use std::num::NonZeroI32;

fn bar(x: NonZeroI32) { ... }

In bar I am allowed to assume that x is non-zero. But if I pass in a type that implements the same interface as NonZeroI32 (let's call it Fake), but returns 0 every time I get the value (which wouldn't require writing any unsafe), then I would be able to use Fake to induce unsoundness in bar. This is disastrous for Rust in the worst possible way. If that is the proposal, it will never gain traction.


As much as I love generic code, being too generic tends to unreadable documentation. This is really off putting when trying to use another crate. For example, see serde, a brilliant trait-only crate. It developed an interface for serialization so that any type that derives a trait can be used with any serializer that implements another trait. But without it's amazing book to go along with it, it would be useless. Even the documentation in the crate docs is insufficient (not blaming anyone, it's genuinely hard to write good documentation for such a general feature). The serde book does an amazing job of explaining serde, so in part the crate docs don't have to. They can just be a reference. But not everyone is capable of writing such a book, or even some decent documentation. And maintaining such a general crate is also hard.

For another example of an extremely general crate, see frunk. It gives you heterogeneous lists, or HList for short. HLists can store different types of elements in the same list. On top of this, they build ad hoc named tuples. Along with HLists they also have generalized co products (CoProd). CoProd are to enums what HLists are to structs. CoProd can represent any number of variants. On top of this, they built ad-hoc enums (anonymous enums, like how tuples are anonymous structs). Again, frunk is a brilliant crate, but it's documentation is really hard to parse, in large part because it is so general. There is an amazing effort to properly document it, but they don't have a frunk book.

Just this lack of a book, and some parseable documentation leads to a staggering difference in usage. Where serde has 15M downloads, and hundreds of dependent crates. Frunk has less than 1k, and only a handful of dependent crates. This difference can also be attributed to the fact that serialization is a common task, but given how often anonymous enums have come up in the past, I would have expected frunk to surpass 10k, on that alone. But given that it is hard to use and hard to learn, why bother?

This is the difference between a well designed interface and the rest. serde has amazing documentation and far more time put into reworking the design to perfection. For frunk, it was more of a research experiment, and so there isn't as much time put into making it usable. (nothing wrong with that, just different motives).

8 Likes
fn foo(h: HashMap<u32, f32>) -> f32 {…}

let spec_map : SpecialMap = …;

foo(spec_map);

This isn't what I meant exactly.
What I meant was: When you want to implement the trait for Hashmap<u32, f32> with

impl Hashmap<u32, f32>::Trait for SpecialMap {…}

for example, then SpecialMap would just be another type that implements a trait (just like HashMap itself) and you would either have to use functions specifically for SpecialMap or with type arguments with a trait bound for the HashMap::Trait:

struct SpecialMap;

impl Hashmap<u32, f32>::Trait for SpecialMap {…}

fn hsh(h: HashMap<u32, f32>) -> f32 {…}

fn spec(s: SpecialMap) -> f32 {…}

fn gen<H: HashMap<u32, f32>::Trait>(m: H) -> f32 {…}

let smap : SpecialMap = …;
let hmap : HashMap<u32, f32> = …;

spec(smap); // SpecMap == SpecMap
hsh(hmap); // HashMap<u32, f32> == HashMap<u32, f32>

gen(smap); // SpecMap <= HashMap<u32, f32>::Trait 
gen(hmap); // HashMap<u32, f32> <= HashMap<u32, f32>::Trait

So it is really just like implementing a separate trait for all public member functions explicitly.

So this would change to

fn bar<I: NonZeroI32::Trait>(x: I) {…}

or

fn bar(x: Fake) {…}

And in the generic case it is already today possible that the trait implementation does not adhere to the semantic requirements of the trait, and unsound implementations are probably pretty common. This is something that would require a lot more requirements to be formalized, probably in form of test cases, to ensure the requirements are met before returning a value with confidence. I don't think you could do this today. You could if traits could define private functions, because then you could have a get function which calls a trait function which is to be overridden by an external implementation, and then get performs the validity checks (x > 0) on the value it got from the overridden function. But I don't think you can do that with traits, because they only provide public interfaces. I notice that I am increasingly thinking in terms of classic class inheritance, where you can basically make any property of a class abstract, which I think sounds very compelling. The question is how much bad design does that really risk and how much would it pay off in usability. Personally, I would like to be able to use generics like this. When the interfaces are atomic enough, then type signatures will more and more speak for themselves. For example

trait HttpsClient: Client::<Connector=HttpsConnector> {…}

fn connect<C: HttpsClient, A: ToSocketAddr>(c: C, addr: A);

Basically one could code only using traits and compose new traits from other traits. But isn't this basically just multiple inheritance? The ramifications would differ if we include "data traits" or not, but personally I think the abstraction utility would be worth the additional complexity. After all, it models the real world perfectly. We have objects, which are just data structures, and they are all in the set of "objects" and then we have subsets of objects and we can define various classes of objects, which get increasingly more specific and define more and more behaviors and properties of "objects", i.e. the subsets get increasingly narrowed down. Objects are basically just coherent strings of bytes in memory and the type classes define how they should be interpreted and what interfaces they may be used with. At least this is how I have come to think about programming, and I found this implemented in Java. However Java is garbage collected and in general has a terrible memory model, imho, so I was hoping there were people here who share the same views. However it seems to be the trend that "object oriented Programming" (which doesn't describe much to be honest) is too mainstream and responsible for a lot of problems in the programming world.

Part of the problem also being that community behind Rust is yet to narrow down on the target market. While OO is still used extensively, the sort of old school OO (GoF patterns, heavy reliance on inheritance etc.) is increasingly limited to enterprise-y business applications. Thanks, to likes of Javascript, Python, Ruby and friends, more people are fairly comfortable in hybrid environments. What is needed is "thinking in traits" instead of "thinking in objects and state" which is so heavily drilled into many of us via our university curriculum.

Rust cannot be everything for everyone yet. May be it is better to identify a target market (which could be systems engineering, web-servers, game-engines or anything else) and focus a couple of years making the language healthy for the target market and see how things shape up.

In any case, Rust still needs it's killer platform (web-framework/game-engine etc.) which provides compelling reasons for developers to adopt it or a big-corp driving the adoption.

3 Likes

On the contrary. From the Rust home page:

A language empowering everyone to build reliable and efficient software.

There it is. The target market for Rust is those that value the quality of their code. Specifically with regard simplicity, hence "empowering everyone", correctness and performance.

If that is not the "killer platform" then we might as all give up and go home.