Better way to read and understand generics in rust

Hello.

Please advise, how to read and understand code containing nested types in <> and generics in general?

Exmpl:
SomeType<SomeOtherType<SomeSecondType>>

Maybe there are some rules or tips that will help to understand this?

Have you read this chapter in "the book"?
https://doc.rust-lang.org/book/ch10-00-generics.html

2 Likes

Ten years ago I read a C++ book with a detailed explanation for such nesting. Generally you read it from inner to outer. Please give a concrete example, then someone should be able to tell in words how to read and understand it.

Generics are type parameters. One could draw a parallel to function arguments: A function argument is variable in its value, a generic parameter is variable in its type.

Some of the standard library types are great for really convincing yourself that this is true: Option<T> is a type whose generic parameter T can be any type:

Option<i32>
Option<MyStruct>
Option<HashMap<String, Vec<(Cow<'a, str>, usize, Vec<u8>)>>>

Ok, that last one was thrown in for a laugh, but it also shows that arbitrary type nesting is mostly possible [1], too. Where the concrete type selected for T can have its own type parameters.

Option<T> is also nice for illustration because we don't have to introduce trait bounds to see how generics are useful. But trait bounds are also useful, and they come in when you want to limit the types that are allowed to be used. Staying with Option<T>, we can look at the signature for its map method:

impl<T> Option<T> {
    pub fn map<U, F>(self, f: F) -> Option<U>
    where
        F: FnOnce(T) -> U,
}

self refers to Option<T>, and the method returns a different type, Option<U>. The connection between the input and output types is the closure passed in: F: FnOnce(T) -> U. This closure takes a T and returns a U. The author of this code isn't specifying any constraints on T or U, which is very useful for the Option type, indeed!

But direct your attention to the closure that is passed to map(). It is a generic whose type parameter is named F, and it is constrained to be "any type that implements FnOnce(T) -> U". I don't think the example warrants further investigation into what that means precisely. The reason I bring it up is because you are not allowed to pass just any old type for F. An i32 doesn't implement the constraint, for instance, making Some(()).map(42_i32) a compile-time error.

By constraining types with traits, you limit those allowable. But it also gives your code a way to access methods implemented by those traits. The closure in question has a call_once method defined, for instance, which is what map actually calls!

Without the FnOnce constraint, the compiler would not know about the existence of the call_once method on F. Just as when given a function argument i32, you don't know whether its value will be 42 or 13 except at the call site or possibly not even until runtime, the concrete type for F is also not known until its use by a caller. (Which may be in a different crate compiled on a different machine from your own.)

It all clicked for me when I was designing an API for software rasterization modeled after OpenGL. In fact, this was the very first topic I opened on the forum with a question: Seeking API design ideas for my software 3D renderer. (Wow, has it really been that long? [2]) Not shown in that thread is the type shenanigans I had to do to link the various vertex attribute buffers, uniform buffers, and shaders together. But the linked blog post does have all of those details covered. Here's a small taste:

// The shader program contains structs that implement the various traits
pub struct ShaderProgram<F, U, V>
where
    F: Varying + FragmentShader<Uniform=U>,
    V: VertexShader<Uniform=U, Fragment=F>,
{
    pub attributes: Vec<V::Attribute>,
    pub uniform: U,
    pub vertex: V,
}

Note that usually you don't want to constrain generic parameters in the type itself. In this case the struct is only allowed to contain types that implement my shader traits with a shared uniform buffer type. This makes sense because any ShaderProgram that does not have these constraints is unusable. E.g., if the uniform buffer type differs between the vertex and fragment shaders, then they just cannot share any information, making the raster() method uncallable. Notably, U is not constrained, because using "anything" for the uniform buffer is semantically valid and a trait is unnecessary!

As I said, the whole topic of generic parameters and trait constraints really came together for me while I was initially writing this code. And I believe it's because I put myself into a position where I had to understand how to connect these things. This experience with a rather complex use of the type system left me fairly comfortable to explore it further. I have since gone on to writing crazy things like a library that does compile-time syntax validation of SQL in the type system, making SQL statements type-safe [3]!

This is my best advice for reading and understanding generics. Start small with (the simple) standard library types. Read articles and code. Write your own code using generics and trait bounds. And eventually you will be able to understand all the wild type systems things done in libraries like plotters and nalgebra. Both are very intimidating when you dig in.


  1. There is a depth limit which makes recursive types impossible to define (because we don't have infinite memory) and some very large types can run afoul of the conservative default depth limit. Such as Bug - Unable to build release Unique<SampleOutputChild> error · Issue #49 · yoshuawuyts/html · GitHub ↩︎

  2. Sidebar: It has been so long that in the linked article, I still believed that the web browser was the only viable platform for desktop/mobile GUI apps! Thank goodness egui and iced have rescued us from that dark fate... ↩︎

  3. The library is not yet (but will one day be) open source. There are only a few crates that do something similar, and most of them either depart from familiar SQL syntax (like sql-builder) or are tied directly to one particular SQL implementation (like scooby to Postgres). Of the ones I know of, the xql and joker-query crates come closest to what I wrote. I'm curious why this idea isn't more popular. Most SQL client libraries prefer users to write stringly-typed SQL that gets parsed by an ORM, as if that somehow meaningful improves anything. I was heavily inspired by jOOQ, particularly: Why You Should Use jOOQ With Code Generation – Java, SQL and jOOQ. | The Java Fluent API Designer Crash Course – Java, SQL and jOOQ. | “What Java ORM do You Prefer, and Why?” – SQL of Course! – Java, SQL and jOOQ. | ORMs Should Update “Changed” Values, Not Just “Modified” Ones – Java, SQL and jOOQ. ↩︎

12 Likes

@steffahn

Thanks for reminding me about "machine generated" content.

Actually, I considered removing the GTP-4 explanation myself after discovered the detailed reply of parasyte.

I don't know if they have created it just for this thread. For me being still a Rust beginner, it is too detailed. The one from the C++ book 10 years ago was OK for my knowledge level that time, but it was a borrowed book, I can not even remember the title or author. But I think C++ generics (templates) are more complicated, and I never used C++ for real work.

For such detailed and advanced explanations as a response to "three word questions" I have my own personal view. But I am sure they had good intentions, and a very detailed and friendly reply is always better than no reply or an unfriendly one. This forum is actually very friendly and helpful -- still I am hoping that the original author will come back and tell us more about his actual problem and if the above detailed explanation helped him.

2 Likes

Nothing complicated here; you can think of it this way:

type A = SomeSecondType;
type B = SomeOtherType<A>;
type C = SomeType<B>;

I don’t quite agree. There isn’t a strict order for reading, using, or understanding nested types. It’s like a box within a box within a box—you can view it from the outside in or from the inside out. Both perspectives are valid. In some situations, you might be more interested in the innermost type, while in others, the outer type may be more relevant. And sometimes, a middle type is what really matters.

Example:

If you're using a Vec<T> you're more focused on the perspective from the outside in. You put elements into the vector, delete them etc. The concrete type of the elements are not so important.

But if you're using an Option<T>, you're more focused on the value inside that option. The case there is None is often more of an outlier.

But of course, these are only approximations. It heavily depends on the surrounding environment, which perspective matters more.

3 Likes

If you are referring to my reply, yes I wrote it specifically to help answer the OP's question. It took me approximately 3 hours to put it together. Several rewrites trying to find the best ways to express what I wanted, trying to find old references again for inclusion as links, etc.

I'm sorry if it was too detailed. Better to include too much information than not enough, no? Perhaps you can revisit this thread sometime in the future (if you still remember it) when you are more experienced and let me know personally if it is still too detailed.

5 Likes

That is even much more than I expected. You might have your reasons why you spent so much time on it. But my generally experience collected over a few decades is, that detailed explanations for very vague and short questions make not much sense. Until the thread starter did not come back and tell us, we do not know his concrete problem, and we do not know how much all your work can help them. When you are hoping others might find your post useful later: Well, the content is something that is already discussed in similar form in many books and blog posts, including the official book -- in my book "Rust for C-Programmers" that I started 8 weeks ago too.

Better to include too much information than not enough, no?

No, I don't think so. Too much information can just overwhelm people. It took me some time to learn that myself. And it has become more true now where AI can answer most of our questions about COMMON knowledge extremely well and customized to our actual knowledge state and interests.

And sorry for my "For me being still a Rust beginner, it is too detailed.". I was not implying that I could not understand your post -- I started with Rust one year ago, read the official book, and the book "Programming Rust" by Jim Blandy, and some more. And I ported my chess engine from Nim to Rust, and created a EGUI and a Bevy 3D GUI for it. So not a real beginner -- but Rust is a complex language, and compared to some of the brights experts here in the forum, I still feel very stupid.

The question was very general, and perhaps it should be more specific, but sometimes it is actually a general question. We can ask for more details, link to materials, or choose to write a very general response. We have all of those in this thread. @parasyte's response obviously took a lot of effort and was based on their experience with the topic, as well as with answering questions here.

1 Like

this is the difficulty, because it is not clear from the context of nesting what the emphasis is on. To the type inside the next nesting, or is it a high-level one (the starting point from which the first one comes <>)

yes, I started with this book and read it, but the problem is that it contains very "general" examples and does not contain examples with a lot of nesting inside <>

"rust" is the first language where I see generics without which it is almost impossible to write anything and where they are tightly integrated in the form in which they are, thx for reply. For example: Pool<ConnectionManager<PgConnection>>. As a top-level user, it is not very clear to me what the emphasis is on in this example

When I first see a library and want to understand its structure and come across a lot of such implementation blocks, it causes me wild discomfort in realizing how it should work and what the idea is. And it seems to me that the problem is with:

  1. Nesting
  2. A large number of types and blocks <> with a restriction

This code:

pub struct PoolTransactionManager<T>(std::marker::PhantomData<T>)

impl<M, T> TransactionManager<PooledConnection<M>> for PoolTransactionManager<T>
where
    M: ManageConnection,
    M::Connection: Connection<TransactionManager = T> + R2D2Connection,
    T: TransactionManager<M::Connection>,

Thanks for the detailed answer, it might be inappropriate, but you didn't waste a few hours compiling it and it really clarified a little better how it should work in place.

I'm still at the very beginning and I'm trying to subtract the internal implementation in an already third-party library and it looks like I haven't had a situation where "I was forced" to overcome a certain problem related to how to link several pieces of my code.

No matter how corny it may sound, but apparently it will come with time and writing a large amount of code

As people in this topic have noticed, my question is too general. I expected to hear an answer like "a general question, but these and such few things helped me when I started with generics." I mean, I was expecting something like "lifehacks" or something similar to this.

In particular, I focus on the contents of the nested <> - to understand what the emphasis is on here from these 3 levels of nesting. Exmlp:

Pool<ConnectionManager<PgConnection>>
or
PooledConnection<ConnectionManager<PgConnection>>

Yes, indeed, the question was meant as a general one. After reading the book and meeting with the world of real libraries, I see that some "basic" things cause me difficulties with reading and understanding. For example, from std, a primitive trate "Into" and its implementation took me 10-15 minutes to understand how it works

impl<T, U> Into<U> for T
where
    U: From<T>,
{

    fn into(self) -> U {
        U::from(self)
    }
}

and the questions were "where did the second type come from?" if the "into" method is called on a specific one type, "which generic self belongs to?" and all that sort of thing. Although it is not difficult and basic at all, and when it comes to "managed" implementations inside third-party libraries, then all this together causes a lot of time to read and try to put everything together one bunch, so I wondered how they could be read and understood a little better and expected something like "life hacks"

15 years ago, when I created Ruby bindings to C++ CGAL and Boost library to have a constrained delaunay triangualtion available, I had a lot of trouble to understand the C++ templates. That time I did not know much about C++. What helped me most were a few tiny concrete example codes. I think I found a few with Google, or perhaps I asked the experts on mailing lists or newsgroup. I don't know if generics in Rust can be that difficult. But today I would also look for tiny concrete examples, and of course ask AI tools to explain and provide code examples.

Sorry to break it to you, but there is no natural emphasis to place on any of these types. I agree with @mroth's comment that it depends on surrounding context. You're dealing with Postgres specifically, which implies a metric ton of incidental complexity in the form of networking I/O, let alone the massive ball of complexity that is the database itself.

I don't know how useful it is to try focusing of the syntactic part of these examples. It may as well be called PoolConnectionManagerPgConnection without any brackets at all, and it could still be considered "the same type". In the sense that it would be a connection pool for Postgres.

The nesting is more of an implementation detail because it turns out that connection pools are broadly more useful than "just for Postgres". And maybe there's a good technical reason for using a different kind of connection manager with the pool. Its purpose is code reuse.

My "lifehack suggestion" is take the definition for abstraction to heart: Dismiss details that do not matter. Knowing which details don't matter is the hard part. I lost track long ago of how many technical conversations went in circles and phrases like "abstraction" and "implementation detail" and "ignore that" or "forget about Foo for a moment..." still doesn't help. Maybe most people get caught up on minor details and "miss the forest for the trees"?

So, this isn't much of a lifehack after all. :frowning:

3 Likes