How do I create self-referential structures with Pin

help
1

I want to create a simple struct (let's say, AppState) which owns a connection and shares it between its own fields. I tried to express it with Pin, but it didn't work well.

#![feature(pin, arbitrary_self_types)]
use std::pin::Pin;

pub struct Connection;

pub struct UserRepository<'a>(&'a Connection);
pub struct BlogRepository<'a>(&'a Connection);

pub struct AppState<'a> {
    connection: Connection,
    user_repo: UserRepository<'a>,
    blog_repo: BlogRepository<'a>
}

impl<'a> AppState<'a> {
    pub fn new(connection: Connection) -> Self {
        Self {
            connection,
            user_repo: UserRepository(&connection),
            blog_repo: BlogRepository(&connection),
        }
    }
}

There are two problems:

  1. I cannot reference connection, because I move it to Self in constructor, but OTOH I cannot reference partially initialized variable
  2. I don't understand how to use Pin api to allow it. Basically, AppState should be immovable after creation so we can use self references. But I don't get how to express it.

Please, any advice?

P.S. Don't suggest reference counting :slight_smile:

2 Likes
2

AFAIK even with Pin, you cannot straightly create struct with borrows to itself. You may safely create self referential struct, but instead of using borrows, you need to use some pointers, and a little of unsafe boilerplate (the second part may be avoided, I didn't play with it recently so maybe there are some new upgrades in API). Hint how to achieve such things may be found here: Another look at the pinning API. Article is pretty old, but AFAIK pretty actual (regarding some Api changes).

3

Sad to hear it. I'd like to have it.

The main problem here is that 'a is a type parameter, when it should be more like:

pub struct AppState {
    connection: Connection,
    user_repo: UserRepository<'self>,
    blog_repo: BlogRepository<'self>
}
4

The problem is not inability for describe lifetime for user_repo/blog_repo, because it may be easly handled in way you proposed. Problem is, that when you borrow structure (self in this case), you cannot move it. In this particular example, let say, that there is some 'self lifetime which describes lifetime of entire struct. Then, lets try construct it with new:

fn new(connection: Connection) -> Self {
  AppState {
    connection,
    user_repo: UserRepository<'self>(&connection), // Borrow moved item
    blog_repo: UserRepository<'self>(&connection), // Borrow moved item
  }
}

So this obviously doesn't work, let try another approach:

fn new(connection: Connection) -> Self {
  let user_repo = UserRepository<'self>(&connection); // 'self outlives &connection
  let blog_repo = UserRepository<'self>(&connection); // 'self outlives &connection

  AppState {
    connection, // Moving borrowed item
    user_repo,
    blog_repo,
  }
}

Also don't work. Only possibility is to create the AppState with uninitialized user_repo/blog_repo, and when it is done, assign those fields to have some references to connection. Problem is it is impossible with references (maybe some playing with optional references? but here lifetimes would also be a problem), but possible with pointers (NotNull<T> is just a fancy pointer to T), so this way its possible. Unfortunetely doing anything reasonable with pointers requires unsafe (but as long, as struct may be created only by tools you deliver, it is safe, as long as you may guarantee that the pointer is always set to some actual object).

Also another problem is, that self-referenced struct cannot be simply moved, so in this form you cannot return Self - because move would change location of connection and things will break internally. Thats why Pin trait exists - by ensuring your type is Pin, you don't allow moving it, and you need to use it via pointers like Pin<Box<T>>, to it is guaranted to never change location in memory.

5

Yep, I understand these conserns. The questions is how it could be done, or maybe there could be created some RFC addressing these problems.

I just want to have some references to myself. I aggree to not move AppState since it was created. How do I do it? No unsafe, no reference counting, just simple reference. It seems that the answer "you can't and you won't be able in the future", beucase I don't want to use mem::unitialized (since it's UB) or Option (since I always create a value and want to express this fact in types). Standard library is fine with pointers, but I strongly dislike it.

But I'd really want to. It's not something fundamental, it's just inavility to express some things. I think about RFC addressing it, but i'm not sure about its details.

Also another problem is, that self-referenced struct cannot be simply moved, so in this form you cannot return Self - because move would change location of connection and things will break internally. Thats why Pin trait exists - by ensuring your type is Pin , you don’t allow moving it, and you need to use it via pointers like Pin<Box<T>> , to it is guaranted to never change location in memory.

That's interesting, I didn't think about it. Then it should be fn new(connection: Connection) -> Pin<Box<Self>>, I guess.

6

Actually its not sure you will not be able to do it in future. I believe you will be. There are some ideas about partial initialization, for eg: Pre-RFC: Partial Initialization and Write Pointers - language design - Rust Internals. In general, maybe in future it will be possible to create some type which is "partialy initialized", but for example cannot be moved until its provable its fully initialized. In this case, you will be able to partialy initialize your struct with connection only, and only when it is already in memory, you will assign proper user_repo/blog_repo. When (if?) this will be possible, only problem will be how to express that "this references livetime is whole struct lifetime", in general - your 'self lifetime, which I belive will be solved somehow.

However Pin types is pretty new thing (not even in stable I think), so give it time. First I want people to fully understand its concept, libraries to properly handle Pin/Unpin types, and then I hope that it will evolve. Don't make Rust to become C++ by implementing half-baked features just because they looks cool, and then find it unextendable at all.

2 Likes
7

Yeah, sounds promising

I couldn't be farther from this idea :slight_smile:

8

More I am thinking about this, it becomes easier. If instead of keeping connection in your struct, you may keep Pin<Box<Connection>>, so its immovable by definition. You may also take reference to such field, and still move whole Pin (which will preserve Connection in same place in memory). Now you may be able to create this object like:

fn new(connection: Connection) -> Self {
  let connection = Pin::new(Box::new(connection));
  let user_repo = UserRepository(&connection);
  let blog_repo = UserRepository(&connection);

  Self {
    connection,
    user_repo,
    blog_repo,
  }
}

This will obviously not work for now, but only because of lifetime of references which cannot be infered (and even named), but I am sure, this is a minor problem.

1 Like
9

Hmm, what about an approach more like this:

pub struct AppState {
    connection: Box<Connection>,
    user_repo: UserRepository,
    blog_repo: BlogRepository
}

Now, if we go from the point of view of user_repo the connection field will live longer than or equal to user_repo, meaning that we could say that they have the same lifetime. In that case, wouldn't changing the definitions and constructions like this be safe? Example:

pub struct Connection;

pub struct UserRepository (&'static Connection);
pub struct BlogRepository (&'static Connection);

pub struct AppState {
    connection: Box<Connection>,
    user_repo: UserRepository,
    blog_repo: BlogRepository
}

impl  AppState {
    pub fn new(connection: Connection) -> Self {
        let boxed = Box::new(connection);
        let (ur, br) = unsafe {
            let cptr: &'static Connection = std::mem::transmute(&*boxed);
            (UserRepository(cptr), BlogRepository(cptr))
        };
        Self {
            connection: boxed,
            user_repo: ur,
            blog_repo: br,
        }
    }
}

Which I believe that in this case is a safe use for unsafe code.

10

You just write a memory leak :slight_smile: And you don't have to write it manually since you already have Box::leak. And I don't want to heap-allocate Connection, I want to have a self-reference as mentioned in title. Of course you can leak a static lifetime object and reference it from anywhere

impl  AppState {
    pub fn new(connection: Connection) -> Self {
        let connection: &'static Connection = Box::leak(Box::new(connection));
        Self {
            connection: connection,
            user_repo: UserRepository(connection),
            blog_repo: BlogRepository(connection),
        }
    }
}

but it's beside the whole point.

1 Like
11

Well, I ended up with following code:

use std::pin::Pin;
use std::ptr::NonNull;

pub struct Connection;
pub struct UserRepository(NonNull<Connection>);
pub struct BlogRepository(NonNull<Connection>);

pub struct AppState {
    connection: Connection,
    user_repo: UserRepository,
    blog_repo: BlogRepository,
}

impl AppState {
    pub fn new(connection: Connection) -> Pin<Box<Self>> {
        let res = Self {
            connection,
            user_repo: UserRepository(NonNull::dangling()),
            blog_repo: BlogRepository(NonNull::dangling()),
        };
        
        let mut boxed = Box::pin(res);
        let ref1 = NonNull::from(&boxed.connection);
        let ref2 = NonNull::from(&boxed.connection);
        let mut_ref: Pin<&mut Self> = Pin::as_mut(&mut boxed);
        Pin::get_mut(mut_ref).user_repo = UserRepository(ref1);
        let mut_ref: Pin<&mut Self> = Pin::as_mut(&mut boxed);
        Pin::get_mut(mut_ref).blog_repo = BlogRepository(ref2);
        
        boxed
    }
}

I don't like I have to use NonNull, but I didn't find any other approach.

12

@Pzixel you need to add a PhantomPinned to your type to make this sound, otherwise someone can just come along and pull it out of the Pin<Box>.

And now I see you do have it in the struct definition, I'm confused how your code is working then, Pin::get_mut should not be callable on a Pin<&mut AppState> since it's !Unpin, you should need to go through Pin::get_unchecked_mut.

Using NonNull and other unsafe code is pretty par for the course with manually implementing self-referential structs with Pin, hopefully there will be higher-level libraries that abstract some of it away eventually, since it's a very new API in std.

4 Likes