Rental 0.5 Released

After a period of dormancy, I'm pleased to announce that the new version of rental is available! This work was largely inspired by all the discussion surrounding the Pin proposal. While I didn't end up using Pin, it really made me think about what capabilities I wanted rental to have and how to achieve them best.

At any rate, this version is much less of a departure from 0.4 than that was from 0.3. The API is mostly the same with a few method renames and attribute adjustments, so updating code should be relatively straightforward. Under the hood there has been substantial rework to upgrade to syn 0.13 in preparation for the new macros infrastructure. This should hopefully greatly improve the error messages once all the parts are in place, but that's still to come.

What really makes this version stand out though are 2 new features whose absence was always a point of frustration for me. In the past, it was only possible to access fields via awkward closures, which was particularly irritating if you wanted to borrow something out of the rental struct. While I can't eliminate that for all cases, there are a huge number of cases where we can do better. The first major new feature is full support for covariant structs.

If the types in your rental struct are covariant over their lifetime parameters, which most will be, then this will allow you to directly borrow the field out of the struct, and the lifetimes will be reborrowed to match the &self param. This opens the gate pretty far and substantially improves ergonomics for many uses. As an example:

#[macro_use]
extern crate rental;

rental! {
	pub mod covariant {
		use std::collections::HashMap;

		#[rental(covariant)]
		pub struct RentHashMap {
			head: String,
                        // Store a bunch of slices of our string, or whatever
			map: HashMap<usize, &'head str>,
		}
	}
}

fn test_covariant() {
	use std::collections::HashMap;

        // Create our rental struct. I don't bother to put anything in the hash map
        // for this example, but you can add whatever you want.
	let rent_hash = covariant::RentHashMap::new("Hello, World!".to_string(), |_| HashMap::new());

        // _Directly_ borrow the hashmap out of the rental struct. No closures!
	let hash_ref = rent_hash.suffix();

        // Do whatever with our borrow
	println!("{}", hash_ref.contains_key(&12));
}

With even just the little experimentation I've done with this so far, it feels SO much better to be able to do this now. Not to oversell it though, you can NOT mutably borrow fields directly under any circumstances, as that is fundamentally unsound because of lifetime invariance for mutable refs. So to update our hashmap we'd need to use a closure, but after it's updated we can directly borrow it again as we please.

Next up, one feature that some smart reference types such as Ref have is the ability to map its contents to another type, as long as it's bounded by the same lifetime. Rental did not support the ability to do this.. until now!

#[macro_use]
extern crate rental;

pub struct Foo {
	i: i32,
}

rental! {
	mod rentals {
		use super::*;

		#[rental(map_suffix = "T")]
		pub struct SimpleRef<T: 'static> {
			foo: Box<Foo>,
			fr: &'foo T,
		}
	}
}

fn map() {
	let foo = Foo { i: 5 };
	let sr = rentals::SimpleRef::new(Box::new(foo), |foo| foo);

        // Remap our rental struct to a direct reference to the i32 inside
	let sr = sr.map(|fr| &fr.i);

	assert_eq!(sr.rent(|ir| **ir), 5);
}

Here we add the map_suffix = "T" option, which marks T as the mappable type param. We can now call map on our rental struct to convert the reference from one type into another. This should eliminate the need to create needless subrentals just to change the type of your suffix field.

With these new capabilities in place, it is now possible to create a struct that is essentially equivalent to an OwningRef using rental. In fact, the new common module contains a collection of premade types with such functionality. The main capability owning-ref still has that rental does not is the ability to erase the type of the head. This would be possible to implement in rental, but I'm not entirely sold on it. Type erasure wasn't ever really a goal of rental, so I feel like that's a little out of scope, but if there is demand for it I'm open to reconsidering.

That about covers the highlights. I hope people out there find this useful. I think this is about the farthest library support for self-referential types can go in rust without explicit language support, so we'll see what the future brings.

11 Likes

Hi, @jpernst--thank you and congratulations on the new release!

I'm curious to learn more about your explorations with Pin--why did you decide to not use it in Rental? Also, under what conditions do you feel it is a viable solution for creating self-referential structs vs using Rental?

I would love to hear about this or any other related thoughts you have had on this topic.

1 Like

Good question, I suppose I should have elaborated on that a bit.

To be clear, it's not that Pin wouldn't have worked with the concept of rental at all, but rather that it just didn't allow the features I wanted or bring any real advantages over how it works now. At its core, Pin is for ensuring a type will never again move, which on its face is what we want for self-referential structs. The devil is in the details, however.

First, Pin works best when the type you create has a period of non-self-referentiality before you pin it. Futures have this property, in that when they are first created, there is no internal self reference until you actually poll it. This gives you ample opportunity to move it around and place it wherever you want before pinning it. With rental, there is no such grace period, and the struct must be immediately pinned on its creation. This is possible to accommodate by just immediately boxing the struct when you create it, but that's a bit limiting, in that you can no longer choose which container type you want. Furthermore, if your head field is already boxed, then you end up double-boxing it, which is unnecessary. With rental currently, if your head is already a StableDeref container, then there is no overhead at all in creating a rental struct.

Similarly, with Pin, the entire struct must be within the pinned box. This would include the suffix field, which means accessing the suffix field incurs an extra deref, where that is not necessary with rental as it is now, since the suffix field is stored directly in the rental struct with no indirection. Furthermore, having the suffix inside of a box would have prevented the new mapping API from working, since you'd need a different size box for the new suffix type, and you can't move the prefix fields out of the box they're already in. With rental's current design, each field must satisfy StableDeref independently, so they can be freely moved to create a new rental struct with the new suffix type.

Now, a few advantages to Pin would be the removal of the StableDeref requirement, which would greatly simplify the code. Second, it would save allocations if you have a rental struct with several members, since they can all live within one box instead of each needing their own. However, rental structs frequently have only 2 fields, or at most 3 or 4, so those gains are relatively minor. In the end I judged the utility of the APIs that the current design can provide to outweigh the benefits Pin would bring to the table.

After much exploration and waffling on this topic, I'm pretty assured at this point that Pin has relatively little utility for generalized self-ref structs, and serves mostly as an elegant stopgap to get us self-borrowing futures as quickly and cleanly as possible. That said, there has been exploration of other potential uses for Pin including intrusive collections, which is really interesting. Now, the story might change if rust gains additional enabling features such as generative existential lifetimes or some other such feature that provides the necessary tools for generalized self-referential structs. Then I'd have to re-examine the interplay of those features and draw a new conclusion.

That's the gist of my thoughts on the topic; let me know if there's anything I missed or any followup questions.

9 Likes

Fantastic answer, my friend-- you've answered everything I was wondering and I learned a few things as well.

Thanks for taking the time!

All the best,
U007D