Which problems does owning-ref solve?

[tathanhdinh]

Hello all,

I'm reading the code of owning-ref which have been introduced in the forum, but still do not understand the problems it aims to solve.

Immediately from the first example

// Create an array owned by a Box.
let arr = Box::new([1, 2, 3, 4]) as Box<[i32]>;

// Transfer into a BoxRef.
let arr: BoxRef<[i32]> = BoxRef::new(arr);
assert_eq!(&*arr, &[1, 2, 3, 4]);

// We can slice the array without losing ownership or changing type.
let arr: BoxRef<[i32]> = arr.map(|arr| &arr[1..3]);
assert_eq!(&*arr, &[2, 3]);

// Also works for Arc, Rc, String and Vec!

I'm struggling with the phrase

We can slice the array without losing ownership or changing type.

which seems not intuitive to me, in which case slicing an array needs to take the ownership (of the array)?, does any slice just borrow (from some array, vec, etc.)?

For example, the following stupid code compiles:

fn main() {
	let a: Box<[i32; 3]> = Box::new([1, 2, 3]);
	let b: Box<&[i32]> = Box::new(&(&*a)[..=1]);
	for i in *b {
		println!("{:?}", i);
	}
	for i in &*a {
		println!("{:?}", i);
	}
}

Many thanks for any help.

[Yandros]

If you intend to, for instance, trim all the lines of a stdin and get something owned (so that you can freely move it around functions), while avoiding reallocating strings, then owning-ref's map comes in very handy:

#![deny(bare_trait_objects, elided_lifetimes_in_paths)]

use ::std::{*,
    io::{
        BufRead,
        stdin,
    },
};

use ::owning_ref::{ // 0.4.0
    StringRef,
};

fn trim (s: String) -> StringRef
{
    StringRef::new(s).map(str::trim)
}

fn lines_trimmed (
    input: &'_ mut dyn BufRead,
) -> io::Result< Vec<StringRef> >
{
    input
        .lines()
        .map(|mb_line| mb_line.map(trim))
        .collect()
}

fn main () -> io::Result<()>
{Ok({
    let lines = lines_trimmed(&mut stdin().lock())?;
    println!("Trimmed lines:");
    lines
        .iter()
        .for_each(|line| println!("  - {:?}", &**line as &str));
})}

• Playground

[tathanhdinh]

Thanks a lot for the explication @Yandros,

I may understand it better but I'm still not sure.

I think I've misunderstood the meaning of "reallocation". Actually, for some types (but not all), the backing storage will not be reallocated when moved (e.g. String with backing storage str). So owning_ref allows us to pack an object of such a type together with a reference to the backing storage (but not the reference to the object, because the object itself can be reallocated when being moved though the storage is not).

For example, it packs (String, &str) but not (String, &String). Then the phrase

We can slice the array without losing ownership...

may be understood that we can have many slice of the same owner, for example:

let st = String::from("hello world");

let sr = StringRef::from(st);
println!("own: {}", sr.as_owner());
println!("ref: {:p}, {}", sr.as_ref(), sr.as_ref());
println!("{:?}", sr);

let ssr = sr.map(|s| &s[6..]);
println!("own: {}", ssr.as_owner());
println!("ref: {:p}, {}", ssr.as_ref(), ssr.as_ref());
println!("{:?}", ssr);

which outputs something likes:

own: hello world
ref: 0x55fd953c6a40, hello world
OwningRef { owner: "hello world", reference: "hello world" }
own: hello world
ref: 0x55fd953c6a46, world
OwningRef { owner: "hello world", reference: "world" }

we will see that the owner packed in sr and ssr is always "hello world" while the references are different.

Or modified from your example:

let st = String::from("    hello    ");
let sr = trim(st);
println!("{:?}", sr);

which will output:

OwningRef { owner: "    hello    ", reference: "hello" }

Is my understanding correct?

[Yandros]

Take

let s: Box<str> = "   Hello   ".into();

Now, what does trimming do?

let trimmed = s.trim(); /* str::trim(&*s) */

Now, all is good, except for one thing: you cannot move s while trimmed exists!! The reason for that is that you shouldn't be able to move the characters around /
free the allocated characters while trimmed exists: so trimmed cannot outlive s, and more generally, the contents s points to should not be mutated while trimmed exists.

This is why we call this a borrow: for the time trimmed lives (with NLL, the time trimmed is used), a duration that we can call / denote 'a, that is, the lifetime 'a, s cannot access the contents it points to since they have been borrowed by trimmed.

Now, the problem is, that Rust cannot make the difference between s itself (i.e., the ptr, len pair) and the "contents s points to". So, while it is safe to move the ptr, len pair around, Rust does not know that, and forbids it (example).

The solution most people use then is to copy the trimmed contents elsewhere, so that they are not tied to s anymore:

let trimmed: String = s.trim().to_string(); // or
let trimmed: String = s.trim().to_owned(); // or
let trimmed: Box<str> = s.trim().into();

• (the only difference between a String and a Box<str> is a third field, capacity: usize, which does not really play any role in my example, so I will stick to Box<str> for the sake of simplicity)

• Playground

As you can see, we no longer borrow from s, at the cost of duplicating / copying the "Hello" string elsewhere in the heap: we have reallocated "Hello"

Doing this just to satisfy the borrow checker is quite saddening.

The solution of owning-ref or a custom implementation based on Pin<Box<..>>, is that, since moving s does not move the contents it points to, and by forbidding mutation of the pointee (you will note that there is no DerefMut what so ever for OwningRef<_>), if we manage to ensure that the reference (trimmed) does not outlive s, then all should be fine (Rust can still not know that, so unsafe is required). And this is achieved by stitching the reference and s together in a single struct: OwningRef.

It starts off with a dummy reference to the whole contents (e.g., " Hello " with the trailing spaces), in which case it "wastes" a little bit more stack memory than a simple Box<str>, but then, when using the .map() method, we can manipulate that reference and "shrink" it at will. And the magic is that whenever we want to access / read the contents the reference points to, Rust uses the Deref trait, which has been overloaded to use the shrunk ref instead of the owning variable.

• Playground

[tathanhdinh]

Fantastic answer, many thanks for a very detailed explication. You clarifiy a lot

@Yandros

Rust can still not know that, so unsafe is required

some of my doubt about why owning_ref needs using raw pointer for the reference part, e.g.

pub struct OwningRefMut<O, T: ?Sized> {
    owner: O,
    reference: *mut T,
}