The many mysteries of Option<&(dyn AsRef<Path>)>

[carlocaione]

Let's start from:

pub fn read_dir<P: AsRef<Path>>(path: P) -> Result<ReadDir>

I can use this function with objects like:

let pb = PathBuf::from("/foo/bar");
let p = Path::new("/foo/bar");
let s = String::from("/");

Let's now say that I want to put both the variables into an Option<> before unwrapping and calling read_dir(), so one possible solution is to use something like:

let mut a: Option<&(dyn AsRef<Path>)>

The rationale here is that I want an option carrying a reference to different objects implementing the AsRef<Path> trait. I don't want my Option<> to carry the whole object, just its reference for the sake of optimization.

Give that, I can do something like (example nonsense code):

let mut op: Option<&(dyn AsRef<Path>)> = None;

let pb = PathBuf::from("/foo/bar");
let p = Path::new("/foo/bar");
let s = String::from("/");

op = Some(&pb);
op = Some(&p);
op = Some(&s);

if let Some(b) = op {
    println!("{:?}", b.as_ref());
}
....

This code compiles fine and I'm here particularly interested in op = Some(&p) that should be an Option<&Path>. But Option<&Path> is also returned from the parent() function so I wast thinking that I could do something like:

let mut op: Option<&(dyn AsRef<Path>)> = None;

let pb = PathBuf::from("/foo/bar");
op = pb.parent();

Unfortunately this is returning

expected trait object `dyn std::convert::AsRef`, found struct `std::path::Path`

So my first question is: why?

While writing this example code I also realized that I have more problems with this code. For example I tried to do something like:

op = Some(pb.parent().unwrap());

That is returning an even more scary:

error[E0277]: the size for values of type `[u8]` cannot be known at compilation time

At last I tried something like:

op = Some(&pb.parent().unwrap());

But this is returning a:

error[E0716]: temporary value dropped while borrowed

That leaves me a bit puzzled since I cannot guess which object / temporary is being freed since all I see are references to the pb object.

I know that the code could be written differently avoiding the ugly Option<&(dyn ... but it's just a toy code to learn more about trait objects & co.

[alice]

I recommend you use the type Option<&Path> instead. All of the types you mention can be turned into a &Path in some way.

let mut op: Option<&Path> = None;

let pb = PathBuf::from("/foo/bar");
let p = Path::new("/foo/bar");
let s = String::from("/");

op = Some(&pb);
op = Some(&p);
op = Some(Path::new(&s));

playground

[carlocaione]

@alice

I recommend you use the type Option<&Path> instead. All of the types you mention can be turned into a &Path in some way.

I know, thanks for the suggestion but this is toy code that I want to use to learn more about trait objects, dyn, etc... I think that to learn something new sometimes you have to put yourself in a difficult position that force you to think about what you are doing and not just take the easiest way

[alice]

Generally the issue you ran into is that you can't cast unsized things to trait objects, so since Path and [u8] is unsized, the cast will fail. This is the case even when they are behind a reference.

[skysch]

Be aware that &(dyn AsRef<Path>) is a double-indirection as well. You have a pointer to a dyn trait which you have to resolve to get a pointer to a Path. It's not very efficient to design interfaces that require users to wrap pointers in pointers.

[daboross]

The problem with your last line of code is that &Path cannot be converted into &dyn AsRef<Path>, as it is not Sized.

To understand the problem, think about what &dyn AsRef really is: it's one pointer to a thing, and another pointer to a vtable describing how that thing implements AsRef<Path>.

The problem is &Path also stores some extra data. This time, there's one pointer to some data ([u8]), and the second piece of data is the length of that slice.

Both of these use the mechanism labeled a "fat pointer", since it's a pointer to data but with an extra pounter-sized bit of information stuck on. This allows for unsized data, like dyn Trait and [T] to exist behind pointers, storing their determining factors (the vtable and the length, respectively) by the pointer rather than by the data. But each pointer can only store one piece of extra data. If you turn your &Path into an &dyn Trait, where does the information for the length of the path's slice go? It has nowhere to be stored.

This leads to one potential fix: double-referencing it. While you can't turn &Path into &dyn AsRef<Path>, you can turn &&Path into it! This solves the storage problem by having the inner pointer store the length, and the outer pointer just point to that inner one.

[alice]

Yeah. The fact that a &Path needs additional data than what it points at to know how long it is, is important to understand here. You need to put this information somewhere, but you can't put it in the vtable as it must be the same for every &Path, and Rust doesn't have fat-fat-pointers with more extra information than a single pointer/length field.

You could make a Path5 object that knows in the type that the length of the data it points at is five, but then every single length would correspond to a different type.

[carlocaione]

Wow, I'm really glad I asked this question, a world just opened up. I still need to digest a bit of info about this.