Smart pointer which owns its target

Either inference is going to choke on that, or the compiler will have to start making relatively arbitrary decisions in ambiguous situations (perhaps driven by "insider knowledge" of std, making std more magical / not something a Rust programmer could achieve themselves).

Why? Language design philosophy aside, there's no language-level guarantee that these do the same thing:

let os_str    = os_string.borrow(); let path =    os_str.as_ref();
let os_string = os_string.borrow(); let path = os_string.as_ref();

And even if OsString and OsStr became baked into the language (vs. std), so that case is known [1], I could have my own struct with a Borrow<OsStr> implementation that has side-effects, for example.

"Maximal reach" with generics tends to throw inference under the bus, because Rust doesn't like to make arbitrary decisions in ambiguous situations; the more wide-reaching your generics, the more chance for ambiguity. Incidentally, your playground does have one (I think just the one) unambiguous case:

    // works
    bar(CurDir);

Adding any Borrow implementation seems to break this [2], which shows there is room for inference improvement.

Any reference is going to be ambiguous due to the two blanket Borrow impls. [3]


  1. to not matter which implementation is chosen ↩︎

  2. i.e. even if the implementation is for something which is not AsRef<Path> ↩︎

  3. And if you think about it, accepting owned structs when (as per the API) you can only act on borrowed data is somewhat of an anti-pattern (or at least a speed bump for your users), so this knocks out most of the ergonomic utility of the function (as borrows are most frequently references). ↩︎

2 Likes

I was (intentionally) not very precise when I used the phrase "choke on that" because I had not fully thought this through. You are right that there are ambiguous "paths" (sorry for the pun) when going from OsString via .borrow() and .as_ref() to Path.

I totally agree, you are right. After reading your footnote, I would conclude that bar should rather be defined like that:

fn bar(_: &impl AsRef<Path>) {}

But remember #98905! We have no impl<T: ?Sized + ToOwned + AsRef<U>, U: ?Sized> AsRef<U> for Cow<'_, T> like we ought to have! Thus the "corrected" version of bar will fail:

use std::borrow::Cow;
use std::ffi::{OsString, OsStr};
use std::path::{Path, PathBuf};

#[derive(Clone)]
struct CurDir;

impl AsRef<Path> for CurDir {
    fn as_ref(&self) -> &Path {
        ".".as_ref()
    }
}

fn foo(_: impl AsRef<Path>) {}
fn bar(_: &impl AsRef<Path>) {}

fn main() {
    foo(OsString::from("."));
    foo(&OsString::from(".") as &OsString);
    foo(&OsString::from(".") as &OsStr);
    foo(PathBuf::from("."));
    foo(&PathBuf::from(".") as &PathBuf);
    foo(&PathBuf::from(".") as &Path);
    foo(CurDir);
    foo(&CurDir);
    // foo(Cow::<'_, OsString>::Owned(OsString::from(".")));
    // foo(Cow::<'_, OsString>::Borrowed(&OsString::from(".")));
    foo(Cow::<'_, OsStr>::Borrowed(&OsString::from(".") as &OsStr));
    // foo(Cow::<'_, PathBuf>::Owned(PathBuf::from(".")));
    // foo(Cow::<'_, PathBuf>::Borrowed(&PathBuf::from(".")));
    foo(Cow::<'_, Path>::Borrowed(&PathBuf::from(".") as &Path));
    // foo(Cow::<'_, CurDir>::Owned(CurDir));
    // foo(Cow::<'_, CurDir>::Borrowed(&CurDir));
    bar(&OsString::from("."));
    bar(&(&OsString::from(".") as &OsString));
    bar(&(&OsString::from(".") as &OsStr));
    bar(&PathBuf::from("."));
    bar(&&PathBuf::from("."));
    bar(&(&PathBuf::from(".") as &Path));
    bar(&CurDir);
    bar(&&CurDir);
    //bar(&Cow::<'_, OsString>::Owned(OsString::from(".")));
    //bar(&Cow::<'_, OsString>::Borrowed(&OsString::from(".")));
    bar(&Cow::<'_, OsStr>::Borrowed(&OsString::from(".") as &OsStr));
    //bar(&Cow::<'_, PathBuf>::Owned(PathBuf::from(".")));
    //bar(&Cow::<'_, PathBuf>::Borrowed(&PathBuf::from(".")));
    bar(&Cow::<'_, Path>::Borrowed(&PathBuf::from(".") as &Path));
    //bar(&Cow::<'_, CurDir>::Owned(CurDir));
    //bar(&Cow::<'_, CurDir>::Borrowed(&CurDir));
}

(Playground)

:frowning_face:

So …

… that could be fixed by using &impl AsRef<T>.

(edit: The striked out text above was wrong.)

But still …

… will keep us from using the "right" solution.


Example: Using deref_owned::Owned (with the correct implementation of AsRef) doesn't exhibit this problem:

    bar(&Owned(OsString::from(".")));
    bar(&Owned(PathBuf::from(".")));
    bar(&Owned(CurDir));

(Playground)


Actually we can just use the original foo in that case:

fn foo(_: impl AsRef<Path>) {}

fn main() {
    foo(&OsString::from("."));
    foo(&(&OsString::from(".") as &OsString));
    foo(&(&OsString::from(".") as &OsStr));
    foo(&PathBuf::from("."));
    foo(&&PathBuf::from("."));
    foo(&(&PathBuf::from(".") as &Path));
    foo(&CurDir);
    foo(&&CurDir);
    foo(&Owned(OsString::from(".")));
    foo(&Owned(PathBuf::from(".")));
    foo(&Owned(CurDir));
}

(Playground)

Why is that? Because "As lifts over &". From std:

// As lifts over &
#[stable(feature = "rust1", since = "1.0.0")]
#[rustc_const_unstable(feature = "const_convert", issue = "88674")]
impl<T: ?Sized, U: ?Sized> const AsRef<U> for &T
where
    T: ~const AsRef<U>,
{
    #[inline]
    fn as_ref(&self) -> &U {
        <T as AsRef<U>>::as_ref(*self)
    }
}

(source)


Concluding: If #98905 was fixed, then using impl AsRef<Path> is perfectly fine. But that was what I said earlier here:

I mostly agree, or rather,

// The addition of `?Sized` is the important part
fn bar<P: ?Sized + AsRef<Path>>(_: &P) {}

More on that below.

The practical fix is to dereference.


Written before at least some of your comment edits, and mostly just a big long exploration for the sake of a thought experiment -- nothing useful pertaining to your problems or practical for the future of Rust. Feel free to ignore everything below.

As a mental exercise, it is interesting to imagine how things could have been implemented differently. AsRef has its blanket implementation for APIs like File::open, and AsRef<Path> in particular was a significant motivator. Could we have done something different and gotten AsRef<T> for T instead of the &-nesting it overlaps with? [1]

An aside about `Into`:

One interesting note in the RFC is the section Why the reference restrictions?. There we can see that the author explicitly wanted to avoid HRTBs [2] such as:

fn bar<P: ?Sized>(_: &P) where for<'any> &'any P: Into<&'any Path>

And later they say that AsRef implies Into. However, the blanket implementation making AsRef imply Into was later removed as it conflicts with Into being reflexive.

This theme of wanting to avoid complex signatures comes up again below. That said, I feel it's a worthy separation of concerns in this case anyway; I'd hate to have to write (&x).into() and hope I got it right.


Now, if we look at File::open and friends, we can see that they use the pattern I asserted was an anti-pattern:

pub fn open<P: AsRef<Path>>(path: P) -> Result<File>

Which I guess I should back up:

  • Open can't do anything but create the &Path so it has no need for ownership (of a PathBuf, say)
  • as_ref takes a reference as well, so you're not really losing much by requiring a reference as an input, just...
  • ...the ability to pass something owned into open [which] is something you usually don't want to do
    • Because then you, the caller, can't use it anymore [3]
    • A hint to use AsRef would help, but there is none in this case, so the unwary will probably .clone unnecessarily upon getting a "used after move" error
    • (Or a hint to just use & for that matter, given the &-nesting support)

And one more pertinent quality:

  • If you use this pattern, you need to support some level of &-nesting in the implementations, because you can't have P = Path (or any other unsized type) -- you need AsRef<Path> for &Path so you can pass in the P = &Path.

We can't change those patterns now, because those functions accept owned types directly. Bummer. If we had instead

pub fn open<P: ?Sized + AsRef<Path>>(path: &P) -> Result<File>

We, arguably at least, would not have needed the nested & implementation on AsRef. [4] As it turns out though, that impl was included from the start. And hey, if you read the RFCs closely, they actually did use this ?Sized and &P pattern in the Path reform discussion (though often forgetting to write ?Sized). So what gives?

PR 23316:

it's much more ergonomic to not deal with ?Sized at all and simply require an argument P instead of &P.

This change is aimed at removing unsightly ?Sized bounds while retaining the same level of usability as before.

So because it's unsightly :roll_eyes:, we lost the more correct (IMO) implementations forever :-1:. [5]

That said, again, the PR is not what prompted the &-nesting implementation, so further insight would have been needed to avoid it. And maybe it justifies its existence in other ways I haven't thought of. [6]

Well then, should we add more conversion traits and deprecate all the functions with the anti-pattern? No, the current situation works well enough in practice, flawed though it may be. A change like that would be massively disruptive to the ecosystem.


One more anti-pattern-esque thing I'll note which applies to both of

pub fn open<P: ?Sized + AsRef<Path>>(path: &P) -> Result<File>
pub fn open<P: AsRef<Path>>(path: P) -> Result<File>

They both potentially save you some typing to call over taking a &Path (no .as_ref()), but as a result you get a monomorphized version of the entire function for every type used. You can mitigate this by doing a one-line as_ref() call and then passing off to a non-generic private method that takes &Path, but I suspect most people copying std conventions don't do so. [7]

In summary, optimizing for writing is a bane upon programming. :wink: [8]


OK -- congrats on making it this far by the way -- I contemplated the above exploration in the context of your AsRef<T> as T thread after seeing your filed issue, because it reminded me about how the P: AsRef<Path> pattern is common but sub-optimal. In particular, I wasn't really taking the context of this thread into account; it ended up here by accident. [9]

But as a post-script, would having AsRef<T> for T solved the supposed problem in this thread? Well, we can't have both of

impl<X: ?Sized> AsRef<X> for X { /* ... */ }
impl<U: ?Sized, T: ?Sized + ToBorrow + AsRef<U>> AsRef<U> for Cow<'_, T> { /* ... */ }

because T might implement AsRef<Cow<'static, T>>, in which case they overlap. So without specialization, I don't think it would have really helped you out. Also, apparently, you found some arrangements of traits you do like which relies on the &-nesting (in a recent comment edit), so that's sort of amusing. It still must lose out on AsRef<T> for Cow<'_, T>.

Question though -- and again, sorry if this is just me not reading the thread thoroughly -- is there a practical reason you care, or is this all an "ideal design" exercise? I see a lot of types and calls in the example that just don't make a lot of sense to care highly about to me (though they can come up due to metaprogramming sort of easily I suppose). Having a Cow<'_, PathBuf> is like taking a &PathBuf instead of a &Path, or a &String instead of a &str, for example.

And particularly, in the context of the anti-pattern, in what situation do you own a Cow that you're okay throwing away in a call to something that wants AsRef<Path>? It can happen, but I think it's pretty rare, for the same reasons accepting PathBuf is a stumbling block -- if you have a (potentially) owned version, you're probably going to need it later. And you seemed okay entertaining the idea of avoiding the anti-pattern.

So if we discard that use-case [10], what are the remaining situations -- those where you don't want to give away the Cow? I think the top few would be

  • You own the cow and you're going to have to pass &cow or cow.as_ref() anyway so as to not give it away
    • So no great gain over having to do &*cow or the like
  • Your Cow is a field and you can't just pass self.cow either
    • So similar to the previous situation
  • You own cow_ref: &Cow<'_, T>
    • but that should probably be a &T instead
  • You own cow_mut: &mut Cow<'_, T>
    • but that should probably be a &mut <T as ToOwned>::Owned (or &T) instead

I think this isn't a bigger deal in the ecosystem because it just doesn't come up much, and when it does it's easy to work around. The most frequent requests do seem to be of the AsRef<Path> for Cow<'_, str> variety specifically. Maybe you're hitting it because you're trying to be super generic?


  1. I use "&-nesting" as a synonym for "As lifts over &". ↩︎

  2. and where clauses more generally ↩︎

  3. unless it's Copy ↩︎

  4. And the "oops I gave away ownership" speed bump would be removed, though you might need to explicitly deref where you don't today. ↩︎

  5. Ergonomics of the std writer shouldn't outweigh good design for the std consumer. ↩︎

  6. So I guess the takes-ownership pothole is the main fallout from that PR, combined with the fact that others (including myself) often copy std's patterns, so the anti-pattern spreads. ↩︎

  7. Related RFE. ↩︎

  8. Less cheekily, using generics for ergonomics is hard to get right, and also hard to change without breaking something. ↩︎

  9. And I've only skimmed this one, so pardon me if I retread old ground. ↩︎

  10. i.e. let's say you don't use the anti-pattern so you can't take owned values ↩︎

1 Like

I don't know where to start! :sweat_smile:

First of all, thanks a lot for your in-depth repsonse. Before I dig into the details (which isn't easy for me, by the way), I will try to make some of my motivations more clear, and maybe that can clear out some misunderstandings that there might(?) be in regard to my bug report #98905. You may see these explanations as "non-normative" addendums to my bug report :stuck_out_tongue_winking_eye:

I didn't file the bug report because I had a practical problem myself (yet). I filed it because I ran into having to do this:

deref_owned

Changelog

  • 2022-07-04: Version 0.9.0 (yanked)
    • Removed implementation of AsRef<U> for Owned<T> and implement AsRef<T> only (this is to allow going from Owned<T> to T where T: !AsRef<T>) (Yanked because .borrow() should be used in that case.)

The practical issue began with this simple wrapper (old version), which I needed here for example, due to the GAT there. Note that this led to the crate deref_owned, which is supposed to provide a general solution for this sort of problem, i.e. mmtkvdb::owning_pointer has been removed in favor of using the deref_owned crate instead.

During the (difficult) process of finding the right API for deref_owned, where @CAD97 helped me a lot, particularly with this post on IRLO, I came across this implementation of AsRef from user "conradludgage" (see his post on IRLO).

So I did the equivalent in my code of deref_owned in version 0.2.0, see here.

It wasn't until later when I saw the similarities between my trait IntoOwned (old/flawed version) and std::borrow::Cow. So I concluded:

(Note that in the above cite I still wrongly use Deref<Target = B> where it should be Borrow<B>, which is now a supertrait of GenericCow.)

So I got from needing a trait for a bound on my GAT in mmtkvdb to creating a generalization of Cow.

I wanted to improve my GenericCow, so I looked at std to see what the Cow type from std does, and what I should add to my type Owned, an implementor of GenericCow, in order to be most complete. This is where I stumbled upon Cow<'_, T>'s implementation of AsRef<T>, and I (wrongly) concluded:

I already had a feeling that something was wrong here:

I later noticed that going from a "maybe owned / maybe referenced" (Cow), "always owned" (Owned), or "borrowed" (&) sort of type to the "inner" borrowed type, the correct way is to use .borrow() instead of .as_ref() (because we cannot go from &T to T with .as_ref()). An alternative is to use (transitive) deref-coercion (which doesn't guarantee Ord/Eq/Hash consistency), but this can't be expressed with a bound:

So after all that (which was a looooooooooong process), I concluded that Cow doesn't implement AsRef properly because it offers using .as_ref() where instead .borrow() is the semantically correct interface. This comes which implications, which ultimately lead to problems again and again, I believe, just like #73390 (which was dismissed here) or workarounds like impl AsRef<Path> for Cow<'_, OsStr> (which made it to std in 1.8.0).

The motivation of my bug report #98905 is to document an (alleged) error in std. This may serve several purposes:

  • If there is a way to fix Cow, it should be done.
  • If there is no way to fix Cow, it should be documented somewhere else (maybe in a documentation comment, or perhaps in an erratum?), except:
    • If I'm wrong with my assumption that Cow<'_, T>'s implementation of AsRef<T> is incorrect (given the current semantics of AsRef and Borrow, including "As lifts over &" aka "& nesting" for AsRef), then I should be informed about it and be given the opportunity to learn why.
  • The library team may get aware of this potential bug which can be helpful during future decisions (or when getting bug reports like #73390).
  • Even if you seemed to dislike using #98905 as an example, I believe it may serve as a good case study when considering the implications of Rust's stability policy regarding to a potential structural problem. As said on IRLO, I see parallels to the Functor Applicative Monad mess-up, which is my own "Schreckgespenst" :wink: (for other people that is "Python 2 / Python 3").

I really do this out of an interest in the language itself! If I just want to get my work done, I wouldn't spend so much time on that issue.

I still want to respond to the technical details of your post (which I'm very thankful for), but I'll do that in a separate post later if I find time (which I think I will). I'm getting hungry and haven't eaten yet. :yum:

Before closing this post, I would like to share something about my personal experiences regarding this issue (going to the meta level here):

Dealing with these abstract considerations is pretty hard for me. I feel like my brain is at its maximum capacity when trying to understand semantics of AsRef, Borrow, etc. I'm all time afraid of not really getting it correctly and being lost when trying to design a good API. Errors in std (if it really is an error) make this even harder.

I often come across situations where Rust's type system isn't providing a good solution for my problems. Instead of going the easy way, I try to do things right™. That is not because I want to be very efficient (it's a huge time sink) but because I want to deepen my knowledge about Rust (and maybe also because I hate inconsistencies :wink:).

I felt like instead of getting positive feedback when bringing up issues that involve criticizing Rust, I often get a lot of negative responses. Moreover, some of those negative responses receive a lot of hearts :heart: :heart: :heart: :heart: :heart: :heart: :heart:, while my posts or bug reports seem to be seen more like a nuisance or threat (e.g. to stability). :broken_heart:

This is pretty frustrating at times, and I would appreciate if I would get some more positive or constructive feedback at times, like “Interesting point! I agree this is a problem, but we cannot solve it like you proposed because of X.” or, respectively, “You made a mistake here. You were right if X, but we have Y, so Z.”

Posts like your most recent one make me love spending time on this forum, but phrases like

  • “seems pretty pointless”
  • “molehill into mountain topic”,
  • “that is irrelevant”,
  • “elegance is irrelevant”,

(from various people)

along with the huge number of hearts received, sometimes make me really wonder if my input here is appreciated. Maybe I'm overly sensitive again, but I wanted to give that feedback from my side. So thanks for your most recent post.

I'll try to write more on the technical details later if/when I find time.

1 Like

You are right, I didn't think of the ?Sized. A mistake that I often make. I tried to contrive an example:

use std::path::Path;

fn bar_sized<P: AsRef<Path>>(_: &P) {}
fn bar_unsized<P: ?Sized + AsRef<Path>>(_: &P) {}

fn main() {
    let path: &Path = ".".as_ref();
    // This won't work:
    //bar_sized(path);
    // But this does:
    bar_unsized(path);
}

(Playground)

Yeah, I see, so you an "work around" this issue when calling the function.

It's not so easy for me to follow you here; I notice you have much more background knowledge about the actual history of Rust. I'm still very new to it.

Uh oh :fearful: … I feel like I understand where this is going to end. :scream:

I said if AsRef for Cow was implemented "right" (as in consistent with &-nesting of AsRef), then impl AsRef<Path> would be okay. But the "root of all evil" (sorry to exaggerate) is in fact this:

// As lifts over &
#[stable(feature = "rust1", since = "1.0.0")]
#[rustc_const_unstable(feature = "const_convert", issue = "88674")]
impl<T: ?Sized, U: ?Sized> const AsRef<U> for &T
where
    T: ~const AsRef<U>,
{
    #[inline]
    fn as_ref(&self) -> &U {
        <T as AsRef<U>>::as_ref(*self)
    }
}

Instead, this should be impl<T: ?Sized> AsRef<T> for T, which then also resolves that, but requires we have to write:

-pub fn open<P: AsRef<Path>>(path: P) -> Result<File>`
+pub fn open<P: ?Sized + AsRef<Path>>(path: &P) -> Result<File>

:sob: :sob: :sob: :sob: :sob: :sob: :sob:

This rises the question whether impl<T: ?Sized + ToOwned> AsRef<T> for Cow<'_, T> isn't wrong (in an ideal world). But I'm not sure on that. Maybe it's still wrong because .as_ref() isn't transitive and that implementation prohibits proper method dispatch. I feel like the correct approach then would be to not implement AsRef for Cow at all and only rely on deref-coercion. On the other hand, we could demand that unwrapping the Cow and doing a cheap-conversion to another type are two subsequent operations and would require a chained .as_ref().as_ref().

Can you comment on that?

So my understanding now (before reading the second part of your post) is:

  • If impl<T: ?Sized, U: ?Sized> AsRef<U> for &T is taken as granted, then #98905 is well-founded.
  • If instead we had impl<T: ?Sized> AsRef<T> for T, then Cow should not be implemented like the suggested fix in #98905, but instead could be either
    • be left as is, or
    • have no AsRef implementation at all.

Would you agree, i.e. did I understand everything correctly?

:hot_face:

I didn't think on overlapping, but I guess I came to the same conclusion already:

  • If instead we had impl<T: ?Sized> AsRef for T, then Cow should not be implemented like the suggested fix in #98905, […]

The practical reason is:

  • I want to avoid unnecessary clones in mmtkvdb (more as an "ideal design" exercise rather than having real problems with performance at this point), which leads me to:
  • Having the deref_owned crate to provide a generalized Cow (GenericCow) and struggling which traits to implement for Owned.

I don't have a real practical problem with #98905 other than that it deeply confused me and made me go crazy almost!! :exploding_head:

I don't use Cow but only GenericCow in my project, so I'm not affected by the practical implications. Other people might, however, (compare #73390, for example)!

But not easy to understand, if you really want to! I wonder how many people even understand what your post or this response is about.

I didn't hit the issue in practice. I merely noticed it and decided to report it.


Now what to do? If you have time, I would invite you to comment on #98905 (or maybe better open a separate thread on IRLO?) regarding the question whether

  • #98905 (including its proposed "solution", disregarding whether that solution could ever land due to backwards compatibility concerns) is correct, or
  • impl<T: ?Sized, U: ?Sized> const AsRef<U> for &T being "wrong" is the "real" issue here.

Even if the latter is true (i.e. if As should not lift over &), I think #98905 still shows an inconsistency in std.

I'm not sure if there's anyone interested in discussing this, and I'm also not sure if I'm having enough understanding to really contribute to this issue in a well-founded manner. I feel like I need to learn more about type theory. Anyway, I would like if #98905 doesn't end up "forgotten", even if std won't be fixed. I would like to see some sort of "outcome" from all of this, whether it's an errata, fixed documentation of std, or a post explaining AsRef / Borrow / As lifting over & / etc. better.

P.S.: I would also like to invite you (only if you have time) to review my deref_owned crate with all that background in mind.

P.P.S.: I think the (hypothetical) clean solution is to deprecate std::convert::AsRef and reintroduce a new trait, which is defined properly. Again, this is just hypothetical (at this stage) to better understand the problem and what the "clean" solution would/could be.


  1. I use "&-nesting" as a synonym for "As lifts over &". ↩︎

  2. Ergonomics of the std writer shouldn't outweigh good design for the std consumer. ↩︎

To me it feels more like "tip of the iceberg" rather than "molehill into mountain". Disclosure: I'm not a native English speaker, so I might have also misinterpreted that phrase.


Rethinking about this, I assume my main problem is that all this background knowledge which I now (hopefully correctly) gained was not accessible to me beforehand (or maybe it was, but I didn't know where to look).

This leads me to another question: I wonder if these details are commonly known within the Rust community. Maybe most people don't usually care or don't need to know, but I feel like these details are really vital if you want to truly understand what AsRef is about. (And they help if you are lost.)

Moreover, deref-coercion and Option::as_ref make things even more complicated for many users.

I feel like the details regarding the AsRef blanket implementations (and possibly also Cow's) need to be added somewhere to std::convert's and std::borrow's documentation. What do you think?


For the record: I entirely disagree. :laughing:

What you wrote was very helpful for me, and I also believe it is important for the future of Rust: Even if std will not be changed in any way, I think this is very important for purposes of documentation and helping people to get a deeper understanding. It may also help to "handle" issues/PRs like #73390 in ways that are more satisfying for the original reporter(s) than just: "It's a breaking change. Not worth the effort. Learn to live with the warts. Wontfix. Closed." (I'm exaggerating a bit to make my point here.)

Asides:

More people should use

Part of this is the venue. On IRLO or the GH bug tracker, there's an assumed "this should be added/fixed/changed" color unless you are very clear and upfront to counteract that default. (E.g. a lot of my irlo topics start with something along the lines of "this is to explore the viability of a half baked idea" or "this assumes that X, and if not X, it is invalid.") If the goal is primarily exploring using the language (even most "did std mess up" inquiries) are a better fit for URLO which has a default color of exploring and understanding the language/library as it is.

There's a limited amount of time between IRLO/RFC/bugtracker frequenters, and there's a not insignificant amount of "defense" that does have to be done where someone legitimately but naively is requesting breaking changes or intractably large migrations. So for better or for worse, if a quick read of the first paragraph and quick skim suggests that something falls into that category, it'll get a terser reply rather than engaging with details and trying to teach (as these platforms are explicitly not a teaching platform, even though you do learn a lot from observing and participating).

Adding on to this, a new face is more likely to get the terser reply, since you don't have the good-faith assumption coming from being a familiar face who has contributed constructive dialogue before. (E.g. I'd probably have more leeway to post a "std did something wrong; how should we handle this" topic than you would, since I'd be more likely to benefit from the assumption that I've done my (heavily cross-linked) research on the topic.)

If you feel like you've been improperly dismissed, please do contact the moderation team. "Calling the mods" on someone isn't an intrinsically bad thing. The job that the moderation team exists (and volunteered) for is to take cases where someone was more aggressive than called for and reach a resolution between the parties. Even just a "hey, remember the human" reminder from the mod team can be a valid resolution and significantly improve the situation going forward.

If you don't feel welcome in an official Rust venue, that is an issue worth contacting the mod team over.

Hey, this is an actionable issue!

Consider (if you have the time/effort/desire) creating a documentation issue with

  • what you didn't know about AsRef/Borrow
  • what misconceptions you had
  • what sources you had consulted to lead you there (if you remember)
  • what you've learned since
  • how this changed your understanding
  • suggestions on what could be added (and where!) to smooth this learning curve for future learners

Onboarding is something the rust project very much cares about, but it's hard to do well because the people with the knowledge to write the documentation are significantly removed from the people who will benefit from what the documentation should be pointing out. Certain things which are naturally intuitive to experienced users (say, the implications of a blanket impl)

The reference transitivity of AsRef is called out in the documentation. Similar sections for "when should I implement AsRef" and "when should I take AsRef" can and probably should be added!

(It's also worth noting that despite often containing really good documentation and explainers, module-level documentation is much less visible than type-level documentation. This is unfortunate, and why many types end up linking the module-level overview to help it be more visible.)

For better or for worse, what I think AsRef (and its &-transitivity) is actually trying to model is a generalized autoderef coercion, i.e. generalizing what allows you to pass any of [&str, &String, &&str, &&&&&&str] to fn(&str) to allowing you to pass any of [&str, &String, &Path, &PathBuf, &OsStr, &OsString] to fn(&Path). I don't have the history books open to place AsRef w.r.t. autoderef removing a lot of required &*. (With modern-day Rust, you'll rarely see &* (outside of highly generic code, anyway, where inference falls apart for lots of reasons) and when you do, it's usually considered a failing of ergonomics that leads to requiring the manual deref coercion. With working deref coercions, &* should be always replaceable with & (recursively for any number of derefs).)

Documenting this as the actual purpose might go a long way to avoiding misconceptions about how AsRef can/should be used.

2 Likes

Well, I'd more say I've developed a knack for tracking it down [1]. Most of what I linked to I looked up as I wrote the post. I've developed the knack by doing the same sort of things you're doing now, exploring the language and so on. There's usually a lot of searching on GitHub involved:

Back to the topic though, I didn't introduce my post well at all, and it would have been better suited to the other thread. Basically the idea was

  • There is no AsRef<T> for T because it conflicts with AsRef<T> for &T where ...
  • The latter (or at least one layer ala Borrow) is needed for the <P: AsRef<Path>>(_: P) (anti-)pattern
  • But it is arguably not needed for the <P: AsRef<Path> + ?Sized>(_: &P) pattern
  • :bulb: So what would the world look like if we had AsRef<T> for T and not the auto-deref emulation?

And the conclusion was... a little less speed-bumpy [2] but not really all that different. In particular, it still doesn't allow the transitive AsRef implementation for Cow on its own (more on that later). Conclusion aside, it was a thought experiment from the start, because there's just no way we're going to revert the design now. [3]

Thanks for the summary, I understand the technical context much more thoroughly now.

It implements Borrow<B> too. Why do you consider AsRef semantically wrong here? It's semantics are "cheap reference conversion", and "return a reference to a field" is one of the common use cases, both of which are exactly what it does here. You can't implement Borrow<_> transitively for Cow either, if that's the idea; it runs into the same coherence conflict as MyAsRef did.

Actually, after continuing on a bit, I think I understand:

  • You really, really want a transative AsRef implementation on Cow
  • To the extent it's like an, I don't know... semantic must
  • Therefore anything that gets in the way of that must be non-semantic

Am I understanding that right? Assuming so, let's move on to...

Ah, so Cow<'_, B> doesn't implement AsRef<B>, so cow.as_ref() calls <B as AsRef<_>>::as_ref after deref coercion (assuming B implements AsRef)? That would "break" patterns like P: AsRef<Path> (of either variety), which is presumably why Cow has AsRef<B> to begin with, ala the conversion trait RFC. [4]

But yes, I believe it would allow your transitive implementation of AsRef for Cow. You lose AsRef<B> for any Cow<'_, B> where B does not implement AsRef<B>, and I don't think there's a way around that without specialization (or some other change in coherence).

Eh, I didn't really have any opinions on those per se. I mean, ideally? If it didn't wreck inference, it would make logical sense to have AsRef<T> for T and AsRef<T::Target> for T: Deref and Into<&U> for &T where T: AsRef<U> and, sure, transitive AsRef for Cow, if coherence could grow to handle it. Though there's a good chance it would wreck inference. "cow.as_ref() what? Here are the 20 targets to choose from..." [5]

To me, these questions are moot outside of mental exercises [6], because they are either inactionable [7] or, like, a decade away from being possible to implement [8]. I didn't take the exercise far enough to come up with my most ideal situation.

The transitive AsRef property is sort of interesting to think about, but I believe it took me so long to dial in to it being the crux of all of this [9] because, in practice, it just doesn't matter much. If you have a Cow<'_, B> in particular, you're usually interested in working with the B. If not, you probably chose the wrong B [10].


Ok, I'll end part one here. If I find the capacity I will return eventually for

  • Some comments on the meta-issues
    • Which would include any comments on what is or isn't "right", "well-founded", "broken", "needs fixed", "correct", etc...
  • Discussion on the technically actionable parts, including documentation

Parting, unexplored thought -- maybe Cow could have a

    pub fn bikeshed<T>(self) -> Cow<'a, T>
    where
        T: 'a + ?Sized + ToOwned<Owned = <B as ToOwned>::Owned>,
        B: AsRef<T>,
    {
        match self {
            Cow::Borrowed(b) => Cow::Borrowed(b.as_ref()),
            Cow::Owned(o) => Cow::Owned(o),
        }
    }

  1. and I guess sometimes I do retain it :sweat_smile: ↩︎

  2. in one area, but who knows, maybe more speed-bumpy in other areas ↩︎

  3. Maybe someday we'll have enough specialization and inference improvements to get some of these ideas implemented, but we're probably talking 5-10 years or more in the future, if ever. ↩︎

  4. I put "break" in quotes because like I discussed before, passing a Cow into that pattern is generally going to involve some sort of annotation anyway, unless you're giving up ownership; really, you lose some ergonomics, like needing &*cow instead of &cow always instead of in niche circumstances. ↩︎

  5. Exaggerating as transitive AsRef situations aren't that common I think, but remember how more AsRef<Path> already breaks things in the compiler and wider ecosystem. ↩︎

  6. which are, granted, plenty of fun ↩︎

  7. in terms of changing their behavior ↩︎

  8. ala loads of specialization or the like ↩︎

  9. if I got that right ↩︎

  10. but see the parting thought below ↩︎

Or the compiler could be improved?

I would like to make a statement in particular regarding #98905:

  • I consider this A BUG (p.s.: not so sure anymore, see the end of this very very long post).
  • I am aware that it cannot be fixed easily, which is why I added: "Fixing this may be a breaking change and/or require a new edition, I guess." I assumed that this is sufficient to put my bug report into context.
  • I provided three potential solutions in the bug report:
    • In my OP: Simply changing impl<T: ?Sized + ToOwned> AsRef<T> for Cow<'_, T> into impl<T: ?Sized + ToOwned + AsRef<U>, U: ?Sized> AsRef<U> for Cow<'_, T>, which is the breaking change that I understand can't happen (but I didn't comment more on that in that issue because this is a bug report and not a pull request).
    • In a comment: Introducing a GenericCow (which might be a good idea anyway :innocent:), and then migrating from Cow to a new Cow type. (In my post, I said "potential migration path", to make clear this is me fathoming possibilities here, and not making a request to change something at this stage.)
    • In a second comment, where I applied @quinedot's ideas. This was to clarify that the assumed root cause of the problem may be somewhere else than what I assumed in my original post in that issue on GitHub. Following these thoughts, the hypothetical solution would be to deprecate AsRef and provide a new AsRef2 (as I called it for purposes of referencing). (Here, I defensively added "Please note that this is not a suggested solution but merely a thought-experiment to better understand the cause of the problem.")

When talking about "the venue", I think there's a huge difference between a bug report and a pull request. Admittingly, I named the issue:

Cow<'_, T> should not implement AsRef but AsRef where T: AsRef

But this is what "the bug" is:

  • Given the current semantics of AsRef and Borrow:
    • Cow has a wrong blanket implementation for AsRef.

Yeah, I use the strong word "wrong" here. Same as it is "wrong" that in Haskell a Monad wasn't a superclass of a Functor if we also have an Applicatative Functor (which took years to be solved and caused a lot of frustration [1], I guess, see Applicative Functor Monad Proposal).

Maybe the issue in Haskell was much worse than what I try to point out with #98905 (or worse than the issue @quinedot brought up to surface in regard to the anti-pattern misuse regarding AsRef for overcoming "unsightly" syntax), but I'm sure that in the Haskell community there were a lot of people who argued: "Is it really a big deal? Let's just leave things as they are. It's not a bug, it's like it is."

But it's not only for semantic reasons that I claim #98905 is a bug. I think the inconsistent implementation of AsRef for Cow will cause practical issues where code doesn't compile [2] where it should compile unless we fix documentation accordingly so people know that their code will not compile and why it doesn't compile.

So I think the venue was exactly right for my concern.

I think such "bugs" should be acknowledged as such. Instead, I feel like the usual way to deal with inconsistencies in std is:

:see_no_evil: :hear_no_evil: :speak_no_evil:
(plus blaming the reporter for being inconsiderate about proposing breaking changes!)

And why is that so? Maybe because there is allegedly no solution to these problems. But that doesn't mean the problems (or errors) don't exist. And what I would like to propose (and I seem to have gotten positive feedback from @CAD97 on that matter), is to not dismiss these issues but at least create a documentation fix. [3]

But why? I feel like there is an atmosphere where you have to be extremely defensive when bringing up new ideas. I posted on a similar issue before. Not everyone who is entering this community "knows" these implicit rules what posting on IRLO or the bug tracker means (edit: and how wording is going to be interpreted!). And I think these rules are not good, because they lead to important matters being dismissed where they should not be. And I believe they lead to people (particularly those who understand the semantics of complicated abstract data models) leaving the community because they are disappointed of getting negative feedback on good ideas. But that's just a wild guess based on my own feelings in that matter.

That said, I would like to add that I overall did not feel bad about @CAD97's and @quinedot's responses (with some minor exceptions maybe, but I know I'm pretty sensitive also) which were all well-founded and helped me to advance my knowledge. I am really really thankful for your help!!

It was more the overwhelming amount of negative responses overall. But that's maybe also because many people don't understand the point of the issues I try to bring up (sometimes including myself).

I understand, but I have brought up topics like that here and I was referred to IRLO. Which again brings me to what I wrote here:

So… is "Smart pointer which owns it's target" a topic for URLO or IRLO?

:man_shrugging:

I think especially the new faces should be treated with a gentle response, but that's a problem I see elsewhere in the tech-scene as well. And being treated differently just because I have made constructive contributions before is nothing I want. I hate having to "prove myself". (But maybe I'm unrealistic here, and that's something I just have to deal with.)

Phew, difficult issue. Not sure what I should say. I'm not sure a moderator could have helped me in the particular matter. Afterall, nobody really did something really wrong (I think). It's a difficult issue where nobody is to blame, I guess. That said, maybe some things could be done better though.

I would like to do that (disclaimer: if I find time). In short that would be:

  • AsRef isn't reflexive (Playground) because of:
    • historic reasons to avoid "unsightly" notation,
    • which require 'As to lift over &" (I have to learn more about category theory),
    • which is conflicting with AsRef being reflexive.
  • AsRef isn't transitive (Playground).
  • Sometimes Borrow can and should be used where this is a problem,
  • Cow's AsRef implementation:
    • is (arguably) semantically wrong (see my OP in #98905),
    • but won't be removed due to backwards compatibility.
    • Instead of .as_ref() you can use dereferencing (*) or .borrow() (of which the latter may be ambiguous in some cases).

Of course, this should be phrased in a more verbose way and garnished with examples such that beginners can understand the implications.

Edit: @CAD97 Of course, this doesn't include yet the points you suggested. I was just listing my (current) understanding of how things are, so I don't forget later. And also not sure anymore about whether Cow's AsRef implementation is semantically wrong. But at least there are inconsistencies that should be pointed out!

But that's only the positive part of the story. :see_no_evil:

I will keep that in mind and I like the way to work around it: Provide good module-level documentation and link to it in the type-/trait-level documentation.

Nice try, but…

use std::borrow::Cow;
use std::ffi::OsStr;
use std::path::Path;

fn foo(_: impl AsRef<Path>) {}
fn bar(_: impl AsRef<i32>) {}

fn main() {
    foo("Hi!");
    foo("How are you?".to_string());
    foo(&&&&&&("I'm fine.".to_string()));
    foo(&&&&&&&&&&&&&&&&&&&"Still doing well here.");
    //bar(5); // ouch!
    //bar(&5); // hmmm!?
    bar(Cow::Borrowed(&5)); // phew!
    foo(Cow::Borrowed(OsStr::new("Okay, let me help!")));
    foo(&&&&&&&&Cow::Borrowed(OsStr::new("This rocks!!")));
    //foo(&&&&&&&&Cow::Borrowed("BANG!!"));
}

(Playground)

@quinedot: Do you see how I feel like this is "buggy"?

Yeah, but also the "flaws" (see Playground above) even if talking about them might not be so "fluffy". :pensive:

:sweat_smile:

So you developed that idea thorughout our discussion and didn't had it up your sleeve? So then this knowledge is indeed something not well-known?

Well, I outlined a way. :innocent: Didn't say it wasn't painful. :smiling_imp:

(Seriously, I understand if that wasn't done because it might leave us with a far greater mess in the end, especially if there's maybe a way out one day with specialization.)

It's difficult to put in words. I'll give it a try. It's because Cow<'a, T> relates to T just like:

We cannot go from either of those two to T via .as_ref() but must use .borrow():

fn main() {
    // using `.borrow()`:
    let _: &i32 = 0i32.borrow();
    let _: &i32 = Cow::<i32>::Owned(0i32).borrow();
    let _: &i32 = Cow::Borrowed(&0i32).borrow();
    let _: &i32 = Owned(0i32).borrow();
    // using deref-coercion:
    let _: &i32 = &0i32;
    let _: &i32 = &Cow::<i32>::Owned(0i32);
    let _: &i32 = &Cow::Borrowed(&0i32);
    let _: &i32 = &Owned(0i32);
    // using `.as_ref()`:
    //let _: &i32 = 0i32.as_ref(); // Won't work
    let _: &i32 = Cow::<i32>::Owned(0i32).as_ref();
    let _: &i32 = Cow::Borrowed(&0i32).as_ref();
    //let _: &i32 = Owned(0i32).as_ref(); // Won't work
}

(Playground)

No, I don't want. Unless you mean I should want? :thinking: I simply think that .as_ref() isn't the right operation to go from a "maybe reference-like type" to a reference. It's for cheap reference-to-reference conversion (just like the docs say). It can only be used in concrete cases, e.g. to go from PathBuf to Path, but not for the generic case (because for that it lacks reflexivity and, oh :sweat_smile:, now I get your point: transitivity).

Not sure what you mean. I merely try to draw conclusions from this failing:

let _: &i32 = 0i32.as_ref(); // Won't work

If that fails (which is a pretty basic operation), then I feel like I can't expect this to work either:

let _: &i32 = Cow::<i32>::Owned(0i32).as_ref();

But the first fails, and the second works. I guess I tried to find a pattern / solution here by claiming that going from a generic "maybe reference-like type" (aka impl GenericCow<T>) to a reference type (&T) is something semantically different. But maybe it's not much different than going from PathBuf to &Path? Not sure. I think if was not semantically different, then AsRef would be messed up completely (which is basically your point, and which I stated here).

Yeah, of course it would break that pattern. I was talking about an "ideal world" here. Remember, in that case we'd have &P where P: ?Sized + AsRef<Path> and where AsRef is reflexive at least.

Not sure what you meant, but this is something I just tried based on your Playground:

// these "unsightly" ones:
fn foo<P: ?Sized + MyAsRef<i32>>(_: &P) {}
fn bar<P: ?Sized + MyAsRef<str>>(_: &P) {}
fn baz<P: ?Sized + MyAsRef<OsStr>>(_: &P) {}

fn main() {
    foo(&0i32);
    foo(&MyCow::Borrowed(&0i32));
    bar("Hello");
    bar(&"Hello".to_owned());
    bar(&MyCow::Borrowed("Hello"));
    bar(&MyCow::Owned("Hello".to_string()));
    // We run into a problem due to lack of transitivity here:
    //baz(&MyCow::Borrowed(Path::new("Hello")));
    let my_cow = MyCow::Borrowed(Path::new("Hello"));
    let path: &Path = my_cow.my_as_ref();
    baz(path);
}

(Playground)

:face_with_spiral_eyes:

Not sure if any of this would work out. And now I'm not even sure if there is a proper solution at all, i.e. if #98905 really is a bug. Maybe this rather leads to an impossibility theorem!? No idea.

If that was the case: Sorry for the noise.


P.S.: I wrote an update to my bug report.


  1. And made learning the language a tiny bit more difficult. ↩︎

  2. Not because there is an explicit guarantee that it compiles but because people apply their knowledge regarding AsRef and Borrow. ↩︎

  3. Which is what I also suggested in a comment to my bug report: "If the cause of the issue cannot be fixed, it should be at least documented as an error in Cow's documentation." Maybe I phrased that a bit hard. ↩︎

  4. Maybe someday we'll have enough specialization and inference improvements to get some of these ideas implemented, but we're probably talking 5-10 years or more in the future, if ever. ↩︎

1 Like

But back to my original idea :grin:. Which of the following is right to do?

That's my actual issue, which I'm facing. :sweat_smile:


I think the first variant (i.e. what I did in version 0.8.2) is the semantically "correct" one because of what I just wrote in a comment on #98905:

I think that generic(!) smart pointers such as Cow, Rc, and Arc should behave the same as ordinary shared references in regard to AsRef, but they do not. Maybe there is a good reason for this inconsistency? It would be good to know if there is.

(I also added a note that this should be documented properly.)

The second variant (i.e. what I did in version 0.9.0) would be consistent with what std does.

Side note:

We should probably distinguish between generic smart pointers such as Cow or Arc, and concrete smart pointers (if we want to call them smart pointers at all) such as String and PathBuf. This distinction is not because of a structural difference but because it would allows us to work around the limitations of Rust's type system (lack of transitivity for traits, overlapping implementations). That could be (or could have been :cry:) used to create a consistent behavior of AsRef, Borrow, generic smart pointers, and things like String or PathBuf.

Part of the issue is that what you're discussing is much more fundamental than "here's an API addition," it's questioning/extending the purpose of the conversion traits. Which, notably, weren't really designed to be used beyond single-level function argument generalization.

Talking to the mod team doesn't (shouldn't) imply anyone is in the wrong, though! What it does mean is that there was a mismatch of communication/assumptions/etc which could be assisted by an outside party helping to moderate a resolution of communication/assumptions/etc, and provide pointers for avoiding future miscommunication.

Long-term, yes, but mid-term applying the transform manually is still necessary and provides a notable benefit.

Maybe the MIR opt framework makes doing this transform in the compiler more practical than it used to be, though...

On the contrary, I think impl AsRef<Path> + Into<PathBuf> (or i.o.w., perhaps impl AsRef<Path> + IntoOwned<Path>) taking an owned type to avoid a copy was/is a major design point; e.g. open taking &impl AsRef instead may make sense, but if ownership may be required it should be passed in if no longer needed. (Though in a super quick shallow docs scan I couldn't find a std API which takes AsRef + Into?)


Either way, there definitely should be a central, official resource covering exactly what/when/why to use/impl {AsRef, Borrow, ToOwned}, as this is a common stumbling block for new users.

One (minor?) thing @jbe: I don't think you've clarified what the semantic difference between IntoOwned<T> and Into<T::Owned> really is. There is the fact that GenericCow implies Borrow<T> (perhaps AsRef<T>, perhaps transitively?) whereas AsRef + Into spells it out.

I suspect the answer is what (blanket) impls are provided/assumed/possible/etc. Per the current std::convert docs, IntoOwned<T> should just be Into<T::Owned> or Into<impl Owns<T>>. The difference is the presence of Into, which e.g. isn't available for Cow<T> -> T::Owned because it conflicts with T::Owned = Cow<T>.


With a time machine, I think I'd like the generic conversion/etc traits to be (plus Mut versions for the by-ref traits):

  • Reciever<T>: is T or transitively derefs to T; method call syntax works at type T (and anything T is a receiver for) by-ref.
    • Rarely used, perhaps not even not provided.
    • Weakest design here; see Borrow.
    • Implied semantically lossless, though may remove capabilities.
  • From<U>/Into<T>: convert T -> U in the single obvious lossless manner (though not necessarily reversable).
    • Transitivity expectation, though a guarantee is impossible when combining independent libraries, as someone has to define the used into chain.
  • TryFrom<U>/TryInto<T>: convert T -> U in the single obvious manner, failing if lossy.
  • LossyFrom<U>/LossyInto<T>: convert T -> U in the single obvious, potentially lossy manner.
  • T: AsRef<U> <==> &T: Into<&U>
  • T: ToOwned <==> &T: Into<T::Owned>
  • T: IntoOwned <==> T: Into<T::Owned>
  • T: Borrow<U> <==> &T: Receiver<U>
    • In English, &T can be used as-a &U.
    • Derived fact: T: Borrow<U>, U: Borrow<V> => T: Borrow<V>

I have not done any real design work to see whether a) this covers all desired and current use cases, or even b) it works without semantic conflicts[1]. That's what the time machine is for; starting with this or a compatible design far enough in the past to iterate on it to an actually workable matrix.

This design has a ToOwned trait which sets a semantic projective owner, but also encourages using the more general form if you don't care about the specific owner. Likely this is best expressed as ToOwned being just a trait alias with a defaulted parameter... though types choosing their own default for such requires (an) undesigned language feature(s).


  1. Things like Cow: Into<Owned> being semantically unable to distinguish in the type system between into_owned and identity. ↩︎

On the issue tracker at least, if the context of an issue evolves from what the OP describes, the OP report should be updated to reflect that. If it's not in the OP, it's really easy to miss in GitHub's model due to hiding interior comments.

And while it may be correct to describe inconsistent trait implementations as a bug/issue on the issue tracker, GH issues on the main repo are typically expected to be somewhat actionable, or to be a C-tracking-issue of actionable items. A "std is inconsistent but probably can't be fixed" doesn't really belong on the issue tracker under current (implicit) guidellines; note that all rust-2-breakage-wishlist tagged issues (which is a tag that exists!) have been closed as unactionable.

Because they are not transitive, or what do you mean?

Maybe that's where impl<T: ?Sized + AsRef<U>, U: ?Sized> AsRef<U> for &T provides a good tradeoff. The idea:

  • Going from reference or smart pointer (or GenericCow) does not use .as_ref(), so AsRef can be used for going from PathBuf to Path, OsStr to str, etc.

But this idea (if it was the original idea behind "As lifts over &" at all), hasn't been followed through with when designing Cow, Rc, Arc, etc.

I'll take a look. You might want to add a short note to this thread.

Note that AsRef<U> won't work as expected with a Cow<'a, T> where T: ?Sized + AsRef<U> because of #98905, right? Moreover, T does not implement AsRef<T>.

So I need to use Borrow<T> to go from an maybe-owned type to the reference type and then I can use AsRef<U> to do a cheap reference-to-reference conversion afterwards.

Into<<T as ToOwned>::Owned> (opposed to ToOwned) isn't consistent in regard to Eq/Ord/Hash, i believe? So it's not the "counterpart" to Borrow<T>.

I think clarification is missing in the documention comment of GenericCow, explaining that the conversion implies that Eq/Ord/Hash must not behave differently for the owned type. Oh wait :sweat_smile:, that is already implicitly guaranteed because of the Borrow<B> supertrait! :smiley: So I do not need that clarification, I guess? std::borrow::ToOwned doesn't clarify either.

@CAD97 Is that what you meant? Oh, I see you wrote a second paragraph after that one, but first some more playing around with AsRef and Into:

use std::borrow::Cow;
use std::path::{Path, PathBuf};

fn foo1(_: impl AsRef<i32>) {}
fn bar1(_: impl AsRef<str>) {}
fn baz1(_: impl AsRef<Path>) {}

fn foo2(_: impl Into<i32>) {}
fn bar2(_: impl Into<String>) {}
fn baz2(_: impl Into<PathBuf>) {}


fn main() {
    //foo1(17);
    foo1(Cow::Owned(17));
    foo1(Cow::Borrowed(&17));
    //foo1(&17);
    foo2(17);
    //foo2(Cow::Owned(17));
    //foo2(Cow::Borrowed(&17));
    //foo2(&17);
    bar1("Hi".to_string());
    bar1(Cow::Owned("Hi".to_string()));
    bar1(Cow::Borrowed("Hi"));
    bar1("Hi");
    bar2("Hi".to_string());
    bar2(Cow::Owned("Hi".to_string()));
    bar2(Cow::Borrowed("Hi"));
    bar2("Hi");
    baz1("Hi".to_string());
    //baz1(Cow::<str>::Owned("Hi".to_string()));
    //baz1(Cow::Borrowed("Hi"));
    baz1("Hi");
    baz2("Hi".to_string());
    //baz2(Cow::<str>::Owned("Hi".to_string()));
    //baz2(Cow::Borrowed("Hi"));
    baz2("Hi");
}

(Playground)

Apart of semantics, we have to deal with:

  • T: !AsRef<T> (edit: for some T)
  • impl<T: ?Sized + AsRef<U>, U: ?Sized> AsRef<U> for &T
  • impl AsRef<str_and_the_like> for StringAndTheLike
  • impl<T> AsRef<T> for SmartPointer<T> :confounded:

This makes AsRef unsuitable to go from a generic reference-like type, including smart pointers (including Cow) but also including ordinary shared references, to a borrowed type.

In short: The usage of AsRef is pretty limited. I think the only "correct" (forgive me that harsh wording) uses are:

  • AsRef<str_and_the_like> for StringAndTheLike
  • AsRef<str_and_the_like> for str_and_the_like
  • AsRef<PathAndTheLike> for str_and_the_like

Which brought me to a third option:

I'd have a third option :grin:: BOYCOTT AsRef and not implementing it at all! BWAHAHAHAHA :rofl:

But there is a reason to go for this, instead:

That is: Both & and Owned have a consistent blanket implementation of AsRef then. This allows use cases where Cow is never used but only & or Owned to correctly use AsRef on the deref target.

Anyway, in that scenario AsRef is used depending on how the pointed-to-type defines it (e.g. str). Going from "maybe owned" to "borrowed" has to be done through Borrow, not through AsRef. The counterpart to Borrow isn't Into but ToOwned (or GenericCow when you want to avoid unnecessary clones).

Maybe that was a bit confusing and maybe I made some mistakes here. I hope you more or less understand anyway what I tried to say.

But what about Eq/Ord/Hash consistency here? If Receiver refers to Deref, then that's not reflected there either.


I think my OP still holds, as I pointed out here:

While I still believe that the statement of my OP is right, i.e. that

Cow<'_, T> should not implement AsRef but AsRef where T: AsRef

this cannot be fixed (easily).

Edit: I just updated the OP in the issue and the comment, explaining that I moved some part up for a better overview. Sorry I'm not so familiar with the best practices yet.

I didn't know about these "(implicit)" guidelines. :pensive:

So couldn't the issue then be just closed (maybe after the documentation task has been finished) and tagged accordingly, so it doesn't get lost for the "rust-2-breakage-wishlist"?

Do you think the issue should/will be closed before the documentation is fixed? Should a separate issue be created for it? I think this really is more of a documentation issue and a wishlist issue, not a tracking issue.

Feel free to give me concrete advice in regard to the issue (here or by private message, or in the issue itself).

Alright, let me try to address some of these. Since I said it, I guess I'll start with:

First, a clarification of the meaning: There is a small problem that is being treated like an extremely large problem.

Why do I think it's a small problem? [1] Because

  • There's generally a simple way around the examples of things that don't work
  • If it was a big problem in the ecosystem, we'd see questions about it all the time in this forum, for example
  • The lack of "I tried to do $concrete_thing but couldn't because..." examples [2]
  • I didn't file the bug report because I had a practical problem myself (yet).

  • I don't have a real practical problem with #98905 other than that it deeply confused me and made me go crazy almost!!

  • I didn't hit the issue in practice. I merely noticed it and decided to report it.

So it's not causing any practical problems, or no major ones anyway, but there's something inconsistent in the standard library, or at least something that violates your mental model of how things work. And I agree they're not ideal, by the way, but that's for a technical or "ideal world" discussion.

And why do I say it's being treated as a very large problem?

  • Most directly, we three at least [3] have been spending quite a lot of time on the general topic; this will be post 51 in this thread alone and there's 3-4 others
  • You started issue 98905 with the tone [4] that something should change even if that means
    • Exercising edition boundaries
    • Making a breaking change
    • Deprecating Cow
    • Deprecating AsRef
  • When it was stated that being a breaking change is a hard blocker, you started a topic on IRLO to explore the idea of discarding Rust's breaking change policies! (Using the bug as an example of why that might be reasonable.)

Any change to std requires motivation, because once it is stabilized, it is in there forever. Any breaking change to std requires extreme motivation, as it may break existing code; a soundness issue say, or breaking something very unused to gain something very valuable. Any disruptive change (like deprecating widely used traits or structs) is somewhere between, but closer to the latter than the former -- and the more widely used it is, the more like-a-breaking-change it is.

There's no motivating case here, or at least no practical one. Nothing to justify massive disruption to the Rust ecosystem, for sure. Could anything rise to that level? Something the scale of Leakpocalypse, maybe. [5]

It would take a mountain, you might say.

I also used that particular phrasing to that particular person to try and communicate that I wasn't concerned with the possibility of issue 98905 leading to a breaking change, but would still welcome some reassurance on the limits of allowable breaking changes. [6] I.e. please don't just say "don't worry about that one", can I have some feedback on the overall limits?


So, implicit in all of that is that Rust's breaking change policies are extremely important. I'm honestly not sure what to say here that wasn't covered on IRLO; the short version is, languages that break all the time don't generally succeed at scale; they remain niche or toy languages. Languages that break also have less rich ecosystems, as not all projects in maintenance mode will get updated. People are very excited about Rust right now, and it is gaining significant adaptation. It's competing for mindshare next to languages with multiple decades of backwards compatibility; parts of your OS and standard utilities were written years and years ago. If Rust starts breaking in significant ways, most feel [7], its momentum will die.

It's not about economics, to me anyway, but it is about mindshare. I want the language to succeed. Assuming it doesn't go off the rails in some way, if I can be programming in Rust until I retire and not programming in C, I want that. Using less buggy software would be a nice change, too :slightly_smiling_face:. If it loses momentum and stalls out at Ada or Haskell levels of visibility, that dream is gone.

I feel this is a pretty common goal, not to mention a founding principal of Rust 1.0.0. Within that context, a breaking change or large disruption [8] with no practical motivation is simply a non-starter. I'm not a team member, but I know the project well enough to confidently state that.

Similarly, by "there's just no way" I didn't mean "isn't possible", I meant "this is such a minor papercut that disrupting the entire ecosystem is a non-starter".

I probably ran into some of these shortcomings before, but I couldn't have told you exactly what implementations did or didn't exist before I looked it up, nope. [9] If I'm in the middle of programming and something like the failing examples come up, I'll usually just slap a deref or whatever on the variable and continue.

There are at least a couple exceptions, one of which may be telling:

  • I'm completely lost and have to go retrain myself on some set of traits
  • I'm trying to do something quite generic and then I have to go figure out why the implementations don't line up

In the second case, I personally find, a solution is often to back up and dial down how generic I'm attempting to be. Yeah, it'd be really nice to consumers of the library to be able to call foo() with a bazillion different types [10], but if there's only really 1 that counts (e.g. &Path), it's not some huge problem for consumers to do the conversion themselves. Or perhaps I'm committing what I internally think of as "generics as macros" when its a bad fit, and I should really just reach for actual macros to get an implementation for the few dozen types I need.

At any rate, I think these episodes where you discover the limits of traits and generics come up more in the design (or "design by implementing") stage, which is what happened to you. I also think most people just search for a solution at that point (and perhaps don't reasonably have the time to do anything else).

Probably they're not commonly known at the level we're talking about, because you usually don't need to know (and there are simple workarounds).

If almost no one really needs to know, and it doesn't cause significant roadblocks, is it really an iceberg of a problem?

Better diagnostics and documentation would be wonderful.

But it's also been that way 7 years and rarely comes up (in this forum, say).

I pretty much agree with @CAD97's reply regarding venues.

A document somewhere explaining why breaking changes are non-starters for most issues might be a good thing to have, although then you're running the risk of turning a lot of issues into form-response type interactions.

On the other hand, I feel any frequent contributor to Rust understands the breaking change policy already, so probably it's a net positive for newcomers.

I'll leave the bug-or-not discussion [11] for another day.

On the topic of brevity more generally, I think it's just a fact of life when it comes to maintaining a popular open-source project. There are only so many people active in the project, and almost no one has the time to take hours to explain policy, or explore alternative histories of how Rust could be, or argue about what the semantics of traits should be... or to hash out some major redesign about how they are. (And when they do, the issue tracker isn't where they do it.)

They're generally not being rude [12]. They're triaging or otherwise putting a dent in one of the 600 open PRs (or 8,000 open issues).

Have you read many Rust GitHub issues or PRs? This Week In Rust is a good source for accepted ones, but it might be good to read some closed ones too. I don't see anything wrong with how PR 73390 or PR 39397 went for example. Routinely rejected PRs aren't personal affronts, it just means things can't work out for whatever reason. Probably the submitters (I recognize the names) are perfectly aware of Rust's backwards compatibility guarantees and didn't bat an eye, even if they sighed.

I do think there's a larger communication issue here, but I don't really know what to do about it. When I read the top comment in 98905 right now (Jul 8 04:05 UTC), it seems quite reasonable -- Cow should implement AsRef transitively when possible and there's a set of things that should be documented better in the meanwhile.

But it when it was opened, it read as a request to make a change, and that's how the first few comments go. (And that's also what issues are typically for.) Then there's about a thousand or two more words with invitations to read these other long threads as well. From a Rust maintainer's perspective, this is that molehill/mountain thing -- they can tell it's a non-starter (in terms of being a near-term change request) and spending hours reading about it or discussing it won't change that. [13] Moreover, expecting them to do so or being upset when they don't is asking a lot!

Maybe it all comes down to what @CAD97 said about choosing venues and stating your intentions up-front. But I also feel some shift in expectations and sensitively could help.

The maintainers and almost all of the regular participants in these forums are here because the love the language too.

Thank you, and I hope you still feel that way. Everything in this reply is meant to be constructive.


  1. "It" is a little vague here because different facets of the various trait implementations are explored, but anyway. ↩︎

  2. I admit I'm much better with concrete problems than abstract ones ↩︎

  3. you both more than I until recently ↩︎

  4. this could be a language barrier at play ↩︎

  5. Or not; that was (just) before 1.0.0. There are a number of outstanding soundness bugs that will just be fixed eventually :tm:, including some as old as Leakpocalypse. We'll probably get scoped threads back in std this year (7.5 years later). If it happens today, maybe we all just live with the std/compiler bug for a decade. ↩︎

  6. Context: there have been a few before that exceeded my model of where that line was or seemed otherwise cavalier, and some serious talk about more. ↩︎

  7. certainly I do ↩︎

  8. even if not technically breaking ↩︎

  9. My memory isn't stellar though, sometimes I'll research something, search for issues, then find one with a comment from myself a year ago :person_facepalming: ↩︎

  10. sort of; recall the over-monomorphizing and too-many-choices-for-inference hazards ↩︎

  11. and the rest of the semantic and technical issues ↩︎

  12. on the Rust repos anyway ↩︎

  13. I could tell too, and that's what my comments and citations were about. ↩︎

9 Likes

(Maybe I'm overly sensitive to this particular change, but changing to panic-in-drop=abort is being considered and feels similarly impactful, if less in-your-face about it.)

1 Like

The German translation feels pretty derogatory and doesn't address the problem but more the person doing so. (Note: I bring this up to clarify why I felt offended, not to call you out.)

See English Wikipedia, for example:

Making a mountain out of a molehill is an idiom referring to over-reactive, histrionic behaviour where a person makes too much of a minor issue. It seems to have come into existence in the 16th century.

To be clear: To me this issue (#98905) truly felt like a major issue (and to some extent it still does). So please let me assure you that I didn't make it bigger than it seems to me. I made it bigger than it seems to you perhaps, and probably bigger like it really™ is. But if the latter is the case (which is a subjective question anyway that cannot be ultimately "decided" upon), I wasn't aware of that.

I will answer only to some points where I disagree, and I probably agree to your other points in regard to the problem being small.

The problem are the generic cases (which is what traits(!) like AsRef are about). And here we have a lot of trouble with AsRef, it seems.

I might say, I had a concrete problem, but:

  • I didn't communicate it in the issue, because:
  • It had to do with me having difficulties finding the right implementations for a very very abstract concept (namely the Owned wrapper, which I felt wasn't mature enough to serve as an example).

Nonetheless, I saw a "flaw" (in my perception) that I wanted to report. Does this make it a small problem because we do not …

I don't know about the history of the Applicative Functor Monad proposal, but I believe that a lot of people didn't "hit" this particular issue when working with Haskell. Nonetheless, I consider it a major flaw in Haskells prelude. There was arguably a time where nobody even knew about applicative functors:

  • Haskell came out in 1990
  • Applicative functors were (probably?) discovered in 2008 (Wikipedia)

Not sure when the Functor became a (direct) superclass of Monad, but likely before 2008. So I suppose there were many many years where nobody "hit" that issue. Yet it was there. And yeah, it was a theoretical issue, just like #98905 maybe is.

Does it make the issue "small"? If we talk about practical issues, perhaps.

Inconsistencies like that (especially if not properly documented) make understanding AsRef (and thus also Borrow and Deref) very difficult. These three traits are a core part of the language. (Note that std::borrow::ToOwned is not, by the way). Thus we're not only talking about std but rather core.

I think it is helpful if there are guidelines to distinguish these three:

Unfortunately, restrictions in Rust's type system force us to skip some implementations that (arguably) should be there. These missing implementations may have an additional impact on which trait should be the correct one to use in certain cases.

Misconceptions arising from

  • inconsistently implemented traits in std or core,
  • lack of documentation, and thus
  • inconsistently implemented traits in third-party crates

do cause a real harm, I believe.

Maybe I'm "making this problem bigger than it is", not sure. I do see a real problem here.

It took me three days to get from

impl<T, U> AsRef<U> for Cow<'_, T> "should" be implemented

to

All three [Cow, Rc, and Arc,] are inconsistent with &

I think that generic(!) smart pointers such as Cow, Rc, and Arc should behave the same as ordinary shared references in regard to AsRef, but they do not.

I noticed something was (arguably) "wrong", so I thought I write a bug report. As I said earlier, this issue is very hard for me to grasp. That is why I was (for my standards) pretty careful with my wording in the issue, using "should" and "suspect":

The cause is subtle and I suspect it is in std.

Maybe I wasn't careful enough. I felt confident enough to see a real problem there and to post a report. Maybe I shouldn't have done it. It caused me a lot of frustration, that is. (Including having posted about my idea of a "smart pointer which owns its target".) But you are not to blame for my frustrations. It's me having had the wrong expectations.

It still worries me that Rust has no way to fix std if things are wrong. I expressed that concerns on IRLO, which I likely also shouldn't have done. (edit: maybe all this is too "trivial" to talk about?)

To clarify: I didn't start the issue on IRLO to push #98905 being implemented!! [2] (It's not even a PR!) I started the issue because I was worried about a structural problem that I was already worried long ago before #98905 had been opened by me.

:+1:

Maybe I was being naive there. That doesn't change anything about the structural problem I see. My point there is mostly Rust's complexity (first part of my subject on IRLO), not the "breaking change policies" (second part of my subject on IRLO). Perhaps I shouldn't have communicated my worries.

That depends on scope. For the theoretical foundation, I see an iceberg. For everyday's programming, no, there is no (big) issue.

I still believe that errors in the foundations can later lead to problems, sometimes huge problems like the commonly feared Python 2 / Python 3 mess (which I don't think is a mess btw [4]).

Noted, and I will keep that in mind.

One tiny thing I would like to add though: Stability is just one side of the story. There's also maintainance and updating your code. But no need to dig deeper into this; I understand that stability is important here.

I still feel good about your response, but I'm overall not really happy :man_shrugging:.

I'm left with the feeling: Better don't post on some issues. (Not your fault [that I felt like not posting here], plus I might need some time to think about this.)

Having a local Rust user group would be nice to be able to discuss these matters from face to face (Covid aside), but I doubt there are many (local) people who are interested in talking about these abstract matters. I really don't have anyone to talk to about Rust other than writing here (with the exception of high-level talk about Rust to colleagues or friends, but never really going into details).

However, thank you very much for your response on these meta-issues and for clarifying some things. I do believe there was some miscommunication (receivingly and sendingly) on my side.

Edit: Striked out the non-constructive parts. I'll try to address things differently when posting here in future.


  1. I admit I'm much better with concrete problems than abstract ones ↩︎

  2. I saw it as a possibility to implement it one day, and I saw it as something that "should" (ideally) be implemented. ↩︎

  3. Context: there have been a few before that exceeded my model of where that line was or seemed otherwise cavalier, and some serious talk about more. ↩︎

  4. I think Python did a great job in fixing flaws. But I'm not familiar enough with Python to truly judge about it, and I'm sure 95% of the people here would disagree with me. ↩︎

1 Like

Leaving the meta level now.

I still would like to get clarity on whether …

I'm asking for the following purposes:

  • I consider to write a short tutorial on AsRef/Borrow/Deref (or even PR to fix the documentation?). For that, I need to truly understand what AsRef is semantically about. Behavior of generic smart pointers can help me in that matter, and I feel like what I find in std might be more of an "accident" rather than intended.
  • I'm asking for my own generic "smart-pointer-like" (*ducks*) struct Owned, where I provide an implementation of AsRef. Currently that implementation is consistent with & but inconsistent with Cow.
  • If my thesis above is wrong, then #98905 could just be closed. (Still might be closed for other reasons anyway.) Maybe there is really a good reason for Cow implementing AsRef as it does (and in particular differently than shared references do).

I know I have already consumed very much of your time, so I don't really want to push that further. :sweat_smile: (Thanks again for all your help!) I feel like I have a good overview already and will be able to solve my own (practical) problems in a good way with all the knowledge I gained so far. However, if you or someone else likes to comment on that, I'm happy to read your opinions/responses on that issue.

1 Like

Have you considered the possibility that the 'negative' posts with lots of hearts are correct.

From my reading of Cow<'_, T> should not implement AsRef<T> but AsRef<U> where T: AsRef<U> · Issue #98905 · rust-lang/rust · GitHub , it's not "nuisance or threat" -- it actually breaks inference on existing code.

Please consider the context of my post:


I ran into inference issues now too (since I replaced Deref with Borrow @CAD97, even though I still think it's better to not have Deref being a supertrait of GenericCow). Some of my syntax might become more verbose.

Anyway, I added a comment on inference to #98905 and updated the OP accordingly:

I think that generic(!) smart pointers such as Cow, Rc, and Arc should behave the same as ordinary shared references in regard to AsRef, but they do not. One possible reason for this difference may be the need of type annotations (see comment below). This might indicate that the problem cannot be generally solved in a satisfactory way (at least if certain conversions shall have an easy-to-write syntax).

This is a good and well-scoped thing to open a new urlo thread about. A two-pronged question of "when/how should a type implement AsRef and "when/how should an API be generic over AsRef" is a well-defined discussion point, so long as it is scoped independently of requiring reading the entire multiple weeks of context leading to that being the question asked.

The conclusion of that discussion can then be used as evidence for the addition of certain implementations to std. Or if the implementations implied by the definition of AsRef cannot be added (e.g. due to unacceptably breaking inference) then we return to trying to define AsRef in a more concrete manner that doesn't imply the implementations that std does not have.

(Oh, and: an item being in core has no implications on how fundamental it is to the language or to the std APIs. It only speaks to it being definable on a free-standing distribution without an OS or dynamic allocator. As two counterexamples: Box is a primitive type which is only defined in alloc, and the Error trait is only in std due to an implementation choice of mentioning Backtrace. An item being fundamental to the language is more correlated to if it has a #[lang] annotation in the source (i.e. the compiler knows about it).)

3 Likes