I proposed that something like deref_owned::Owned (similar to Cow but known to be owned at compile time) and deref_owned::IntoOwned (similar to ToOwned but able to consume a reference-like type (shared reference or smart-pointer) instead of only working on ordinary shared references) could/should be made part of std in order to avoid unnecessary clones or other run-time overhead.
Nobody liked the idea. (But @CAD97 gave some very constructive input, and another user did something similar to avoid runtime overhead.)
I believe that IntoOwned is a generalization of ToOwned as can be seen here: Playground. (But perhaps could not fully serve as a replacement unless issue #20671 was solved.)
A use case for Owned can be seen here: Playground, and a real-world example is mmtkvdb::storable::Storable::AlignedRef. (Note there is a tiny error in the documentation comment yet, as Vec does not implement IntoOwned anymore, reason given here.)
I thought your Owned seemed useful, especially having seen Optional reference in generics first. (I don't know about your traits, though; I don't think I understand their use-cases.)
I don't think you should call it a "smart pointer", though. I understand that it fits your definition of "smart pointer", but to me (among others) a mere newtype (unlike, say, Rc<T>) is not a pointer[1], and I don't see that the type is "smart" either (it seems too passive or POD-ish to be called "smart"). I very much can't guarantee that your idea would be received better if you'd not call it a smart pointer, but maybe it's possible.
I saw your "smart pointer" link, but I don't think that reference section supports your definition of "smart pointer". ↩︎
Note that with deref_owned version 0.4.0 this is obsolete (see Changelog of 0.4.0), as String doesn't implement IntoOwned anymore.
I think that doesn't always work. That's why I need OwnedRef (instead of Owned) sometimes. (But not sure.)
This is also obsolete with deref_owned version 0.4.0. I'm not fine to not remove the Box. I don't feel good at all anymore to implement IntoOwned for Box<T>, so it's removed in version 0.4.0. If we had specialization, there could be an implementation of IntoOwned for any T or T: 'static (but not sure about that either and whether it would be semantically correct).
I wonder why ToOwned::Owned has a bound Owned: Borrow<Self> then? Or is it something else? I need to re-read what you wrote later as I don't understand (yet) what "consuming by generic users" means.
On IRLO, there was some reservation in regard to the idea of making IntoOwned: Deref. I'd like to give a short explanation why I think it should be:
IntoOwned is not a generic conversion from a type into an owned type. IntoOwned should be more understood as RefIntoOwned or RefWhichManagesOwnershipIntoOwned. These reference-like types are:
&'a T (always borrowed)
Cow<'a, T> (sometimes borrowed, sometimes owned, determined at run-time)
Owned<T> (always owned)
IntoOwned can still be used to go from some non-owned type like str to String. This is because IntoOwned is implemented for &str:
use std::borrow::{Borrow, Cow};
use std::ops::Deref;
pub trait IntoOwned: Sized + Deref {
type Owned: Borrow<<Self as Deref>::Target>;
fn into_owned(self) -> Self::Owned;
}
impl<'a, T> IntoOwned for &'a T
where
T: ?Sized + ToOwned,
{
type Owned = <T as ToOwned>::Owned;
fn into_owned(self) -> Self::Owned {
self.to_owned()
}
}
fn main() {
let hello: &str = "Hello World!";
let owned: String = hello.into_owned();
println!("{owned}");
let cow_borrowed: Cow<'static, str> = Cow::Borrowed("I'm a cow.");
let my_own_cow: String = cow_borrowed.into_owned();
println!("{my_own_cow}");
}
Oooops… in the above Playground I accidentally used Cow::into_owned, which is also a method of the enum (not a trait method) having the same name and doing the same in that case. But it works as trait method of IntoOwned as well, see fixed Playground.
For the other readers of this thread: Yeah, that's what I was pointing out here:
One reason why call it smart pointer can be found here:
Hmmmm, where would such a bound be used? For the GAT, e.g. return type of mmtkvdb::Txn::get? Not sure if I fully understand it or whether it's also a bit ugly.
Meta: I feel like we're discussing now more about how Rust could be extended rather than helping me with a particular problem. (edit: or is MethodDispatchAs<T> something that can be implemented in today's Rust?) Thus I felt like IRLO was not a bad place to talk about these issues. I'm interested in how Rust could be improved to make these things better, not just how I can write a crate with today's Rust. Of course, I'm also not experienced enough yet to understand everything, so when writing on IRLO this also helps me understand things better, i.e. it's some sort of "support" I get. Anyway, I feel like both URLO and IRLO are not quite fitting for my purpose. On IRLO I feel like a "dumb user" who just doesn't understand the language, and on URLO I feel like questioning the language concepts itself instead of using the language (as it is). What does the pinned post on IRLO say?
Here is the hub for discussing everything related to the implementation and design of the Rust programming language .
This is not a support forum. Questions or requests for support using Rust should be directed to the user forum. This forum is for people contributing to the Rust compiler and standard toolchain, or otherwise working on changes to the language and its implementation.
Did my post on IRLO help me to get support? It wasn't my main intention, but it did help me to get support also. So the latter would be a reason to not post there. Did I want to discuss the implemenation and design of the Rust programming language? Yes, that would be a reason to post there. Am I contributing to the Rust compiler, standard toolchain, or am I otherwise working on changes to the language and its implementation? Not really, though I do occasionally write bug reports and you could see my posts as contribution. Perhaps the quality isn't well enough to make me eligible to post them on IRLO? Either way, I felt like my input wasn't really appreciated by most people (with some exceptions of course). I didn't feel fully comfortable.
Perhaps it would have been easier if there was just one forum instead of two. Also shifting this discussion from IRLO to URLO has been technically tedious. Perhaps I should just go to URLO next time even if I want to discuss language features and never post on IRLO unless I'm 100% sure about a topic?
Anyway, now we're here on URLO, and I think we can stay here. Unless someone tells me to go to IRLO with this!!!!1!!
The GAT determines at compile-time whether I return a borrowed or an owned value (or maybe both, depending on actual memory addresses encountered, though I didn't implement optional realignment, and it might come with more overhead than it saves). Thus the return type would either be &'a B, Owned<O>, or Cow<'a, B>. That's what the IntoOwned bound is about:
There's still the curious case of OwnedRef, and I have the feeling that this might bring some things to surface that aren't well thought out yet.
Anyway, the generic AlignedRef<'a>: Deref<Target = B> + IntoOwned is an generalization of all of:
&'a B (always borrowed)
Cow<'a, B> (sometimes borrowed, sometimes owned)
Owned<O> (always owned)
(Edit: Note that all three types are reference-like types, even Owned despite its name. Owned<O> isn't the "owned" type, but O is.)
Does that make sense? I need Deref to go to B where needed (and to have deref ergonomics, otherwise I could use Borrow), and I need IntoOwned to go to O without doing an extra (useless) clone.
I will look into what that "yoke" is. Maybe it's something I could need.
Providing fn(&self) -> &T with the guarantee that &Self and &T behave “the same.” ↩︎
I saw your "smart pointer" link, but I don't think that reference section supports your definition of "smart pointer".) ↩︎
To be completely clear: it feels like the deref_owned crate's API is the result of not a principled abstraction but more "find what compiles."
For what you're using IntoOwned for then, perhaps it would be better described as StaticCow. Since the actual thing you seem to be generalizing over is -> T, -> &T, and (maybe?) -> Cow<'_, T>.
Thus I have one basic question I'd like you to answer for me with two parts:
What does it mean for an API to take some inputT: IntoOwned?
Corollary: how do you call such an API?
Which APIs do you have which take some inputT: IntoOwned?
Corollary: how do you use these APIs?
For this, we care nothing about the implementers. We care solely about what IntoOwned means to the user of an API. Note also that an associated type bound on IntoOwned communicates nothing to a user of the API unless the user themselves is generic over the trait with the associated type.
As it is, it feels like IntoOwned doesn't exist for the right reasons. It feels like it exists not because it's a useful bit of vocabulary of what you require from an input type, but rather just to define what you provide when returning a value read from the DB.
Searching through mmtkvdb a bit, the one place I found that satisfies the use of IntoOwned as an input is with Txn, that given (removing other methods, buying some vowels, renaming per my understanding, etc)
If not, the Txn trait is pure incidental complexity. TxnRo and TxnRw share the Txn API, but this doesn't mean that it has to be a trait. For example, std HashMap and BTreeMap share most of their API, which could be a trait Map. But this isn't provided, because writing code generic over HashMap and BTreeMap generally isn't necessary or useful. If nobody is expected to write code generic over Txn, it shouldn't be a public part of the API.
Is saving callers a single [in]to_owned() worth the additional complexity required to provide get_owned? Note that when I provide some known Value type, I also know its associated types, so you don't need to provide the bound. And if I'm generic over some Value type, I can add the bound myself.
Without fully digging into trying to redesign your API and seeing where things go wrong, I don't and can't know what parts are incidental complexity and what parts are inherently required by something you're doing. But just generally from what I've seen skimming just now, the first angles of attack I'd be taking in trying to simplify the API are:
Remove traits and trait bounds which exist to provide functionality.
Concrete callers know concrete types. The bounds aren't helping them.
Generic callers can add require bounds themselves if they require functionality.
To that end, I'd investigate splitting Storable into two, a DbPut trait for the required functionality for inserting a type into the DB, and a DbGet trait for the required functionality for recovering a type from the DB.
To that end, investigate replacing Value::AlignedRef<'_> with Realigned<'_, Value>. It'll probably still be implemented with something like the AlignedRef<'_> associated type under the covers, but
It's easier to instead use a emulated-GAT style Value: for<'a> Realign[1].
The Realigned type provides a perfect place to lift a Borrow requirement into a Deref provider.
The same conversion for Value::BytesRef<'_> into Packed<'_, Value> as well.
I can't not draft this: trait Realign<'a> { type Aligned: Borrow<Self>; fn realign(bytes: &'a [u8]) -> Self::Aligned; fn take(this: Aligned) -> Self; }↩︎
First of all, thank you very much for your in-depth response and for looking into mmtkvdb. Despite my interest in the language design, I still feel like being quite new to Rust, and it's still sometimes difficult to choose the right concept for an API or implementation. Advice from more advanced users is vital for me to improve my skills (and code). So thanks!
I also agree that going through mmtkvdb in this detail certainly is not something for IRLO. For me it's just difficult to keep those two issues apart:
Proposing adding somthing like Owned or IntoOwned to std (note that this is not a suggestion I would propose to realize immediately, just some ideas to discuss at this stage)
Referring to mmtkvdb's API.
The reason why I brought up mmtkvdb in the other forum is that people asked for a use case. And so far we only have two use cases: mmtkvdb and this one from user "conradludgate" in the other thread. And we don't even know yet if these two use cases are semantically the same, i.e. would justify the same API or naming convention.
But I think it's good to take a step back, forget about std for now, and talk about mmtkvdb instead.
Hmmmm… Before going through each issue, let me say that I feel a bit unhappy with BorrowStorable and StorableRef. These two were introduced later (in versions 0.3.0 and 0.3.1) to overcome some issues with DSTs (see Nomicon on Dynamically Sized Types). I would not be surprised if they could be substituted with something easier. I tried multiple times to get rid of them, but failed so far.
Moreover, I don't distinguish between types that can be used as keys and types that can be used as values. For example, it would be fine to use an f32 as value in LMDB, but we can't (natually) use an f32 as key, because it doesn't implement Ord. So that's another thing that deserves being improved in the future.
I assume you mean IntoOwned should/could be named StaticCow, and not that Owned should be named StaticCow? I always thought of Owned as a StaticCow in terms of functionality, but I think I see your point now:
What do I generalize over? I generalize over a pointer-like type that can be converted into a particular owned value (which supports borrowing the original pointee). This "converting into a particular owned value when needed" is exactly what Cow is about!
But is StaticCow the right name for it? I don't think so. I would rather name it Cow.
So that would lead to the following terminology. (Sorry to talk about how types in Rust could be renamed, this is just a thought-experiment for understanding the issue. Of course I do not propose to rename std::borrow::Cow):
Thus, switching back to the non-renamed terminology, the generalization of
&'a B
the "old" std::borrow::Cow<'a, B>
Owned<O> (where O: Borrow<B>)
is
(IntoOwned + Deref<Target = B>)
Note that:
(IntoOwned + Deref<Target = B>) has one type argument B for the borrowed type (which is then used for the associated type Deref::Target), and no type argument for the owned type O (which is determined through the assoiated type IntoOwned::Owned and not to be chosen).
All of &'a B, std::borrow::Cow<'a, B>, and Owned<O> implement Deref.
That is because all three (1. reference; 2. dynamically owned or borrowed, 3. always borrowed) refer to (i.e. point to) to a value of type B. The detail on how the underlying data is managed (1. referring to an external value, 2. sometimes having an owned value, 3. always having an owned value) is hidden behind the trait (IntoOwned + Deref<Target = B>).
Let's look at the difference between Deref<Target = B> and (IntoOwned + Deref<Target = B>).
Deref<Target = B> just tells us we have a pointer to a value of type B.
(IntoOwned + Deref<Target = B>) means we have a pointer to a value of type B but additionally a way to perform a conversion from this pointer-like type(!)[1] into an owned type
Let me get back once more to what you wrote:
I would instead say:
IntoOwned could be renamed to GenericCow.
std::borrow::Cow could be understood as DynOwned or DynCow.
Owned is named properly.
The generalization of the old Cow<'a, B> would then be (Deref<Target = B> + GenericCow).
With that findings, let me try to answer your questions:
It means it takes a reference (with an unknown target type) that can be converted into the associated owned type. This usually isn't much useful, you'd most likely use it in combination with Deref<Target = B> to specify the pointee type.
Then P: Deref<Target = B> + IntoOwned is a generalization of Cow<'a, B>.
The best name would likely be impl Cow. But it's taken by std, so we could name it GenericCow.
Then the usual use case would be impl Deref<Target = B> + GenericCow, which is a generalization of
I use Txn::get when I want to retrieve a value (or key) from the database and only need a reference to it. I use .into_owned() when I want to convert the reference into an owned value (e.g. for keeping it longer than the transaction is open, or to be able to modify it).
Note in this context that IntoOwned (or GenericCow) by itself is pretty useless because it doesn't specify which type we point to. We will always need it in conjuction with some B and the bound Deref<Target = B>.
I don't agree. I think (Deref<Target = B> + IntoOwned) is a generalization (and optimization) of Cow<'a, B>.
No, and I even considered making Txn sealed.
Not yet, but there will be (project is not public yet). This is for code to accept a read-only transaction but to be generic such that also read-write transactions may be passed.
It has to be a trait because functions may want to get a reference to a transaction which they only use for reading (but might not care whether it's a TxnRo or TxnRw).
Maybe Txn could be named TxnRead.
It's a different case, I think. Those are two different data structures. Txn, in contrast, (which could be named TxnRead) provides an abstraction for the read-part of the database transaction (whether it's a read-only TxnRo or a read-write TxnRw transaction).
Side note: I hate how I have to import the trait in order to be able to use it. Maybe sealed traits should/could be automatically imported when you import a module?
Yes, I was thinking about this (as said in my introduction earlier):
[…], let me say that I feel a bit unhappy with BorrowStorable and StorableRef. […] I tried multiple times to get rid of them, but failed so far.
I felt like removing them makes things even more complex, but I'll try again, and I like the idea to name them DbPut and DbGet (or something like that).
What are Realign or Packed? A trait? A type? I don't understand yet. (And please reconsider after reading my other input above whether it still makes sense.)
P.S.: I still think that a generalization of Cow deserves to be in std (some day).
Update: I still think there is still one misuse of Deref in case of Owned (where Deref can be replaced with Borrow), but I'm working on elaborating that. This will not change anything about IntoOwned requiring Deref as supertrait. I believe after fixing that, I can get rid of OwnedRef, though, and Owned becomes a bit more complex.
No, I think it's all fine. I thought about adding the borrowed type B as an extra type argument to Owned, like this:
/// Smart pointer to owned inner value
#[derive(Clone, Default, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct Owned<O, B> {
owned: O,
borrowed: PhantomData<&'static mut B>,
}
Such that you can specify the "borrowed" type B which you actually point to. But that doesn't make sense because it can only point to one particular type, namely Deref::Target.
(We could do this if we use .borrow() instead of dereferencing, of course. But I don't think that's semantically correct or a wise choice.)
Owned is a pointer type, I believe, but arguably not very smart:
It also doesn't fit into the other categories.
I think my idea may have used the wrong terminology in other matters (see previous post). Maybe if I had IntoOwned instead named GenericCow, this would have been more clear for everyone (including me).
But regarding whether Owned is a pointer-like type, I would definitly say yes (but it's arguable whether it's "smart", of course). I also believe that Deref needs to be a supertrait of IntoOwned (which could/should/might be renamed into PointerIntoOwned or GenericCow).
Update:
IntoOwned is the right terminology if we see it as a replacement for or as a generalization of ToOwned, because:
Or more generically phrased: IntoOwned is implemented for &'a T where T: ?Sized + ToOwned (or, if ToOwned didn't exist, would have to be implemented for &T where we can go from &T to an owned type).
impl<'a, T, O> IntoOwned for Owned<'a, T, O>
where
T: ?Sized,
O: Borrow<T>,
{
type Owned = O;
fn into_owned(self) -> Self::Owned {
self.owned
}
}
mod tests {
use super::*;
use std::borrow::Cow;
#[test]
fn test_owned() {
let smart = Owned::new("Alpha".to_string());
let reference: &String = &*smart;
assert_eq!(reference, &"Alpha".to_string());
let owned: String = smart.into_owned();
assert_eq!(owned, "Alpha".to_string());
}
#[test]
fn test_cow() {
let cow_owned: Cow<'static, str> = Cow::Owned("Bravo".to_string());
assert_eq!(IntoOwned::into_owned(cow_owned), "Bravo".to_string());
let cow_borrowed: Cow<'static, str> = Cow::Borrowed("Charlie");
assert_eq!(IntoOwned::into_owned(cow_borrowed), "Charlie".to_string());
}
#[test]
fn test_ref() {
let reference: &str = "Delta";
assert_eq!(IntoOwned::into_owned(reference), "Delta".to_string());
}
#[test]
fn test_slice() {
let owned_ref: Owned<[i32], Vec<i32>> = Owned::new(vec![1, 2, 3]);
assert_eq!(&*owned_ref, &[1, 2, 3] as &[i32]);
assert_eq!(IntoOwned::into_owned(owned_ref), vec![1, 2, 3]);
}
#[test]
fn test_generic_fn() {
fn generic_fn(arg: impl IntoOwned<Owned = String> + Borrow<str>) {
let borrowed: &str = arg.borrow();
assert_eq!(borrowed, "Echo");
let owned: String = arg.into_owned();
assert_eq!(owned, "Echo".to_string());
}
generic_fn(Owned::<str, _>::new("Echo".to_string()));
generic_fn(Cow::Owned("Echo".to_string()));
generic_fn(Cow::Borrowed("Echo"));
generic_fn("Echo");
}
}
P.S.: Maybe BorrowMut(edit: and DerefMut) and the mutable reference in the PhantomData could be removed (and be replaced with a shared reference) such that Owned::new can be a const fn(edit: and Owned becomes covariant over its target type). But not sure about these details yet.
P.P.S.:@CAD97: Note how we have to specify the str type now (using turbofish notation) in this line:
Does this resolve your feeling of Deref being used wrongly?
Update:
I think the above proposal can be simplified to:
/// Smart pointer to owned inner value
// NOTE: derive doesn't work here, needs manual implementation
// #[derive(Clone, Default, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct Owned<'a, B>
where
B: ToOwned + ?Sized,
{
borrowed: PhantomData<&'a B>,
owned: <B as ToOwned>::Owned,
}
impl<'a, B> Owned<'a, B>
where
B: ToOwned + ?Sized,
{
pub const fn new(owned: <B as ToOwned>::Owned) -> Self {
Owned {
borrowed: PhantomData,
owned,
}
}
}
impl<'a, B> Deref for Owned<'a, B>
where
B: ToOwned + ?Sized,
{
type Target = B;
fn deref(&self) -> &Self::Target {
self.owned.borrow()
}
}
impl<'a, B> Borrow<B> for Owned<'a, B>
where
B: ToOwned + ?Sized,
{
fn borrow(&self) -> &B {
self.owned.borrow()
}
}
/// Pointer types that can be converted into an owned type
pub trait IntoOwned: Sized + Deref {
/// The type the pointer can be converted into
type Owned: Borrow<<Self as Deref>::Target>;
/// Convert into owned type
fn into_owned(self) -> Self::Owned;
}
impl<'a, B> IntoOwned for &'a B
where
B: ToOwned + ?Sized,
{
type Owned = <B as ToOwned>::Owned;
fn into_owned(self) -> Self::Owned {
self.to_owned()
}
}
impl<'a, B> IntoOwned for Cow<'a, B>
where
B: ToOwned + ?Sized,
{
type Owned = <B as ToOwned>::Owned;
fn into_owned(self) -> <Self as IntoOwned>::Owned {
Cow::into_owned(self)
}
}
impl<'a, B> IntoOwned for Owned<'a, B>
where
B: ToOwned + ?Sized,
{
type Owned = <B as ToOwned>::Owned;
fn into_owned(self) -> Self::Owned {
self.owned
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::borrow::Cow;
#[test]
fn test_owned() {
let smart: Owned<str> = Owned::new("Alpha".to_string());
let reference: &str = &*smart;
assert_eq!(reference, "Alpha");
let owned: String = smart.into_owned();
assert_eq!(owned, "Alpha".to_string());
}
#[test]
fn test_cow() {
let cow_owned: Cow<'static, str> = Cow::Owned("Bravo".to_string());
assert_eq!(IntoOwned::into_owned(cow_owned), "Bravo".to_string());
let cow_borrowed: Cow<'static, str> = Cow::Borrowed("Charlie");
assert_eq!(IntoOwned::into_owned(cow_borrowed), "Charlie".to_string());
}
#[test]
fn test_ref() {
let reference: &str = "Delta";
assert_eq!(IntoOwned::into_owned(reference), "Delta".to_string());
}
#[test]
fn test_slice() {
let owned_ref: Owned<[i32]> = Owned::new(vec![1, 2, 3]);
assert_eq!(&*owned_ref, &[1, 2, 3] as &[i32]);
assert_eq!(IntoOwned::into_owned(owned_ref), vec![1, 2, 3]);
}
#[test]
fn test_generic_fn() {
fn generic_fn(arg: impl IntoOwned<Owned = String> + Borrow<str>) {
let borrowed: &str = arg.borrow();
assert_eq!(borrowed, "Echo");
let owned: String = arg.into_owned();
assert_eq!(owned, "Echo".to_string());
}
generic_fn(Owned::<str>::new("Echo".to_string()));
generic_fn(Cow::Owned("Echo".to_string()));
generic_fn(Cow::Borrowed("Echo"));
generic_fn("Echo");
}
}
This removes the type argument O from Owned. Not sure if this makes the whole thing too restrictive, e.g. we couldn't pass a Box<[u8]> but only a Vec<u8> to Owned::new.
Now Owned<'a, B> is a specialization of std::borrow::Cow<'a, B> (where B in both cases is the same type).
Update #2:
I just published version 0.5.0 of deref_owned. It incorporates the findings I was writing about above. I decided to use two type arguments for Owned to be most generic (otherwise I can't have an Owned struct which contains a boxed slice of integers, for example).
This also includes renaming IntoOwned to GenericCow.
The Owned type doesn't actually need the lifetime, but otherwise its shape seems quite reasonable after this iteration. The version allowing an arbitrary owning type rather than just the canonical one is also fine, of course. (Though I do wish it could be defaulted...)
I still question whether the Deref bound on IntoOwned is a input requirement, but the current definition of “Cow<'_, B>, Cow::<'_, B>::Borrowed<'_, B>, or Cow::<'_, B>::Owned” is at least well defined. However, I still hold that the “verb trait” IntoOwned is the wrong name for this, and that this is more of a “noun trait.” Ultimately, it's a kind of emulation for “enum variant types” or maybe “enums as traits”. [Generic]Cow isn't quite the right name, either, though, because it's not copy-on-write anymore, since you can't write to a statically borrowed instantiation. It's take-to-own. I don't really know how to properly name this kind of “noun trait.”
These represent the specific needs of mmtkvdb:
struct Realigned<'_, Value> is a Value loaded from &[u8], which is either borrowed from the source slice or an owned copy if necessary to provide a properly aligned Value.
Along with the associated type defining what Realigned actually stores.
Realignedcould be understood as just an alias for the associated type, but actually provides some real additional benefits:
Functionality (such as Deref and .into_owned()) can be defined directly on Realigned, rather than being trait bounds on the associated type.
An API signature using a struct is less complex than one using a trait associated type, and can potentially benefit from implied bounds in the future.
struct Packed<'_, Value> is a view into Value where you can get the bytes &'_ [u8] (implied: suitable for realigning).
This replaces the use of Value::BytesRef.
Value: trait Pack<'_> defines the conversion &'_ Value -> Packed<'_, Value>.
Along with the associated type defining what Packed actually stores.
The benefit of making this a normal type is much the same as Realigned.
The name Pack comes from what the transformation is doing, taking the bytes that need to be stored and packing them into a slice without any alignment/padding.
This design quite likely still requires IntoOwned like trait functionality as part of its implementation, but the important part is that users truly don't have to care about that. They just use normal struct types, and the implementation complexity is encapsulated behind a single trait that to a user is just “can be used in Realigned.”
I think I need it for the PhantomData, because if I use 'static there, I would implicitly impose a 'static bound on B.
Edit: Maybe I could use a PhantomData<*const B> ?
I decided to use the version allowing an arbitrary owning type to allow something like this:
let smart: Owned<[i32], _> = Owned::new(vec![1, 2, 3].into_boxed_slice());
assert_eq!(&*smart, &[1, 2, 3] as &[i32]);
assert_eq!(
GenericCow::into_owned(smart),
vec![1, 2, 3].into_boxed_slice()
);
Note that std::borrow::Cow cannot support this because it might be a Cow::Borrowed variant, which doesn't know how to go to an arbitrary O: Borrow<B>. But Owned is never borrowed, so it's fine to have a different type O: Borrow<B> other than <B as ToOwned>::Owned as "backend".
Can you explain what you mean with "input requirement" or give an example?
I decided to rename it to GenericCow (for now), and I would like to bring up some pro-arguments:
impl Deref<Target = B> + GenericCow is a generalization of Cow<'a, B>.
Since Deref is a supertrait of GenericCow, GenericCow implies Deref, which in turn means that every GenericCow is a generalization of Cow.
But you are right, not every implementation of it is Cow. The implementation Owned<'a, B, O> is rather "unwrap on write".
(Edit: But for this case, we don't need GenericCow because we simply use the owned type O(!), unless we want to be generic, in which case we have some sort of generic Cow again. It's like a "Cow where it needs to be, otherwise a simple wrapper".)
But both Cow and Owned (as well as &'a T where T: ToOwned) can be converted into an owned value. So maybe IntoOwned is the better name?
I'll answer the issues regarding mmtkvdb in a separate post.
So it's basically a wrapper? Or what does the struct contain? Why does it need a lifetime? How can it be either borrowed from the source slice or be an owned copy (without being an enum)?
I think that GenericCow is a good name right now. In an ideal world, maybe the following would be better:
Add Owned as "smart pointer which points to a value that can be borrowed from an owned inner value"
Rename std::borrow::Cow to MaybeOwned
Replace std::borrow::ToOwned with IntoOwned (that consumes self when converting into an owned value)
But we already have Cow and ToOwned in std, plus we have issue #20671 still open, which might cause complications. (And maybe using T: ToOwned, which then would be &T: IntoOwnedfor<'a> &'a T: IntoOwned, would be less handy to write down.)
And writing .into_owned() instead of .to_owned() would be misleading/confusing when operating on references like shown here:
let hello: &str = "Hello World!";
let owned: String = hello.into_owned();
This is false. See for example Box<&T>; a type parameter does not introduce an implicit 'static bound.
It's the bit about “concrete abstractions;” you're requiring Deref not because this is required semantics to implement the functionality generic consumers need (AIUI this is Borrow, can-be-used-by-ref-like), but because all implementers provide it (and because it's convenient to have method syntax).
I'll throw together a small demo of how I see Aligned<'_, Value> working in a bit; having an example will make it easier to explain.
Tangentially related: I think in general you should probably prefer “impl Deref” instead of how you're using “smart pointer.” To that end, I have what I like as a definition of “smart pointer:”
/// A “smart pointer” that can temporarily suspend being smart.
unsafe trait SmartPointer<T> {
// Forget that you are smart.
fn into_raw_pointer(this: Self) -> *mut T;
// Remember that you are smart.
unsafe fn from_raw_pointer(this: *mut T);
}
(Perhaps the type parameter should be an associated type. Perhaps it should be a subtrait of Deref. Additionally, omits documentation on pointer validity rules.)
Note that this includes&T. I don't see that as a problem; &Tis smarter than a raw pointer, just at compile-time via borrowck rather than at runtime.
use std::marker::PhantomData;
/// Smart pointer to value that can be borrowed from owned inner value
#[derive(Clone, Default, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct Owned<B, O>
where
B: ?Sized,
{
pub borrowed: PhantomData<&'static B>,
pub owned: O,
}
Compiling playground v0.0.1 (/playground)
error[E0310]: the parameter type `B` may not live long enough
--> src/lib.rs:9:19
|
9 | pub borrowed: PhantomData<&'static B>,
| ^^^^^^^^^^^^^^^^^^^^^^^ ...so that the reference type `&'static B` does not outlive the data it points at
|
help: consider adding an explicit lifetime bound...
|
7 | B: ?Sized + 'static,
| +++++++++
For more information about this error, try `rustc --explain E0310`.
error: could not compile `playground` due to previous error
The structurally correct phantom is PhantomData<for<'a> fn(&'a O) -> &'a B>. (Perhaps even more correctly, it's a ZST function reference to <O as Borrow<B>>::borrow.) This will achieve the correct autotrait behavior (e.g. UnwindSafe being the one most likely impacted, or Send/Sync if using PhantomData<*const B>.). The version of Owned that stores <B as ToOwned>::Owned instead doesn't need a phantom at all.
You have convinced me. I think I can get rid of Deref as a supertrait for GenericCow entirely (while maintaining ergonomics).
I got rid of Deref in deref_owned::GenericCow version 0.7.0 by adding a type argument (as you originally suggested, I believe).
This doesn't break mmtkvdb, i.e. I can make some minor adjustments which don't cause problems, but still working on that.
Many many thanks!
Note that deref_owned::Owned still implements Deref, but that should be a good thing and not a bad thing.
Here is how it works now (some parts omitted):
/// Smart pointer to value that can be borrowed from owned inner value
#[derive(Clone, Default, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct Owned<B, O>
where
B: ?Sized,
{
pub borrowed: PhantomData<for<'a> fn(&'a O) -> &'a B>,
pub owned: O,
}
impl<B, O> Deref for Owned<B, O>
where
B: ?Sized,
O: Borrow<B>,
{
type Target = B;
fn deref(&self) -> &Self::Target {
self.owned.borrow()
}
}
impl<B, O> Borrow<B> for Owned<B, O>
where
B: ?Sized,
O: Borrow<B>,
{
fn borrow(&self) -> &B {
self.owned.borrow()
}
}
/// Generalization of [`Cow<'_, B>`](Cow)
pub trait GenericCow<B>: Borrow<B> + Sized
where
B: ?Sized,
{
/// Owned type
type Owned: Borrow<B>;
/// Convert into owned type
fn into_owned(self) -> Self::Owned;
}
impl<'a, B> GenericCow<B> for &'a B
where
B: ?Sized + ToOwned,
{
type Owned = <B as ToOwned>::Owned;
fn into_owned(self) -> Self::Owned {
self.to_owned()
}
}
impl<'a, B> GenericCow<B> for Cow<'a, B>
where
B: ?Sized + ToOwned,
{
type Owned = <B as ToOwned>::Owned;
fn into_owned(self) -> <Self as GenericCow<B>>::Owned {
Cow::into_owned(self)
}
}
impl<B, O> GenericCow<B> for Owned<B, O>
where
B: ?Sized,
O: Borrow<B>,
{
type Owned = O;
fn into_owned(self) -> Self::Owned {
self.owned
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::borrow::Cow;
#[test]
fn test_owned() {
let smart: Owned<str, _> = Owned::new("Alpha".to_string());
let reference: &str = &*smart;
assert_eq!(reference, "Alpha");
let owned: String = smart.into_owned();
assert_eq!(owned, "Alpha".to_string());
}
#[test]
fn test_cow() {
let cow_owned: Cow<'static, str> = Cow::Owned("Bravo".to_string());
assert_eq!(GenericCow::into_owned(cow_owned), "Bravo".to_string());
let cow_borrowed: Cow<'static, str> = Cow::Borrowed("Charlie");
assert_eq!(GenericCow::into_owned(cow_borrowed), "Charlie".to_string());
}
#[test]
fn test_ref() {
let reference: &str = "Delta";
assert_eq!(GenericCow::into_owned(reference), "Delta".to_string());
}
#[test]
fn test_vec() {
let smart: Owned<[i32], _> = Owned::new(vec![1, 2, 3]);
assert_eq!(&*smart, &[1, 2, 3] as &[i32]);
assert_eq!(GenericCow::into_owned(smart), vec![1, 2, 3]);
}
#[test]
fn test_boxed_slice() {
let smart: Owned<[i32], _> = Owned::new(vec![1, 2, 3].into_boxed_slice());
assert_eq!(&*smart, &[1, 2, 3] as &[i32]);
assert_eq!(
GenericCow::into_owned(smart),
vec![1, 2, 3].into_boxed_slice()
);
}
#[test]
fn test_generic_fn() {
fn generic_fn(arg: impl GenericCow<str, Owned = String>) {
let reference: &str = arg.borrow();
assert_eq!(reference, "Echo");
let owned: String = arg.into_owned();
assert_eq!(owned, "Echo".to_string());
}
generic_fn(Owned::<str, _>::new("Echo".to_string()));
generic_fn(Cow::Owned("Echo".to_string()));
generic_fn(Cow::Borrowed("Echo"));
generic_fn("Echo");
}
}
Regarding the title of this thread:
Smart pointer which owns its target
Now Owned<B, O> is still a smart pointer (or let's say a pointer-like type, i.e. something that implements impl Deref). But it doesn't necessarily point to (or "targets") type O but instead points to a type B.
Why is this nice? Because we can do things like this:
Thus, if we happen to have a Box<[i32]>, we can turn it into an impl GenericCow<[i32]>.
Hmmmm… but I see your other point now:
impl GenericCow<[i32]> doesn't imply that .into_owned() gives us a Vec<i32> (which we would need to work on it freely). This is only the case if B: ToOwned and <B as ToOwned>::Owned = O.
So I think I'll go this way:
/// Smart pointer to value that can be borrowed from owned inner value
pub struct Owned<B>
where
B: ToOwned + ?Sized,
{
pub borrowed: PhantomData<for<'a> fn(&'a <B as ToOwned>::Owned) -> &'a B>,
pub owned: <B as ToOwned>::Owned,
}
But I'll rest now and think about it later…
… PhantomData<for<'a> fn(&'a <B as ToOwned>::Owned) -> &'a B>
The only reason for the PhantomData was to define how B was used by Owned<B, O>, which only stored O.
Well, it wasn't so small after all. There may be usage limitations still; I have only proven it compiles, not tried it. Fully stable-compatible, and should cover all of mmtkvdb's Storage functionality except for the unsafe optimizations (which should be fairly obvious how they would be added). [playground]
code dump
#![forbid(unconditional_recursion)]
use std::{borrow::Borrow, ops::Deref, str};
// omitted: a bunch of useful delegating trait impls; trivial
// omitted: equivalent of StorableRef; potentially nontrivial
// included: fun exercises that may not be all that fun
// fun exercise: all code below is safe. determine where unsafe can (should?)
// be used for performance. required artifact: benchmarks showing improvement.
// implementation details that are only pub because of the private_in_public lint
mod details {
use std::borrow::{Borrow, Cow};
pub trait IntoOwnedBorrow<Borrowed>: Borrow<Borrowed>
where
Borrowed: ?Sized + ToOwned,
{
fn into_owned(this: Self) -> Borrowed::Owned;
}
// look ma, no struct Owned<T>
impl<T> IntoOwnedBorrow<T> for T
where
T: ToOwned<Owned = T>,
{
fn into_owned(this: Self) -> T {
this
}
}
impl<T> IntoOwnedBorrow<T> for Cow<'_, T>
where
T: ?Sized + ToOwned,
{
fn into_owned(this: Self) -> T::Owned {
this.into_owned()
}
}
impl<T> IntoOwnedBorrow<T> for &'_ T
where
T: ?Sized + ToOwned,
{
fn into_owned(this: Self) -> T::Owned {
this.to_owned()
}
}
impl<T: Clone> IntoOwnedBorrow<[T]> for Vec<T> {
fn into_owned(this: Self) -> Vec<T> {
this
}
}
impl IntoOwnedBorrow<str> for String {
fn into_owned(this: Self) -> String {
this
}
}
// This blanket impl is almost what we want, as it covers all of the owning
// types. However, <&T as ToOwned>::Owned == &T; we want <&T>::Owned == T.
// This is the reason deref_owned@0.7 still uses an Owned<T> wrapper; to
// differentiate between an owned type projection of &T => &T or &T => T.
// Can this be resolved without the Owned<T> wrapper? I think it could, but
// I'm not sure. At the very least, it would necessitate IntoOwnedBorrow not
// using an associated owning type projection, and using an input generic
// type instead, as the Borrowed => Owned relation is no longer projective.
//
// But this is really only a formality. With the public API defined here,
// IntoOwnedBorrow is entirely an implementation detail and we control all
// of the impls which require it, so we can just provide the required impls.
//
// You can apply the same logic to using the Owned<T> wrapper, though!
// The user never actually sees the chosen impl IntoOwnedBorrow type.
// I personally just think this design is a bit cleaner.
//
// impl<T> IntoOwnedBorrow<T> for T::Owned
// where
// T: ?Sized + ToOwned,
// {
// fn into_owned(this: Self) -> T::Owned {
// this
// }
// }
}
use details::IntoOwnedBorrow;
// public API types
pub trait Realign<'a>: ToOwned {
#[doc(hidden)]
type Realigned: IntoOwnedBorrow<Self>;
// NB: pseudo sealed by returning publicly unconstructable type
fn realign(bytes: &'a [u8]) -> Realigned<'a, Self>;
}
pub trait Pack<'a> {
#[doc(hidden)]
type Packed: IntoOwnedBorrow<[u8]>;
fn pack(&'a self) -> Packed<'a, Self>;
#[doc(hidden)]
/// An optimization point like `Clone::clone_from`.
fn pack_into(&'a self, bytes: &mut Vec<u8>) {
let packed = self.pack();
bytes.extend_from_slice(&packed);
}
// fun exercise: what would it take to generalize the above to pack directly
// into any io::Write byte writer? why would you? why wouldn't you?
// fun exercise: provide a size_hint method like Iterator's, and use that
// to preallocate the Vec used for pack_into'ing tuple elements. likely
// minimal benefit, as each tuple element is extend_from_slice'd already.
// fun exercise: prove that your size_hint is ExactSize + TrustedLen. use
// this to pack into &mut [u8] instead of &mut Vec<u8>. likely beneficial,
// as now packing a tuple can preallocate the exact size correctly up front.
}
pub trait ConstPackedLen {
const PACKED_LEN: usize;
}
pub struct Realigned<'a, T: ?Sized + Realign<'a>> {
inner: T::Realigned,
}
pub struct Packed<'a, T: ?Sized + Pack<'a>> {
inner: T::Packed,
}
// examaple impl Realign
impl<'a> Realign<'a> for str {
type Realigned = &'a str;
fn realign(bytes: &'a [u8]) -> Realigned<'a, Self> {
unsafe {
Realigned {
inner: str::from_utf8_unchecked(bytes),
}
}
}
}
impl<'a> Realign<'a> for u8 {
type Realigned = &'a u8;
fn realign(bytes: &'a [u8]) -> Realigned<'a, Self> {
Realigned { inner: &bytes[0] }
}
}
impl<'a> Realign<'a> for u16 {
type Realigned = u16;
fn realign(bytes: &'a [u8]) -> Realigned<'a, Self> {
Realigned {
inner: u16::from_ne_bytes(bytes.try_into().unwrap()),
}
}
}
impl<'a, A, B> Realign<'a> for (A, B)
where
// NB: Clone bounds are for (A, B): ToOwned. ToOwned<Owned = Self> are for
// constructing (A, B); otherwise we only have (A::Realigned, B::Realigned)
// or (A::Owned, B::Owned). Realigned = (A::Realigned, B::Realigned) is more
// formally what is *necessary*, but (A, B) matches mmtkvdb@0.4, but not
// reusing Borrow/ToOwned will make the API significantly more complex.
//
// In the 🦄future🦄, std might be able to utilize specialization in order
// to provide (A, B): ToOwned<Owned = (A::Owned, B::Owned)> over the more
// general (A, B): ToOwned<Owned = (A, B)> where A|B: ToOwned<Owned = A|B>.
// This general implementation comes from impl Clone: ToOwned<Owned = Self>.
//
// If/when that is the case, we can drop the ToOwned<Owned = Self> bounds.
// However, I don't think this generalization is useful in practice, as all
// impl Sized: ToOwned almost certainly are Clone / ToOwned<Owned = Self>.
//
// To realign to (A::Realigned, B::Realigned) is a much bigger undertaking.
// In that case we no longer can even use Borrow's &self -> &Borrowed; we
// need `&self -> (&A, &B)`. Way more complexity than it's worth, imho.
A: Clone + ToOwned<Owned = A> + Realign<'a> + ConstPackedLen,
B: Clone + ToOwned<Owned = B> + Realign<'a>,
{
type Realigned = (A, B);
fn realign(bytes: &'a [u8]) -> Realigned<'a, Self> {
// fun exercise: change Realign::realign to return the number of bytes
// read, allowing this impl to remove the A: ConstPackedLen bound.
let (bytes_a, bytes_b) = bytes.split_at(A::PACKED_LEN);
let a = A::realign(bytes_a);
let b = B::realign(bytes_b);
Realigned {
inner: (a.into_owned(), b.into_owned()),
}
}
}
// example impl Pack
impl<'a> Pack<'a> for str {
type Packed = &'a [u8];
fn pack(&'a self) -> Packed<'a, Self> {
Packed {
inner: self.as_bytes(),
}
}
}
// fun exercise: blanket impl for T: Pod. not actually useful, though, because
// we want to be able to pack !Pod types which aren't coherent with that impl.
impl ConstPackedLen for u8 {
const PACKED_LEN: usize = 1;
}
impl<'a> Pack<'a> for u8 {
type Packed = &'a [u8];
fn pack(&'a self) -> Packed<'a, Self> {
let bytes: &[u8; 1] = bytemuck::cast_ref(self);
Packed { inner: bytes }
}
}
impl ConstPackedLen for u16 {
const PACKED_LEN: usize = 2;
}
impl<'a> Pack<'a> for u16 {
type Packed = &'a [u8];
fn pack(&'a self) -> Packed<'a, Self> {
let bytes: &[u8; 2] = bytemuck::cast_ref(self);
Packed { inner: bytes }
}
}
impl<A, B> ConstPackedLen for (A, B)
where
A: ConstPackedLen,
B: ConstPackedLen,
{
const PACKED_LEN: usize = A::PACKED_LEN + B::PACKED_LEN;
}
impl<'a, A, B> Pack<'a> for (A, B)
where
A: Pack<'a>,
B: Pack<'a>,
{
type Packed = Vec<u8>;
fn pack(&'a self) -> Packed<'a, Self> {
let mut bytes = vec![];
self.pack_into(&mut bytes);
Packed { inner: bytes }
}
fn pack_into(&'a self, bytes: &mut Vec<u8>) {
let (a, b) = self;
a.pack_into(bytes);
b.pack_into(bytes);
}
}
// impl for Realigned
impl<'a, T: ?Sized + Realign<'a>> From<&'a [u8]> for Realigned<'a, T> {
fn from(bytes: &'a [u8]) -> Self {
T::realign(bytes)
}
}
impl<'a, T: ?Sized + Realign<'a>> Borrow<T> for Realigned<'a, T> {
fn borrow(&self) -> &T {
self.as_ref()
}
}
impl<'a, T: ?Sized + Realign<'a>> Deref for Realigned<'a, T> {
type Target = T;
fn deref(&self) -> &T {
self.as_ref()
}
}
impl<'a, T: ?Sized + Realign<'a>> AsRef<T> for Realigned<'a, T> {
fn as_ref(&self) -> &T {
self.inner.borrow()
}
}
impl<'a, T: ?Sized + Realign<'a>> Realigned<'a, T> {
pub fn into_owned(self) -> T::Owned {
IntoOwnedBorrow::into_owned(self.inner)
}
}
// impl for Packed
impl<'a, T: ?Sized + Pack<'a>> From<&'a T> for Packed<'a, T> {
fn from(this: &'a T) -> Self {
T::pack(this)
}
}
impl<'a, T: ?Sized + Pack<'a>> Borrow<[u8]> for Packed<'a, T> {
fn borrow(&self) -> &[u8] {
self.as_ref()
}
}
impl<'a, T: ?Sized + Pack<'a>> Deref for Packed<'a, T> {
type Target = [u8];
fn deref(&self) -> &[u8] {
self.as_ref()
}
}
impl<'a, T: ?Sized + Pack<'a>> AsRef<[u8]> for Packed<'a, T> {
fn as_ref(&self) -> &[u8] {
self.inner.borrow()
}
}
impl<'a, T: ?Sized + Pack<'a>> Packed<'a, T> {
pub fn into_owned(self) -> Vec<u8> {
IntoOwnedBorrow::into_owned(self.inner)
}
}
Types and comments should explain what's going on, but feel free to ask questions; this is I think the abstraction mmtkvdb is actually wanting for. It's built around a GenericCow-like implementation trait (I called it IntoOwnedBorrow), but this is fully encapsulated. The public API surface is impressively simple (pseudo rustdoc style; what you get with [-]):
(But @CAD97 gave some very constructive input, and another user did something similar to avoid runtime overhead.)
.
I thought about adding the borrowed type 
).
It's like a "
Additionally, omits documentation on pointer validity rules.)
but I see your other point now:
