I'm upgrading my project hopper's dependencies such that serde 1.0 comes into play. The update seems smooth enough but I have concerns about the ergonomics of the approach I've taken. The PR is here:
In particular I have questions around the Deserialize trait bound I've chosen: DeserializeOwned. The documentation describes this trait bound being used when:
... the data that is being deserialized from is going to be thrown away before the function returns, so T must not be allowed to borrow from it. For example a function that accepts base64-encoded data as input, decodes it from base64, deserializes a value of type T, then throws away the result of base64 decoding. Another common use of this bound is functions that deserialize from an IO stream, such as serde_json::from_reader.
Hopper I think fits that bill. In the relevant code hopper reads an exact amount of data from disk into a temporary buffer, passing it on to bincode to be decoded. My concern is that the use of DeserializeOwned is overly strict for users of hopper. It's not clear to me that this can be any other way, unless I require that clients pass in their own storage or otherwise contrive to keep the data read up from disk in-memory. I do note that bincode uses the more relaxed Deserialize<'a>.
I guess what I'm asking is, does my use of DeserializeOwned make good sense in this context or should I work toward Deserialize<'a> to give maximal flexibility to end-users of hopper?
I don't know anything about hopper but based on this super helpful comment:
// The receiver works by regularly attempting to read a payload from its
// current log file. In the event we hit EOF without detecting that the
// file is read-only, we swing around and try again. If a Sender thread
// has a bug and is unable to mark a file its finished with as read-only
// this _will_ cause a livelock situation. If the file _is_ read-only
// this is a signal from the senders that the file is no longer being
// written to. It's safe for the Receiver to declare the log done by
// deleting it and moving on to the next file.
it sounds like hopper is not going to keep the input data around in memory (nor on the filesystem for that matter). So as you saw on the Serde website, that means values that are deserialized are required to own all of their contents.
DeserializeOwned is equivalent to the Deserialize trait from serde 0.9 so it will be just as flexible.
You're right, in its present state hopper does not keep the input data around in memory. What I wasn't sure about was if I ought to go to the trouble of ensuring that it does. But! If DeserializeOwned is equivalent to 0.9's Deserialize then there's no regression of behaviour and that's good enough for me.