Writing a "Flatten" trait

#1

Up until recently, this was a valid way to write a “Flatten” trait, to convert things to contiguous blocks of bytes that can be sent between processes. I didn’t use rustc-serialize because I couldn’t find a way to optimize the Vec<T> where T: Copy case to a bulk memcpy.

Unfortunately, despite the fact that this code is entirely unambiguous, and worked until about a week ago, the Rust type system intentfully no longer allows this. Any ideas for how I can rewrite these traits in a way that works with the latest rustc?

#2

Random thoughts (based on https://github.com/frankmcsherry/columnar). None of these are the solution you probably want (how to make code from last week build properly again).

  1. For the limited cases I bothered to implement (I’m not touching macros yet), the above does memcpy serialization of vectors of Copy data. But it does other weird things you might not want. And it may not do them as well as you’d like, either.
  2. I got around my conflicting implementations by having another type implement the functionality (for me, the temp storage where things are staged), and then having a Columnar trait with an associated type to indicate the preferred associated implementor. It means that I more often need to scribble down an implementation rather than use type inference, because I don’t define impls that might conflict, but that implementation is usually just indicating the preferred helper type. It’s not a brilliant solution.
  3. Not helpful for your problem: Unless I misunderstand, you might be able to use &[u8] instead of your MemStream type. Rather than maintain a position in the slice, you just update (mutate) the slice to point to a sub-slice. This is how the Read impl for &[u8] works, I think. Similarly, you might be able to use generic Read and Write implementors rather than &[u8] and Vec<u8>.
#3

Ah, nice!

I notice you don’t have a impl<T:Copy> ColumnarStack<T> for T; I think that would cause the same conflicts I’m seeing. I’ll probably just go with your solution, though: manually impl for the Copy types I know I want (u32, u64, etc.)

It’s disappointing to me that Rust has removed legitimately useful, difficult-to-replace functionality, though. It’s just been replaced by the vague hope that it will come back.

Good point about the slicing, I’ll give it a shot, thanks!

#4

I haven’t investigated this deeply, but I would say that it’s better to start with a small solid base and add things later than starting off with as much as possible, and this is probably such a case. I bet (at least parts of) it will come back later, when they know how to do it in a good way. It would be much worse to realize after 1.0.0 that the rules are wrong, because they would not be able to change them in a back compatible way. The current situation is at least fixable.

#5

@frankmcsherry Turns out that manually implementing for each Copy type I want doesn’t work either, because it’s dumb for flatten to have a dependency on literally every package that provides a type that could be flattened (e.g. cgmath for Vector3).

@ogeon This isn’t starting off with as much as possible (in which case, I’d be trying to create the #[derive(Flattenable)] annotations that rustc-serialize supports). This is the minimum base used in Playform for packet serialization; rustc-serialize + JSON was an incredible bottleneck. We already knew how to do provide this conflict resolution in a good way, at least from the user’s point of view - I didn’t hit any strange edge cases using this well-defined behavior that has now been removed. It may have been gross and edge-casey internally, I don’t know, but if we’re okay with removing legitimately useful and difficult-to-replace user-facing features for the sake of internal cleanliness, why stop there?

#6

I’m not sure if you misunderstood my post (I admit that it wasn’t as direct as it could be and I may have misunderstood your previous post). You haven’t done anything wrong. I meant that the new orphan and overlap rules are the “minimum base” and you fell victim for its minimization.

#7

OH sorry, I totally misunderstood. I agree. Of course Rust isn’t even released yet, and it’s not reasonable for me to expect that every massive change will come in a nice atomic chunk. I guess part of my concern is that I haven’t really been given the overwhelming impression that this will be added back, but more that it would be nice if it were added back at some point. I feel sort of indefinitely broken right now.

#8

I understand the feeling. I don’t know if it were in the same change or something else, but the lifetime inference changed slightly and crippled one of my projects. I haven’t managed to sort it out yet and it’s really annoying when you think you have a clear image in your head, but the compiler just won’t understand…

#9

Fingers crossed for the official release! Lifetime inference feels like it’s probably more important than impl edge cases; I feel like they’re so baked in that working around them means abandoning half of what makes Rust great…

For now, I guess my workaround will be to add a Copyable struct, so then I can impl<T> Flatten for Copyable<T> where T: Copy. It adds a lot of noise, but at least it’ll run…

Thanks for the help @frankmcsherry @ogeon!