Is it possible to fake a slice?

help
1

I'm interested in building an abstraction that represents a non-contiguous buffer of u8 bytes (my own implementation of the bytes' crate Buf trait, abstracting over multiple Bytes objects).

So for example:

struct MyBuffer {
    buffers: Vec<Vec<u8>>
}

impl MyBuffer {
    pub fn slice(&self, start: usize, end: usize) -> &[u8] {
        //....
    }

In this case a slice might overlap two inner vectors. Is it possible for me to abstract this away as one contiguous slice?

1 Like
2

There's no (good/easy) way to do that. Unless you want to go down a similar rabbit hole as slice_deque :slight_smile:.

3

Weird I assumed it was easier as that seemed to be the advertised use case for the Buf trait but I couldn't find any easy way to do it. Maybe I just misread then.

4

Right - I'm not sure how bytes(&self) is intended to be implemented if underlying storage is not contiguous, as the Buf docs do mention as intended use case.

I guess one example is http://carllerche.github.io/bytes/src/bytes/buf/chain.rs.html#154-160, which chains 2 separate bufs. It looks like the contract on bytes() is a bit weaker, which is it allows returning a value in the range [0, remaining]. I guess the idea is you mostly use Buf as a cursor, and occasionally can ask for a slice view of some length in that range.

To that end, you can follow the same approach using Vec<Vec<u8>>.

@carllerche?

5

Ah I missed that example. Interesting, so each call to bytes() would only return the slice for the currently active Vec<u8> and once we advance on I make the next Vec<u8> current and reset the remaining and offset values. Though that might make things pretty confusing for end users to see remaining go from 1 to some arbitrarily higher number after advancing.

6

remaining() has to always say how much total is remaining, which should account for your “chunked” Vecs. As the user cursors (moves) over the buffer, the remaining() changes. bytes() will return slices of varying length (within the range I mentioned however) as you jump across the Vecs.

Put another way, bytes() is a window over a contiguous segment of the buffer, not over the entire buffer. My initial post in this thread was referring to a slice over an entire buffer, but that’s not the meaning of bytes() in Buf/BufMut.

7

That seems against the documentation of the buf trait though, as it says:

Returns a slice starting at the current position and of length between 0 and Buf::remaining().

So in the case of the chain that would violate that documentation as it's not returning a slice of the full bytes, only a partial slice.

I think there are too many edge cases right now for me to go forward with this idea I had of chaining buffers together in an abstraction.

8

I think the documentation is correct. Let's take an example:

  1. Your buffer has Vec<Vec<u8>>. The outer vec contains 2 inner Vec<u8>s, each of length 100.
  2. At this point, remaining = 200 and position in your buffer is 0. bytes() would return a &[u8] of length 100, corresponding to the entirety of the first Vec<u8>.
  3. User reads (i.e. advances) your buf by 50 bytes. At this point, remaining = 150. bytes() returns a &[u8] of length 50 (i.e. balance of 1st Vec's data).
  4. User reads another 50 bytes. Now remaining is 100. bytes() returns a &[u8] of length 100 again.
  5. User reads 100 bytes. remaining is 0 and bytes() returns a zero length slice.

So at every step bytes() returns a slice whose length is >= 0 and <= remaining.

9

Oh right, my eyes were glossing over the word between. So as long as it's not a slice greater than remaining it's ok. Sorry about my bad reading comprehension skills today :smiley: