Take multiple mutable slices of a single vector

A vector, like all data in Rust can only be borrowed mutably once at a time, however, if I can guarantee that various slices of this buffer will never interfere with each other, I can guarantee that the multiple-borrow perfectly safe. However, I can't seem to find a valid way to convey this to the Rust compiler. I'm hoping for some advice on how I can do this.

Context

I'm working on a database system which stores information in pages. A page is nothing more than a description of where the data is to be found. A page's contents are scattered over the backing object, in slices of bytes I call chunks. I now need to piece these chunks together to retrieve the information from within the page.

  • A page must implement AsRef<[u8]> in order to be useful to me, therefore the page's contents must be made contiguous somehow.

  • Since the backing object is bound by the traits Read + Write + Seek (as in most cases it will be a file), I can allocate a page-sized buffer and pass various slices of the buffer to the Read::read() method in order to populate it.

The backing buffer does not implement Send nor Sync, so all read/write operations are done on a single dedicated worker thread. Pages send various slices of their buffer to this thread in order to do the read/write. My question therefore is, how can I convey to the Rust compiler, that chunks (slices) of the page buffer never interfere with each other, and sending mutable slices to the worker thread is perfectly safe?

There’s split_at_mut for this, or chunks_mut. Alternately you can make a shared slice of Cells (single threaded).

If you need something more complicated, you may need to use unsafe.

1 Like

Wow that was quick! Thanks!

It seems that I need a cross between split_at_mut and chunks_mut, as the chunks are not fixed-size.

The as_slice_of_cells method unfortunately also misses by a few points,

  1. Something which isn't clear to me from the docs, is since Cells are owning types, whether the slices are moved. If this is the case, then I cannot guarantee that the final buffer remains contiguous.
  2. the same problem of fixed-sized chunks remains.

I'm hoping to avoid unsafe for obvious reasons, I'm hoping you have a suggestion for the variable-sized slice point above?

Can't you just call split_at_mut in a loop?

I mentioned this mostly for emphasis on point 2. Of course I can do this in a loop, but this is not feasible when using as_slice_of_cells

Problem solved then?

I'm implementing this at the moment. I'll hit solved if no more questions arise :slight_smile: thank u very much for your help

The cell slice will point into the same memory as the mutable slice.

Ah perfect thank you

They can't be moved. The signature doesn't allow that. The cells to which the return value refere has to be owned somewhere too, but the lifetime means that they can only come from the argument. If thex were moved into the function, a reference to then couldn't be returned.

Why not?

Because as_slice_of_cells returns each item wrapped in a cell. Each item in this case happens to be a byte. If I have slices of any substantial length, acquiring mutable access to each byte is an absolute waste of resources. Hence I'm searching for a method to lock/restrict slices. I'd considered my buffer to be a vector of vectors, but this again breaks the contiguity guarantee

I don't really get what the problem is here. You can split the returned slice-of-cells just like you would split any other slice. If you think that accessing individual cells would be more expensive than accessing individual bytes, then that's almost certainly not the case due to optimizations.

Please note that T and Cell<T> have exactly the same memory representation, so there's no actual conversion going on. The operation is just a pointer cast that changes the type, and none of the elements in the slice are actually touched by this operation.

As an example of using Cell, see:

1 Like

@H2CO3 It's more about quantity. If I were to wrap each chunk in a Cell, I'd have less overhead than if every byte of every chunk was wrapped in a Cell.

@alice How can they have the same memrep? Doesn't the cell have to hold internal state?

But I did not know this. I'll do some more research

No, Cell has no internal state. Everything is managed safely using only the type system. Perhaps you mixed it up with RefCell, which does have internal state?

I think Cell doesn't help you in your scenario. Cells are not designed for multithreading, Cell isn't Sync, so you can't share a slice of Cells with another thread. They don't seem relevant to your problem.

Just use split_at_mut.

In that case, possibly.

@tczajka Yep. I'm currently working on this at the moment. In my example, I don't need Sync, only Send which cells are, but it's far less overhead to just use split_at_mut, so that's what I'm doing.

Thanks everyone

No, again, that is not the case, as it has been pointed out to you several times.

It could easily be far less mental overhead to loop over split_at_mut, and sometimes that's what matters.

1 Like