Is there a better way to overload Index / IndexMut for a Rc<Refcell<_>>?


#1

I have a data structure “Cache” where one of the fields is a Rc<RefCell<[u8;512]>> type. I’d like to overload indexing for Cache so that indexing will walk down the interior array. I can’t do this b/c Index and IndexMut would need to return a &u8 and &mut u8 (respectively) and the compiler pukes b/c of the borrow() on the RefCell.

Let’s stipulate for this post that I need a Rc<RefCell<>>. Given that requirement is the following code the only real way to overload Index/IndexMut? If so, is it guaranteed safe?

use std::ops::{Index,IndexMut};
use std::rc::Rc;
use std::cell::RefCell;

struct Sector([u8;512]);
struct Cache(Rc<RefCell<Sector>>);

impl Cache {
        fn new() -> Self { 
                Cache { 0: Rc::new(RefCell::new( Sector{ 0:[0u8;512] } )) }
        }
        fn clone(&self) -> Self { Cache { 0 : self.0.clone() } }
}

impl Index<usize> for Cache {
        type Output = u8;

        // hack to get around the RefCell borrow() error
        fn index<'a>(&'a self, index : usize) -> &'a u8 {
                unsafe { &(*self.0.as_ptr()).0[index] }
        }
}

impl IndexMut<usize> for Cache {
        // hack to get around the RefCell borrow() error 
        fn index_mut<'a>(&'a mut self, index : usize) -> &'a mut u8 {
                unsafe { &mut (*self.0.as_ptr()).0[index] }
        }
}

fn main() {
        let mut f = Cache::new();
        let mut g = f.clone();
        for i in 0..10 {
                f[i] = i as u8;
        }
        // guaranteed?
        for i in 0..10 {
                g[i] = g[i] + f[i];
                println!("f[{}]={}", i,f[i]);
        }
        println!("done");
}

#2

I’m presuming that a RefCell is required because other code is mutating the data through a shared reference. If so, that implementation is guaranteed unsafe. Mutable references must never be aliased, but the Index impl is explicitly breaking that.


#3

Other code does have mutatable references. However, these mutations would only occur through IndexMut.

As far as I can tell, the &mut returned by IndexMut only lives for the duration of the [] operation and so there can be no overlap with other []'s that occur in the program and are applied to the same data. Also, the as_ptr() accesses the underlying UnsafeCell, which I’m hoping tells rustc enough so it all-bets-off in terms of aliasing.

However, I could be wrong.


#4

But you can clone a Cache, and both copies refer to the same memory:

let a = Cache::new();
let b = a.clone();

// two mutable references to the same address
let a_ref = &mut a[0];
let b_ref = &mut b[0];

Mutable references can’t alias, period. Unsafe code is responsible for ensuring that that invariant is upheld.


#5

So there is no way safe way to overload Index / IndexMut in this case?

This is the second time I’ve been smh over the type signatures of rust operators. (Deref’s prevents doing a safe volatile implementation unless you do some compiler hacks.)

There should be versions that don’t require &T’s — e.g., by return-ing T as an rval / taking a T as an rval for Copy types.


#6

Yep, there is no way to implement Index/IndexMut through a RefCell.


#7

You can implement your own RefCell, which safely check for borrows and allows indexing.


#8

I’m not seeing how you can do this — if you track borrows, you’d need to override Drop so that it releases the borrow when the reference goes out of scope of the caller. If you return &u8 for Index[Mut] this can’t occur. If you don’t return a &u8, I don’t see how to write the code.


#9

Thanks. That’s too bad.


#10

For Copy types you’d typically use a Cell rather than a RefCell - you don’t get references that way though.

One way to work with RefCell based storage is to invert the API - instead of returning a reference, take a closure that does what’s needed while the RefCell is borrowed internally. This doesn’t allow for external indexing, of course.


#11

I meant: if the type being returned is a Copy type (in this case, u8).


#12

@sfackler — actually, if I remove Rc this seems to fix the issue with allowing two mut references — at least I can’t get rustc to allow two concurrent mutable references.

I can have an immutable and a mutable at the same time, though.

        let mut f = Cache::new();
        let mut g = f.0.borrow_mut();
        println!("f0={}", f[1]); // no complaint even though we have a borrow_mut
        g.0[1] = 1;
        f[1] = 1;               // rustc error b/c already did a borrow_mut() 

The wording around UnsafeCell is confusing. I can’t tell if this violation (immutable + mutable) at the same time is blessed or not. E.g.,


#13

immutable + mutable is forbidden.


#14

Yeah, but you’re returning a reference to it so the fact it’s Copy doesn’t really help. You can certainly return a copy of the u8 (ie the value and not a reference) but that won’t work for indexing (which returns references by design). There’s been some discussion about allowing indexing that copies.


#15

Yeah, I know it’s returning a ref. That was my point. There should be an alternate Index that does not need this, at least for types returned that are copy types. I’d also like an option for IndexMut along the lines of:

fn IndexMut(&mut self, index:usize,  rval: &T)

or

      fn IndexMut(&mut self, index:usize, rval: T)

Thanks for the pointer to the discussion. Those would be nice. Unfortunately it looks like it’s currently not going anywhere.

I think I have a working solution with just RefCell; at least I don’t see how to get multiple &T and &mut T in client code.

mod cache {
        use std::cell::RefCell;
        use std::ops::{Index,IndexMut};

        pub struct Cache(RefCell<Sector>);
        struct Sector([u8;512]);

        impl Cache {
                pub fn new() -> Self { 
                        Cache { 0: RefCell::new( Sector{ 0:[0u8;512] } ) }
                }
        }
        impl Index<usize> for Cache {
                type Output = u8;

                // hack to get around the RefCell borrow() error
                fn index<'a>(&'a self, index : usize) -> &'a u8 {
                        unsafe { &(*self.0.as_ptr()).0[index] }
                }
        }
        impl IndexMut<usize> for Cache {
                // hack to get around the RefCell borrow() error
                fn index_mut<'a>(&'a mut self, index : usize) -> &'a mut u8 {
                        unsafe { &mut (*self.0.as_ptr()).0[index] }
                }
        }
}
use cache::*;
fn main() { 
        let mut f = Cache::new();

        println!("f0={}", f[1]);
        f[1] = 1;
        for i in 0..10 { f[i] = i as u8; }
        for i in 0..10 { println!("f[{}]={}", i,f[i]); }
        println!("done");
}

#16

At this point, why do you need a RefCell at all?


#17

It was just an exercise. The whole thing is useless w/o the Rc. I was just trying to see if it was possible to do at all.