I apologize for the topic title, not sure how to describe my situation concisely.
I have a library that takes user callbacks with a 'static bound, and I need to share a common mutable state among those callbacks. Here is an illustration (does not compile):
// comes from a 3rd-party library, cannot be changed
struct Engine;
impl Engine {
fn on_more<CALLBACK: FnMut() + 'static>(&mut self, _: CALLBACK) {}
fn on_less<CALLBACK: FnMut() + 'static>(&mut self, _: CALLBACK) {}
fn run_event_loop(&mut self) {/* invokes callbacks in reaction to external events */}
}
// the rest is my code that can be freely changed
struct State {
x: i32,
}
fn update_state(state: &mut State, delta: i32) {
state.x += delta;
}
fn main() {
let mut state = State { x: 0 };
{
let mut engine = Engine {};
// `move` is necessary because of 'static
engine.on_more(move || update_state(&mut state, 1));
engine.on_less(move || update_state(&mut state, -1));
// *** error: use of moved value: `state`
engine.run_event_loop();
}
}
Now, this "obviously" should work. (Engine is not doing any unsafe shenanigans behind the scene, it does not save the callbacks in a global variable or spawn any threads or anything). Only one callback at a time can access state (because single thread); when engine goes out of scope it will drop the references to the callbacks and references to state along with them. If I could remove 'static from the callback signature and move from the callbacks, it would compile.
It seems that I have to use Rc (to counteract the 'static lifetime on callbacks) and RefCell (to handle what looks to borrowck like concurrent mutable borrows). However, resorting to runtime checks when they will never ever trigger goes against my instincts.
Is there some way I can explain to borrowck that here is in fact no problems with ownership?
Or maybe I'm missing something or misunderstanding my problem completely?
(Engine is not doing any unsafe shenanigans behind the scene, it does not save the callbacks in a global variable or spawn any threads or anything).
The compiler could reason about threads because there's no Send or Sync constraint, but there's nothing to stop the Engine from stashing those in a global (or rather a thread-local). We would have to run whole-program analysis to prove otherwise, and the fact of not doing that would be an implicit part of that crate's API.
Even setting aside the 'static lifetime, you can't have two closures that mutably borrow the same state, unless you use shared mutability like RefCell. There's no compile-time way to prove to the compiler that these will never be concurrent, so RefCell makes that a runtime check.
Or maybe I'm missing something or misunderstanding my problem completely?
I think you do understand, but the compiler is not as smart as you.
Note that RefCell isn’t your only option for interior mutability: Both Cell and std::sync::atomic::* will also work, and neither relies on runtime checks.
Change your instincts, then. Almost all countless CVEs which you can see around and which Rust was supposed to eliminate come from failures of that mode.
The typical story is always the same: person A eliminated “needless checks”, then person B (who may very well be a person A few years later) forgers about invariants that guaranteed that they would never trigger and bam, suddenly you have a crash or vulnerability.
In rare cases where you really want to fight for the very last iota of speed you have unsafe, but because this business is usually so hard to do correctly and fragile it's better to not to reach out to it unless absolutely necessarily (and for gods sake, don't combine unsafe and “business logic”).
Basically: I want to eliminate these checks because “… two pages of convoluted reasoning omitted …” is not something you want to ever see in a “normal” Rust program.
Leave these to separate crates designed to encapsulate crazy and convoluted data structures while providing simple and safe to use APIs .
Do you mean .take() - modify - .set()?
Or in terms of my example,
fn update_state(mut state: State, delta: i32) -> State {
state.x += delta;
state
}
...
// using Option for Default which is needed by Cell::take()
let state = Rc::new(Cell::new(Some(State { x: 0 })));
...
let s = Rc::clone(&state); engine.on_more(move || s.set(Some(update_state(s.take().unwrap(), 1))));
let s = Rc::clone(&state); engine.on_less(move || s.set(Some(update_state(s.take().unwrap(), -1))));
there's nothing to stop the Engine from stashing those in a global (or rather a thread-local). We would have to run whole-program analysis to prove otherwise, and the fact of not doing that would be an implicit part of that crate's API`
Good point.
but the compiler is not as smart as you.
Nah, seems pretty smart to me. I'm just new to Rust and wasn't sure I'm not missing some options.
Sure, I'm not going to go unsafe for this. I just haven't truly grokked Rust's ownership and lifetime tools and asking in case I missed a good solution.
Cell works best with Copy types, which removes the need fir Option. If State is small enough you can derive Copy for it, but I often prefer putting the interior mutability inside the struct:
struct State {
x: Cell<i32>, // or AtomicI32
}
impl State {
fn update(&self, delta: i32) {
self.x.set(self.x.get() + delta);
}
fn new(x:i32)->Rc<State> {
Rc::new( State{
x: Cell::new( x )
})
}
fn close<F:Fn(&State)->O,O>(self: &Rc<State>, f:F)->impl Fn()->O {
let s = self.clone();
move || f(s)
}
}
...
let State = State:.new();
...
engine_on_more( state.close( |s| s.update(1)));
engine_on_less( state.close( |s| s.update(-1)));
I often prefer putting the interior mutability inside the struct
That's a though. Not sure it'll work for me in this case, but I'll keep an eye open for it as my project evolves. Maybe I can restructure my State with this in mind.
Cell works best with Copy types, which removes the need for Option.
In my real project, State is more complicated and expensive to copy. I should have noted that in my question, but then this thread might not have happened, and it was useful for me.
Thank you, I feel I better understand how to use Cell now.
No, there are no such solution. The required signatures gives Engine the right to organize nice shenanigans with these callbacks.
E.g. it may easily stash them somewhere in the global registry to use them later (as others have pointed out).
That ability is precisely what 'static represents: some object which either have no references to other objects, or one which keeps all the required objects around as long as it's alive (most ofthen that's done with the use of Rc). Some object which may exist for as long as Engine wants it to exist, basically. Including the right to keep them around after Engine itself goes out of scope.
The whole thread was, basically, discussing about how can one lie to the compiler to counter another lie (makers of the Engine lie to the compiler about how they are planning to use CALLBACK and thus you are trying to invent another lie to make compiler content) using the knowledge about what Engineactually does (and not what it promises to do according to the types).
Right. And if I wanted to fix this (and was in a position to modify Engine), I'd have to add a named lifetime to Engine (Engine<'e>) and use 'e instead of 'static on callbacks.
