Thread-safe shared reference to Fn closure without holding onto read lock?

I have Fn's that need to be called from other Fn closures.

I'm currently using RwLock's in such places, but this code causes a deadlock here:

when running this test:

cargo test --test flatten -- --nocapture --test it_flattens_a_two_layer_finite_pullable_sources

This is because there are read locks (RwLockReadGuard's) that are still held due to nested function calls.

Is there a way to atomically get a shared reference to a Fn closure, without holding onto a read lock to it?

You must hold an RwLockReadGuard for the full duration of your use of the value, but why do you have an Fn in an RwLock in the first place?

1 Like

I guess that's kinda my question. What is the right thing to do here, given the above requirements?

once_cell::sync::OnceCell looks promising, but it has the limitation of only allowing a single assignment.

https://docs.rs/once_cell/1.8.0/once_cell/sync/struct.OnceCell.html

So it sounds like the answer to my question is that you want to be able to change the function?

1 Like

In certain cases, yes, it's necessary to be able to change which function is being called.

As an example in the above test case, the flatten operator would pull from the outer source once the current inner source has terminated:

which would later lead to a new inner_talkback being assigned when the outer source emits a new data item (which is actually the new inner source):

Sequence diagram for this test case (view without word wrapping):

You could perhaps use the arc-swap crate instead of an rwlock

1 Like

Thank you! ArcSwapOption works great for my use case.

I'm not sure it'll help in my case, but I've also come across GhostCell which seems very interesting:

https://plv.mpi-sws.org/rustbelt/ghostcell/

As a follow-up question:

What can I use in cases where I need a mutable reference to the held value? For example, to call next on an Iterator:

Or is that not possible without locking?

What should happen if:

  1. Multiple threads call the closure concurrently?
  2. The closure calls itself recursively?
1 Like

Yes. :laughing:

Different answers to those questions result in different solutions.

1 Like

Right... I tried a combination of ArcSwapAny::into_inner / ArcSwapAny::swap + Arc::try_unwrap but it caused a panic. Probably I'm running into one of those scenarios as you've described...

But yeah, at this point I've no idea what's the right way to handle them, so I think I'm sticking to RwLock for those.