Strange lifetime error with custom arena and once_cell

help
1

Hi! I've been trying to unravel a lifetime issue I've been having. First, some background:
I've been trying to build a modular synthesizer using a single array for storing and passing audio. For example, all audio is a Vec<Cell<f32>>. Data is passed between nodes by taking turns accessing the array. For example, an oscillator node writes to data[0], and then a gain node gets that data by retrieving data[0]. Each node's process function is called sequentially by the traverser, ensuring that two calls never have simultainous access to the data. Hopefully this example helps explain it:

struct Node<'a> {
    inputs: &[Cell<f32>],
    outputs: &[Cell<f32>],
    node: impl Node,
}

let audio: Vec<Cell<f32>> = repeat(Cell:new(0.0)).take(2).collect();

let oscillator = Node {
    inputs: &audio[0..0], // no inputs to oscillator
    outputs: &audio[0..1], // output to first position
    node: OscillatorNode::new(),
};

let gain = Node {
    inputs: &audio[0..1], // takes in the audio from oscillator
    outputs: &audio[1..2], // output of gain
    node: GainNode::new(),
};

let nodes = vec![oscillator, gain];

oscillator.process();
gain.process();

That approach works well, but I start running into issues when I try to introduce OSC (Open Sound Control). OSC messages have dynamic length, and I'm working in a realtime situation, so I can't trust the global allocator. I've made a (probably poor) wrapper around a buddy allocator to handle message allocation.

My type for storing all OSC data is this (I have all my example code at this gist):

struct TraverserIo<'arena> {
    osc_io: Vec<UnsafeCell<Alloc<'arena, Osc>>>,
}

And Alloc is this:

pub struct Alloc<'a, T> {
    pub value: &'a mut T,
    buddy_ref: &'a BuddyArena,
    ptr: NonNull<u8>,
    layout: Layout,
}

I'm trying to construct a self-referential struct that tracks the OSC messages and arena for ease of use, and I attempted this using the once_cell crate, but I'm getting a really strange lifetime issue:

self_cell!(
    struct TraverserIoWithArena {
        owner: BuddyArena,

        #[covariant]
        dependent: TraverserIo,
    }
);

error: lifetime may not live long enough
  --> src/lib.rs:19:1
   |
19 | / self_cell!(
20 | |     struct TraverserIoWithArena {
21 | |         owner: BuddyArena,
22 | |
...  |
25 | |     }
26 | | );
   | | ^
   | | |
   | | lifetime `'y` defined here
   | |_lifetime `'x` defined here
   |   function was supposed to return data with lifetime `'x` but it is returning data with lifetime `'y`
   |
   = help: consider adding the following bound: `'y: 'x`
   = note: requirement occurs because of the type `TraverserIo<'_>`, which makes the generic argument `'_` invariant
   = note: the struct `TraverserIo<'arena>` is invariant over the parameter `'arena`
   = help: see <https://doc.rust-lang.org/nomicon/subtyping.html> for more information about variance
   = note: this error originates in the macro `$crate::_covariant_access` which comes from the expansion of the macro `self_cell` (in Nightly builds, run with -Z macro-backtrace for more info)

The error goes away when I either remove the Arena or UnsafeCell from the type, but I need those to be able to access the underlying data. Where did I mess up in my type? Should I approach this with a completely different direction?

Thank you!

2

Interior mutability causes invariance, so your TraverserIo-typed field can't be covariant (if that's what the #[covariant] annotation is trying to convey. I'm not familiar with self_cell.)

3 Likes
3

The relevant documentation can be found here.

$Dependent:ident Name of the dependent type without specified lifetime. This can’t be a nested type name. As workaround either create a type alias type Dep<'a> = Option<Vec<&'a str>>; or create a new-type struct Dep<'a>(Option<Vec<&'a str>>);. Example: Ast.

$Covariance:ident Marker declaring if $Dependent is covariant. Possible Values:

  • covariant: This generates the direct reference accessor function borrow_dependent. This is only safe to do if this compiles fn _assert_covariance<'x: 'y, 'y>(x: $Dependent<'x>) -> $Dependent<'y> {x}. Otherwise you could choose a lifetime that is too short for types with interior mutability like Cell, which can lead to UB in safe code. Which would violate the promise of this library that it is safe-to-use. If you accidentally mark a type that is not covariant as covariant, you will get a compile time error.
  • not_covariant: This generates no additional code but you can use the with_dependent function. See How to build a lazy AST with self_cell for a usage example.

(context for the identifiers:)

    …
    $Vis:vis struct $StructName:ident $(<$OwnerLifetime:lifetime>)? {
        owner: $Owner:ty,

        #[$Covariance:ident]
        dependent: $Dependent:ident,
    }
    …

(relevant type signatures:)

// Only available if dependent is covariant.
fn borrow_dependent<'a>(&'a self) -> &'a $Dependent<'a>

fn with_dependent<'outer_fn, Ret>(
    &'outer_fn self,
    func: impl for<'a> FnOnce(&'a $Owner, &'outer_fn $Dependent<'a>
) -> Ret) -> Ret
1 Like
4

Guess I should have read the docs :upside_down_face:. This is my first time hearing about covariance and invariance, would you mind checking my understanding in this situation?

If I understand correctly, the borrow checker needs me to guarantee that any value I put in Alloc lives exactly as long as BuddyArena, correct? Which also means my Alloc API is flawed, as it shouldn't directly expose the reference... How can I guarantee any object in Alloc is created by the Allocator, and not from a mem::swap? It seems it would be almost better to store an entry ID, and to interact with the actual allocation and requesting with a different API. I'll try that out and report back :slight_smile:, thank you again!