Help with complicated generics: a sync_channel based generic cache

help
1

I'm trying to implement a library that will allow me to "schedule" function calls that will get called in another thread where they will have access to the scheduling context and be able to schedule additional function calls.

I have a non generic functioning example in this develop branch, the tests/integration.rs shows some usage.
https://github.com/x37v/rust-seq/tree/develop

cargo test

The basic structure essentially clones channel so that you get a sender and receiver where the sender can queue objects to schedule from one thread and the receiver can get those, update its schedule queue, and execute those, with the ability to queue more from or reschedule the called functions.

In that code I have a struct called MidiCache which has a pop method which simply allocates an item I can schedule. I want to have no allocation in the schedule execution thread as I want to support running this is a "realtime" audio callback.

Which brings me to my problem, and a branch:
https://github.com/x37v/rust-seq/tree/feature/cache

In the develop branch I created a queue of pre-allocated "nodes" to use for my schedule, I use a sync_channel that I can grab nodes from in the execution thread and that I fill up in a separate, non real-time, thread. My goal is to make this operation generic so that I can create my scheduler with a variety of different cached items that are accessible in the "execution" thread. The sender side of the cache must implement a trait that fills up these sync_channels, and the receiver side of the cache must at least implement the node allocation that I had in the develop branch, but then it should also be able to implement additional "pop" methods that get other types of boxed items for scheduling in the execution thread, replacing the MidiCache::pop() that allocates.

I'm hoping for help to get this working. I feel I'm I'm just on the edge of it. cargo test

First of all, its looking like my generics aren't set up quite right, but I'm not sure how. The compiler wants the receiver trait implemented for the sender and visa versa and I'm not sure why. Also, it wants me to give my Cache a 'static lifetime, which I'm not going to want I don't think, and is probably related to my second question below.

Second, maybe I'm wrong about this but I'm wondering if I can embed the allocation of the cache send/receive into the creation of my "sequencer" so I have less to pass around and could box these objects so I could dynamically allocate a new scheduler pair down the line?

Thanks for any guidance you can provide!

BTW, I have a non-updated midi.rs module that can be ignored for now, I want to eventually flesh that out and make it another crate with its own cache trait so that I can allow users to require midi object caching and be able to schedule midi objects in their calls.

2

It's a bit difficult to really help as there's a good chunk of code that you have (and I'm not currently able to pull your code locally). Is there any chance you can provide distilled examples on the playground?

From a cursory look, you have type aliases like:

pub type SeqFn<Cache> = Box<SchedCall<Cache>>;

This is implicitly:

pub type SeqFn<Cache> = Box<SchedCall<Cache> + 'static>;

That means wherever you use SeqFn, there will be a requirement for the inner SchedCall trait object to be 'static. I can't quite tell if that's what you're alluding to but it seems relevant.

To allow picking the lifetime at use sites, you'd define it as:

pub type SeqFn<'a, Cache> = Box<SchedCall<Cache> + 'a>;

But yeah, if you can provide stripped down pieces in a playground it would help.

3

I don't have time to look at your code right now, but I wanted to say that this sounds a lot like futures-rs. Except, they haven't really nailed down the scheduling context thing. You can make it work, but it's pretty ugly (task-local storage). I would recommend at least looking through the design of futures-rs because you will probably run into some similar problems. You might even be able to build this on top of futures by implementing its Executor trait.

4

hi @vitalyd,
Yeah.. I guess my 2 branches are quite complex for this forum. I'll get a distilled example of the features I'm trying to implement together and re-post.. the note about 'static is something I'll remember though, thanks!

5

@nwoeanhinnogaehr interesting. I'll look into futures-rs. One of my key requirements is no allocation in the execution thread which I'm guessing isn't a requirement for the futures executors but I'll investigate either way. What I have right now is almost there for what I'm trying to do, I think but either way it is worth checking out.

The "context" is going to be pretty important because one of the things I'm trying to do is modulate "time" in a sense. I want to use this for a variety of things but a major thing is to create automatic musical accompaniment based on a main melody where the accompanying notes could be based off the main melody but actually come before the main melody notes in time.

Thanks for the heads up!