Is it possible to insert a struct instance with lifetime annotation into a static array?

Hi, I am a newbie of rust. I am trying to implement a global static array (which might be accessed in multi-threads) and insert some struct instances into it. By iterating the array, I can call the functions of these instances to do something. But I met an issue that the the struct instances with lifetime annotation cannot be inserted. The sample codes and error info are below. Could you please provide some suggestions? Thanks in advance!

The workable sample code is here: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=beac9af22e8ddd9f91751ae1821841d9

The failed sample code is here: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=9380d4308487cd7dde7192dafd84f9f5

The error info is below:

   Compiling playground v0.0.1 (/playground)
error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
  --> src/main.rs:62:29
   |
62 |             cb_fn: Box::new(self),
   |                             ^^^^
   |
note: first, the lifetime cannot outlive the lifetime 'a as defined on the impl at 51:6...
  --> src/main.rs:51:6
   |
51 | impl<'a> InstTest<'a> {
   |      ^^
   = note: ...so that the expression is assignable:
           expected InstTest<'_>
              found InstTest<'a>
   = note: but, the lifetime must be valid for the static lifetime...
   = note: ...so that the expression is assignable:
           expected std::boxed::Box<(dyn CallbackFn + std::marker::Send + 'static)>
              found std::boxed::Box<dyn CallbackFn + std::marker::Send>

The answer to your title question is "no": a 'static array can only contain 'static instances.
You can "upgrade" structs 'static using mem::forget, but 'forget' is usually s strong indicator that your design is not "Rust-friendly", and you should try other approaches.

Since the array MUST be homogeneous (arrays store lots of copies of "the same" thing), you cannot express that each item has an individual lifetime at compile time. You'll have to do some runtime checks.

This Stack overflow question discusses how to translate listener/observer patterns to Rust, using RC and Weak references. That is roughly what GC languages do automatically for you, under the hood.

Your example uses the word "callbackFn" a lot. In general, callbacks translate badly to rust, because they require a lot of connections/references,but no object clearly "owns" the other.
In garbage-collected languages, "the runtime" is the effective owner, and figures out when the value can be gone. in Rust, you must decide this yourself (but you don't need a big expensive runtime! It's always a trade off!)

In general though: what are you trying to do that requires arbitrary threads to access arbitrary structs with arbitrary lifetimes?
It sounds like there is a lot of room for improvement, and more "rustic" solutions, if we can narrow down what you are trying to do.

3 Likes

Thank you very much for the info!

The scenario is below:

  1. There are some threads in which some struct instances exist.
  2. One thread need to access these struct instances to get values of some fields or set values into those fields.

So, I need a mechanism to make 1st step instances register themselves (the trait instance) into somewhere (the static array in my codes). Then, the 2nd step can iterate the array to access the fields of these registered instances.

Unless you store your data in Thread Local Storage (TLS), then you are thinking about this wrong, and it is making your design inefficient.
If you store your data in an Arc<Mutex<>>, it is accessible to all threads equally, it doesn't exist "only in some threads".

WHY does this thread need to do this? There are probably other, better solutions, but we can't identify them with the very generic description you have given.
If it is only ONE thread that needs to do this, then the better solution is to give all data to that thread.

If each thread only needs one (or a few) structs, then it is better to split the data before you create the threads, and give each thread its own segment to work with.

If all your threads must access all your data, then probably your fundamental design is wrong.
The general advice here is quoted as "don't communicate by sharing memory, share memory by communicating"
You might want to learn about channels in Rust , which offer very different solutions to multithreading problems. Channel-based solutions (as made popular by Go-lang), work far better in Rust than shared mutable state.
(Sharing memory is very difficult in general, because there are lots of ways to do it badly.)

Recommended reading on Channels:

https://doc.rust-lang.org/book/ch16-02-message-passing.html

Thanks a lot for the suggestions!

My scenarios is that some threads with some instances are created at first. Then, the last thread is created and needs to access these instances to get/modify fields of them.

I will investigate your suggestions and try to do some improvements.

Aha! would you say that the last thread is a "controller thread" that manages/directs the others?
Or is it more a "receiver" that needs some work done by the others.

Do I understand correctly that the
If this last thread is the only one that needs to talk to the "first some"? In that case it might really pay off to work with channels and "message passing", as it is called.

As for your answers: you keep answering "how" you are doing it, the code implementation you have.
I want to know "why", the use-case you want to solve.
"Why" answers are like "I need to load balance Webserver requests" or "I am crawling the filesystem in parallel, but need to alphabetically sort the output", or "I have an expensive engineering simulation, some parts of which I can parallelize, but the final part needs all intermediate results".

If we know the user problem you are solving, we can suggest better-fitting alternatives for the code problem you run into.
This is also known as the "XY problem"

3 Likes

Thanks a lot for your guidance and patience and sorry for the confused description.

You are right. The last thread is the "controller" thread. It starts after other threads and need to access the data of the instances existing in other threads. So, there must be a place to store these instances and this place can be accessed by "controller" thread. Because of some limitations, I cannot provide more details. Sorry for that.

In that case, a channel based solution is definitely going to be easier for you.
The kind of design you are currently trying is the only way to do it in C/C++, but is incredibly tricky to get perfect.

If you compare it to an office with humans, the shared-data-structs is like having a big table in the center, where the boss writes something on a random piece of paper, and then hopes the workers notice it. ("communicate by sharing")
Meanwhile, the workers are frantically reading all the papers repeatedly, hoping they didn't miss an update from the boss. (and also getting into fights if they want to read the same piece of paper at the same time)

With channels, you instead get an orderly system: The boss writes a note "please do X for me", and gives it to a selected employee. The employee (who now has his own desk, no sharing required!) handles the request, and writes a note back to the boss: "done! I have attached your answer to this note". ("share by communicating")

In Rust, Channels and message passing is the correct solution, that will not explode your brain (or cause you to fight endlessly with the borrowchecker, which hates shared tables in the middle, and wants every employee to have his/her own desk).

You'll need two types of channels:

  • one that goes Worker-to-Controller (the "feedback" channel)
  • one for each worker that goes Controller-to-Worker (the "order giving" channel)

Channels are strongly typed, so we will need an enum that defines the possible Message.
each message can contain different sorts of data, which we will use
(rought pseudo code only, sorry.. You could have two enums, one for boss-to-worker, and another for worker-to-boss, but that's just an optimisation)

enum Messages {
  // infrastructure part that we need for a working channel solution
 WorkerToBoss_HiIamNew_HereIsMyInbox( Sender-of-Worker),
  BossToWorker_YouAreDone_Shutdown(),

  // messages that replace the "raw" field-accesses you are doing right now
  // you will likely need multiple sets of these, one for each different type of order the boss can give
  BossToWorker_PleaseGiveMeDataOf(description-of-what-you-want),
  WorkerToBoss_AnswerData(the-data-you-want),
  BossToWorker_PleaseSetDataTo(description-of-what-to-set, new-value),
}

This enum of messages is where the "message passing" name comes from.

We will use "mpsc" channels ("multiple producer, single consumer") from Rust std library (you can also use the excellent crossbeam lib, which is almost holy for multithreading in the rust world)

The steps are then as follows:

  1. Main Thread: create "feedback" channel; you get a Sender and a Receiver back.
    the Receiver is unique ("single consumer"), the Sender can be .cloned infinitely ("multiple producer").
  2. create the "Controller" thread. This thread gets the feedback-Receiver (move it in)
    the main thread keeps the Sender
  3. main thread creates the workers. For each worker:
    1. give the worker a clone of the Controller-Sender (so they can reply to their boss)
    2. the first action each worker does is create a new Channel (their personal "order-channel".
      they keep their receiver (so they can hear what the boss wants).
      their second action is to introduce themselves to the boss (WorkerToBoss_HereIsMyInbox) with a message containing their personal sender)
    3. after that, the worker loops 'forever' doing match on its order-channel receiver.
      - if it gets a "tell me something" message, it sends the answer as a message on its feedback-sender.
      - if it gets a "set data to X" message, it sets it (maybe with a feedback message "ok done", if you need it.
      - if it receives the "you are done" message, it breaks its forever-loop, cleans up, and exits. (controlled shutdown)
  4. the Controller-thread has a inboxes: Vec<Sender-of-Worker>: this is your thread pool.
    it then listens on its feedback-receiver, and matches on the incoming stream of messages. You unpack each of the "here is my inbox" messages, and you add the order-Sender to the inboxes list.
    Tricky part: You must know when to stop listening (e.g. after receiving 10 here-is-my-inbox-messages, if you create 10 workers)
    5.Our infrastructure setup is done. Now comes your actual work. the Controller thread does whatever controlling it wants, sending orders to workers, and listening for answers. The details here depend on what it is you are doing, but I can respect that you are not allowed to discuss specifics.

I hope this helps you get started. Searching for keywords like "crossbeam example" "message passing in rust", "multithreading with channels" or "rust share by communicating" will help.
This pattern is also very common in Go-lang, so maybe you can find some good tutorials there too. (the pattern is the same, the code-parts just have different names).

Good luck!

5 Likes

Wow! Thank you very much! :smiley: I will check crossbeam and try your suggestions.

1 Like

An update. Your suggestion is very helpful. crossbeam can solve my issue. Thank you very much!

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.