I need to fetch data every n minutes

help
1

I have a program that would need data from an external api. I would like to update said data every n minutes. When I had to implement this in golang, I would spawn a goroutine that received a pointer to the data (an array of objects), lock the data with a mutex, update the data and unlocked the data so the program could use it. I'm not entirely sure if what I'm trying to do is feaseble in rust.
Thanks in advance

2

You can do this with Rust. You would wrap the mutex in Arc, to allow you to share it. Note that there are both std::sync::Mutex and tokio::sync::Mutex — you should use tokio::sync::Mutex if you ever need to .await while the mutex is locked.

There are also alternatives to mutexes: The arc-swap crate is very nice. There's also the tokio::sync::watch channel, though its more focused at the case where you want to wait for the value to change.

1 Like
3

As a follow-up there are two things I do not quite understand. The first one would be how would I go about spawning the task every n-minutes and the second one would be if I can use a Mutex with a Vec<Structt> or would I have to do something like Box<Arc<Vec<MyStruct>>>.

Thank you for your response!

4

Usually, the way you run something on an interval is with tokio::time::interval.

tokio::spawn(async move {
    let mut interval = tokio::time::interval(duration);
    loop {
        interval.tick().await; // This should go first.
        my_repeating_operation().await;
    }
}

The above code has the potential issue that if my_repeating_operation takes more than five minutes, then it doesn't start a new one until it finishes. You can control how it should "catch up" using this method. If you want it to start multiple of them when the operation takes too long, then you do this:

tokio::spawn(async move {
    let mut interval = Interval::new(...);
    loop {
        interval.tick().await; // This should go first.
        tokio::spawn(my_repeating_operation());
    }
}

As for how to put together the types, the thing you want is an Arc<Mutex<Vec<MyStruct>>>. The Arc lets you share its contents, but makes them immutable. The mutex makes them mutable again. And the vector is the contents.

Note that the way an Arc lets you share stuff is via the .clone() method. Cloning an Arc gives you a new "handle" to the same shared value — the value inside it is not cloned. Now, any single clone of the Arc can only be in one thread, so the way you share them is by first making a clone, and then giving the clone to the new thread/task. (You can't call clone inside the new task! It has to happen before you spawn.)

2 Likes
5

A few notes, depending on your rust experience :

Async with tokio is the equivalent to goroutines, but it generally things like this also work just as well with standard threads. The real benefit to the additional complexity of async is when you would otherwise end up with hundreds or more of threads or doing something complex enough that deadlocks are likely. It's worth learning at some point, but you can delay that getting the rest working if it becomes an issue.

As with go, channels are extremely handy. As with mutex, don't try to mix sync and async in general, and be aware there's a lot more possible implementations than in go out the standard library. For example tokio provides, as well as the unbounded and bounded mpsc channels that go has, broadcast, watch, and one shot channels that are all quite handy to express the interaction cleanly.

Public crates can be a lot more successful than in go packages (and most other languages) at providing safe low level support, especially for data structures and multithreaded programming - it's a central goal of the language design. For example your use case seems like it might be able to use Rayon — Rust concurrency library // Lib.rs or DashMap — Rust concurrency library // Lib.rs, or perhaps one of the other Concurrency — list of Rust libraries/crates // Lib.rs crates.

1 Like
6

Just to add onto what @alice already said: there's never really a reason to put an Arc into a Box :slight_smile:
An Arc already is a reference so you're only adding another layer of indirection.

7

I'm sorry to continue bothering but there is something I don´t understand.
I wrote this code

#[tokio::main]
async fn main()  {
   let code = "0";
   let mtx = Arc::new(Mutex::new(DatabaseTable::mock_new()));
   let cl1 = Arc::clone(&mtx);
    tokio::spawn( async move {
        let mut interval = tokio::time::interval(duration);
    loop {
        interval.tick().await;
        let mut data = cl1.lock().await;
        *data = apiget(code).await.expect("Error obteniendo datos");
    }
   }
);

   let datar = mtx.lock().await;
   rocket::ignite().mount("/", routes![gett, gett_qs, second_test])
   .manage(datar)
   .launch();
}

The error I get is:

Screenshot from 2023-03-14 19-33-01
Screenshot from 2023-03-14 19-33-011090×348 32.1 KB

As I understand it (very much a begginer) is that the strong count of the Arc is 2 by the time the main function ends. If that is the case, can I drop a reference manually?

8

You should probably be doing .manage(mtx) instead.

2 Likes
9

Specifically, the compiler is complaining that:

  • datar is "borrowing" (has a reference to, or into) mtx (because it's a lock for a mutex, and it needs to unlock the mutex when it's dropped),
  • and then you are passing that to manage(),
  • but that requires (as part of its type) that the argument can be kept alive at least as long as 'static, that is, for the rest of the program (think: it can only reference static data)
  • but mtx will be destroyed when main returns
  • so then datar might be alive and pointing to a destroyed mtx, which is not safe.

As a rule of thumb, try to keep borrowing short and simple: if you see lifetime errors, it generally means you're trying to keep a borrow around, rather than recreating it when you need it.

1 Like