Create a queue service/feature

Hey everyone,

I want a bit of a general advice how to approach one thing.

I want to create a queue feature that will based on user API key create a queue to execute requests with a delay of 2 min between it.

Right now all request executes at the same time and I want to change that.

I have an idea to create a database that will store active/done tasks and check it, but decided to check if maybe there's some better approach for that. (Maybe there's some issues like possibility to get the same data record for 2 requests that happened at the same time)

I will appreciate any guidance, thanks

Do you mean:

Option 1: A task comes in, and you want to wait 2 minutes before you start executing it?
Option 2: A task in, and you can start executing 2 minutes after the previous task started execution
Option 3: Same as Option 2, but with a separate queue per user

Also, are you using async Rust?

And what kind of user load are you expecting? 10 requests / minute? 100?

Hey, yes I'm using async.
Option 3.

More closely to a 100 at peak hours.

OK, so you want a user-based rate limit (1 request / 2 minutes / user) for your system.

If you are using a web framework, you could check if it has a rate limiting middleware. Otherwise, you could implement something yourself.

One simple solution would be to store each API key in a HashMap along with the timestamp when that key last ran a task. Then when a new request comes in, accept the request and update the timestamp if it's more than 2 mins ago, or reject the request otherwise. This would require the user to re-submit the request at a later time. This is how a lot of rate-limited APIs work.

Alternatively, you could accept the request anyway and pass a pending Job ID to the user. The user would then have to poll your API using the Job ID to fetch the result when it is ready. Internally, you could spawn a new async task for each pending job, calculate the time when it can start running (hashmap timestamp + 2 mins), and put the task to sleep until that time using Tokio's sleep_until or a similar function if you're not using Tokio. After waking up, the task should lock the hashmap and make sure 2 mins have elapsed, then update the timestamp and start running. If another task from this user has started running in the meantime, go to sleep again until new timestamp + 2 mins. This handles the case of a user submitting multiple requests at once.

2 Likes

I heavily disagree here: the job ID needs to be placed in persistent storage like a SQL database before you return the ID to the user. Otherwise, when you try to reboot your server to upgrade the code, it will forget all pending jobs, and you already promised to take care of the job at a later date.

1 Like

Good point. For the same reason, results will also need to be placed in persistent storage.

1 Like

Thanks for the answers, will try to implement something like this.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.