Best framework for complex cpu bound web api service

Hi .

I like actix but i dont think be good choice for complex and near cpu bound service

Take request struct as json and run near cpu bound algorithm and generate data and response as json.

Axum
Tower-http
Wrap
Tide
...
???

Which is lower overhead and simpler ??

There's no difference in overhead. In all cases you will be required to spawn CPU-bound work on its own thread, and sending results back to the async runtime is generally cheap.

1 Like

but this type make some question in my mind.

for io-bound scheduler how many cpu we need
and for cpu-bound scheduler how many cpu we need ??

in current version we process all in a single runtime with 15 cpu
and 30 Ram.

and also i read a article about tokio, she said for 100ms is ok not need another runtime (written by alice)

Are you referring to this?

To give a sense of scale of how much time is too much, a good rule of thumb is no more than 10 to 100 microseconds between each .await . That said, this depends on the kind of application you are writing.

Async: What is blocking?

Please be aware that this says microseconds, not milliseconds.

If you have an I/O bound server that means it is spending much more time waiting on input from somewhere than actually processing any data itself. A waiting processor is doing no work so the limit of the number of requests that can be in flight (open connections) at the same time is limited by the amount of memory available. One processor should be enough.

If your server is having to do a lot of work to answer requests, cpu-bound, then the number of processors you need running threads performing the work depends on what that work is. How long does it take to complete, 1ms, 100ms, 1s, 100s, an hour, more?

It also depends how many clients you have simultaneously and how long they are prepared to wait for their results.

This is something you will have to assess for yourself.

I'm pretty sure that if a request takes a minute to calculate an answer for then it pretty much does not matter what frame work you use, as long as it can actually spawn threads to run work that run on different cores.

2 Likes

ooooh, special thankkk.

code of this component used to much pattern matching.
today finished with 4370 line.
i dont take a benchmark yet

outer pattern maching is 12 Case
inner pattern maching is 8 Case
and processing occur here,

you think above case , take much cpu time ?!

I have no idea.

4 Likes

take benchmark,
~70ns.

need i use rayon side web framework or no ??

There's no correct answer here: All the options are reasonable and make different tradeoffs. Ultimately, you'll have to make a decision for yourself, and potentially revisit it if your first solution doesn't meet your needs.

1 Like

If one core can run that calculation in 70ns then it can run about 14 million of them in 1 second.

Of course accepting the request and sending the response will take time, likely more than 70ns.

Scheduling that calculation onto a thread will likely take even longer.

You would have to measure the time it takes a client to make a request and get a response to see how much.

The does not look like a compute bound task to me. An async run time on a single core is likely all you need.

Of course that assessment still depends on how many millions of these calculations you want to be ale to do every second.

2 Likes

70ns is less than 10us and lies well inside the range where rayon or similar is unnecessary.

3 Likes

thank you but sorry. my means was 70us

sorry my friend . i wrote wrong,
correct was 70us

No matter. ns or us. I think you are working on the edge here. The only thing to do is measure the performance of the whole system for yourself.

1 Like

It's the kind of situation where it's impossible to tell before-hand whether wrapping it with rayon is necessary. You will have to see how it performs in the real world to answer that.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.