let (rects64, rectshash) = {
use base64::prelude::*;
let rectsbin = bincode::serialize(rects)?;
let rects64 = BASE64_STANDARD.encode(&rectsbin);
let rectshash = blake3::hash(&rectsbin);
(rects64, rectshash)
};
Which smells to me as wanting to be put inside a spawn_blocking, but I can not because I must borrow rects, and the lifetime is not 'static.
Perhaps it is enough/good to put a tokio::task::yield_now().await; before every operation?
Additionally, on the receiving end, I have this code:
pub fn deserialize_base64<'de, T, D>(deserializer: D) -> Result<T, D::Error>
where
T: Deserialize<'de>,
D: Deserializer<'de>,
{
use base64::prelude::*;
let s = String::deserialize(deserializer)?;
let b = BASE64_STANDARD
.decode(s)
.map_err(serde::de::Error::custom)?;
bincode::deserialize(&b).map_err(serde::de::Error::custom)
}
I can not even yield here because i can not await? What should I do? Curiously reqwest.json() also does not spawn_blocking deserialization of HTTP bodies to JSON, which seems an odd choice to me, but I doubt I know better than everyone who has worked on that code.
Adding yield_now is a reasonable option. Another thing to consider is to do it locally when the value is small, and when the value is large, you make a clone and use spawn_blocking. Even if your value is borrowed, do not forget the option to copy. You can also experiment with block_in_place, though I generally don't recommend it.
Well, in this case both rectsbin, rects64, and rectshash are owned values, so you could do the bincode::serialize on the Tokio runtime and then spawn_blocking the rest of the operations. That could be fast enough for it to not be a problem.
As for your deserialize_base64 case, you should wrap the top-level deserialize call instead of trying to do it deep inside the serde logic.
when it comes to async code, you should make a note on IO bound tasks and CPU bound tasks. your example code seems very likely to be CPU bound to me.
spawn_blocking() only really helps when the code is IO bound. for CPU bound tasks, spawn_blocking() just move the execution from the async runtime worker thread to another thread (of a thread pool).
for a multi-threaded async runtime, typically the worker threads are configured to match CPU core numbers, thus, spawning CPU bound tasks onto a thread pool makes no difference whatsoever, and it can only negatively impact your throughput because of the extra context switching.
on the other hand, for a single threaded runtime, inserting some yields and breaking the code into smaller pieces might help reduce user perceived latency ("lag" if you will) if your program is interactive or have some timing requirements, which is often the case for async.
technically, spawn_blocking is for any tasks that is not async. you can use it for IO bound tasks or CPU bound tasks, it's purpose it to not blocking the async runtime worker threads. my point is, use it only if you understand the performance characteristics, otherwise, it might not be benifitial as expected.
It can definitely make a difference. The runtime assumes that tasks will yield in a reasonable amount of time, and if that doesn't happen other tasks may be starved as a result, even if there are other threads available to poll them. spawn_blocking/block_in_place will inform the runtime that you're running a blocking operation and it will take the necessary steps to avoid starving other tasks.
IMO it's the opposite, if you don't understand the implications of blocking a task then you should prefer using spawn_blocking().
It may be that there is a maximum size limit, after all memory is not unlimited so some maximum needs to be enforced. If there are maximum sizes in your case, perhaps you can convince yourself that the time spent is reasonable and you don't need to do anything special, or perhaps at most put yield_now between the expensive parts. It depends on how big the maximums are, of course.