Rust newbie: Need help porting Go code

help
1

Hi,

In Go goroutines makes this easier but not sure how to do this in Rust. I've been trying with Tokio and tasks but the code is not working.

I want millions of short lived tasks being spawned in the background running on few kernel threads and sending back the result to main thread. This will go on throughout the lifetime of the app.

Thanks and best regards

2

You haven't given us enough information to help you. At a minimum:

  1. Please describe, at high level, the problem you're trying to solve.
  2. Please show the code you're trying to port.
  3. Please show your attempt at using tokio.
  4. Please explain what "not working" means.
3 Likes
3

Thanks for the reply.

Basically, running millions of green threads in the background without blocking the main thread. I assume this can be done with tokio and tasks.

I'll post the code later.

Best regards

4

Here is example go code for what I'm trying to port: Sign up to continue coding - Replit

5

That's describing the problem at a low level. Describe it at high level, ideally without using the word "threads" ( or similar implementation level details).

1 Like
6

I think you can do this with futures, Tokio, and the tokio-threadpool executor.

7

I don't think that is possible, because it depends on lower level details for what I want to do. I want to fire up millions of functions that doesn't block.

Here is an attempt, check lightweight_process: Sign up to continue coding - Replit
[dependencies]
futures = "0.1.17"
tokio = "0.1.15"
tokio-threadpool = "0.1.11"

It has problems. For start thread_pool.spawn(print_message_fut(&s)); is not concurrent. It appears to be blocked until thread_pool.spawn returns.

8

thread_pool.spawn works fine, but your function print_message_fut is not asynchronous. When you call it, it first prints a message, and then it creates a future. However, the future doesn't do anything, so it's ready immediately. If you want to output to stdout asynchronously, you should use tokio::io::stdout.

1 Like
9

Hi jethrogb,

Thanks for the reply. I want to make it asynchronous.

Stdout was just an example, I'm learning how to make asynchronous calls in rust. It could be stdout, calls to rest api to another server, or doing small computations in the background.

Thanks again

10

Updated the code, it is working somewhat similar.

Following change made it asynchronous.

                   thread_pool.spawn(lazy(move || {
                        future::poll_fn(move || {
                            tokio_threadpool::blocking(|| {
                                print_message(&s);
                            }).map_err(|_| panic!("the threadpool shut down"))
                        })
                    }));
11

I want to add timeout on tokio_threadpool::blocking.

the other futures in this task will not make progress until the closure returns. If this is not desired, ensure that blocking runs in its own task (e.g. using futures::sync::oneshot::spawn ).

How do I use futures::sync::oneshot::spawn?
It will work same as timeout and release the resources used by the closure?
Is there a way to add timeout?

Thanks in advance.

12

Without knowing the larger picture, it seems to me that there's no sense in porting working Go async code to Rust, especially given how nascent Rust's async story still is.

13

Posted here related to this topic: How to timeout tokio_threadpool::blocking? - #3 by Kruzer

Seems like rust at the moment doesn't have a way to efficiently cancel blocking code, but I'm still learning rust futures and tasks library.

In C/C++ you can call interrupt on the thread and Go also has pattern for canceling with their select statement on go channels. Crossbeam has select macro but it is not stable and from reading some comments it seems like it may never become stable.

One way would be to use timeouts to cancel but with more than million background tasks it will probably become problematic.