Tokio, Futures, and UDP


#1

Hi all,

I’ve been looking at two examples in tokio-core related to futures & UDP as well as the tutorial that Alex Crichton wrote (https://github.com/alexcrichton/futures-rs/blob/master/TUTORIAL.md). He’s done quite a lot and I keep seeing his name pop up, much praise to him. I’m still confused a bit confused as to how futures operate in the context of a larger application. I think this is partly due to the fact that I haven’t heard of futures until Rust, and I am still very new to Rust.

From what I understand, tokio builds on top of the futures crate to handle asynchronous I/O for protocols (tokio==mio+futures). Okay, I could have pulled that definition off of any website. Well then, what does this mean to me? I need to receive a response (or fire off a request which has its own response), but I don’t want to sit there waiting for it. Ideally I’d like to go off and do other things. In essence, I need to process something that will arrive in the future, and I am not ready now (or rather it is not ready).

So specific to my problem, I think this means I would use tokio to set up a future for a “UDP handler” which could send and receive udp packets, populating and pulling from queues as needed.

So far, I can only really see the receiving aspect via poll(). It seems to me that poll() would be the perfect spot to listen for new packets on a socket. I gleamed this from the echo-udp.rs example in tokio-core. What I don’t understand is how this same socket could be used to send data to a destination. It looks like accessing that socket can only occur within the context of poll(), but I do not want to send data there, only receive it. I do understand that poll, rather the Server future owns the socket, so yes I can only access it from poll(). I considered perhaps I need two futures, one for sending and receiving, but I threw that idea out as I don’t see why/how the sender would need to poll.

The other example I saw was the udp-codec.rs example in tokio-core. This sets up a UpdCodec which is then split into its sink (recv) and stream (send) halves. I am able to follow how and why we define our codec, but the rest is lost on me once it’s split into halves. It looks like this future already knows what to send and receive, and is expected to return because of that. Are futures not supposed to be kept alive, meaning once I bind a socket and tie it to a future which receives, am I expected to continually create a new “receiving” future after each one has been consumed?

Finally in both of the examples above, echo-udp.rs and udp-codec.rs, the tasks are actually run when we give it to the Core trait Core::run(...). This doesn’t return until the future completes, meaning we’re stuck there. It looks like the future dies once run returns, meaning anything associated with that future would be inaccessible after it runs and returns. All the examples I looked at do not perform any addition steps after the future is run. Furthermore, the examples I saw all tied sending and receiving to each other on localhost, not a foreign entity (ie: send and receive to ‘myself’ which are joined to each other). Are we supposed to create futures in their own threads so we can do other things while the future is running?

Ultimately, I am trying to port a UDP library which implements reliability, congestion avoidance, among a few other things. When it was first suggested to me to use futures to accomplish my goal, I envisioned that they would help with the tx/rx at the lower layer, where higher layers would would handle the algorithmic aspects of the library. Is my topography skewed on how this should be layered?

I know I’m pretty lost right now and look like I need some hand-holding. Any help would be appreciated. Thanks.


#2

If you are both new to Rust, and haven’t heard of futures before, may I suggest that implementing a network library with futures in Rust might not be the best project to start with?

I would recommend first to get comfortable with Rust, then to use futures and tokio on a simple text-based protocol (like ident or IRC or FTP) so you’re at ease with the concepts and the framework, and finally to get on with your (ambitious!) project.


#3

Maybe I misspoke about my comfort level with Rust. I’ve been fiddling around with it since September and am decently comfortable with syntax, the Rust book has been extremely helpful, among other resources online. This isn’t something I am afraid of, but I do like to let people know mostly so they don’t assume I’ve been at this for years :stuck_out_tongue:. Where I am lacking is with the concept of futures and general exposure to them.

I know this wasn’t going to be an easy task to begin with, though in honesty, I imagined hooking up futures & tokio to a udp socket as something straightforward than what I encountered. Now I realize this is where I bit off more than I can chew. I think I need to go read up on futures and remind myself that cities aren’t built in a day. I do not want to give up on this project however, even if no one ever uses it I still see value in the journey. I think it’ll help if I stop focusing on my end goal w/ udp and see how other applications of futures work.

The text-based protocol approach is a great suggestion, thank you for that. Do you know of any crates using futures which are non text based? It’d be nice to glimpse over the fence once I’m ready.


#4

You can actually use crates.io to explore crates that are using tokio-core: https://crates.io/crates/tokio-core/reverse_dependencies

For instance, this Redis client might be interesting to look at: https://github.com/Connorcpu/tk-redis/tree/master/src

I have started an HTTP client that you could also find useful to take a look at, as it is small and simple: https://github.com/matt2xu/async-http-client/blob/master/src/lib.rs


#5

Thank you for your reply and your comments Matt2xu. I found these two videos extremely helpful and want to share them with everyone else in case they are stuck in the same boat as me.

Futures in Rust
Consuming Futures in Rust and C#
Edit: Just wanted to add another link into the mix for visibility. The Getting Started and Going Deeper sections are very helpful, take some time to dig through!


#6

I’ve encountered the same issue as you and I don’t feel like your question was answered. Pointing to libraries that use futures doesn’t help with the basic question of how do you actually send something to a remote UDP port directly using tokio-rs which clearly seems to be what the OP is trying to do.

Both of the examples you referenced (udp-codec / echo-udp) do not solve the problem you asked since they assume that the socket is bound to 127.0.0.1 and assume that this is where the send is supposed to occur. To connect to a remote UDP port and send it a message one way to do it would be to use send_to directly on the socket. So bind to “0.0.0.0:0” and then use socket.send_to(&buf, ("", port)). If there is a better way of doing this with tokio-rs it would be nice to know what it is.


#7

There’s https://docs.rs/tokio-core/0.1.9/tokio_core/net/struct.UdpSocket.html#method.send_dgram if you want to get a Future back that resolves when the data is sent (or an error). That’s probably the more tokio way of doing it.


#8

It’s been a while since I looked at this code but I believe I did get something functional (though incredibly rudimentary) here.