So I've got a very simplified example of my problem (at least I think so) here: Rust Playground
Obviously it won't compile, but I'm not really sure how to solve the issue. Basically what I would like to achieve is to have a mutable reference to the server's poll passed to the connection that is valid up until the connection is. I thought I could simply add a lifetime to the Connection structure, but that only won't really solve it.
Could someone please explain how to do this with a little bit of details?
You are essentially trying to create a struct which contains a reference to its own content (a Server's Connections have a reference to the Poll stored next to them). The Rust ownership and borrowing model does not support this at the moment, and is unlikely to support it in the medium term as it will require a lot of work on the core semantics of the language.
To understand why, consider how self-referentiality affects core Rust operations like moving or borrowing. In current Rust, moving a struct is always safe if no one holds a reference to them. With self-referential structs, however, moving is dangerous even then because it invalidates the inner reference. Similarly, the order in which struct members are dropped is largely an implementation detail at the moment, whereas dropping the fields of a self-referential struct in the "wrong" order could result in memory unsafety if a reference is dropped later than the thing it refers to.
To work around this, you have several options. One of them is to use shared ownership (e.g. Rc<Poll>), which eliminates the backreference by moving the Poll to the heap, but introduces a slight performance hit and worsens usability when mutability is involved. Another is to review your design so that Connections do not hold a long-lived reference to the Poll (which is rarely a good idea in Rust), and favor transient references instead (for example by passing a Poll reference down to the Connection's methods when they need one).
EDIT: Looking at the playground further, even if Rust supported self-referential structs, your code would still be rejected because it attempts to create multiple &mut references to a single Poll object (one per Connection).
I see, thanks. Initially I was doing the second option, but then came the problem (the play example was really simplified): I wanted to move the connections via a mpsc channel to a worker thread. Since I have to deregister the connection from the poll after I've dealt with it, I'm not sure how to do this now at all.
I'm not sure if I fully understand your problem, but if this is only about deregistering the connection at the end, it sounds like you could have another mpsc channel which flows back from the worker thread to the thread hosting the Poll and is used to sends disconnection requests.
Main thread Worker thread
| |
[ Register connection ] |
| |
[ Send connection to worker ] ------------------------>|
| |
| [ Do the work ]
| |
|<------------------------[ Notify main thread that we're done ]
|
[ Deregister connection ]
Basically I have a couple event loops, and a couple worker threads. Since I need to shard the underlying database, I have to peek into the request, reading a couple bytes before I can send the connection to the exact worker thread. That's really it.
I've considered this option, but coming from Go I kinda tried to avoid channels since their performance aren't really there in Go. Do you reckon this should be fine here? The work that needs to be done in the worker thread lasts around ~2000 ns, not sure how much overhead would this be, sadly.
I know I shouldn't worry about performance at first, but that's the main reason I'm actually rewriting the whole app in Rust.
Why do you need to deregister the connection after servicing a single request? Wouldn’t you want to keep read interest on it for further data? What’s the connection lifecycle exactly?
Where do you want to do the IO back to the peer from? You mention a couple of event loops and a couple of workers - I’d expect the workers to not do any IO in this setup and instead respond with data sent over a channel back to the IO thread running the event loop.
An alternative is you run an event loop per thread and service the connection solely from that thread. If your request handling takes 2us then you may as well combine IO and processing on the same thread. Shard the data over these workers and if a shard needs data from another shard, ask for it over a channel. That’s roughly the seastar model.
Nope, I don't want to keep reading anything. This is the main reason I went with mio instead tokio, since all the connections will send a Connection: close HTTP header and I wanted to use level-triggered epoll since it's much faster than edge-triggered for this scenario and tokio (at the moment - checked a couple weeks ago) won't allow me to set this option.
Well yeah, you're right, that would be the same amount of send/receive, might aswell just do that, although I'm not sure I want that many event loops running, I like the idea to have them separated, so I'll probably go with the first option and send the connection back to the event loop thread.
So you mean you accept a tcp connection, service it with the 2us handler, write a response and then close the connection?
I don’t quite follow why level triggered is faster here (one typically avoids level triggering out of performance concerns), but that’s likely because I didn’t fully understand the scenario.
Edit: oh I see what you mean - I misread the Connection: close http header that you mentioned as something else. I don’t see why level triggering is better, however.
By the way, I don’t think I’d worry about channel latency here given clients don’t maintain a persistent tcp connection - they’re going to be more impacted by tcp handshakes and small(ish) congestion windows.
That's exactly what happens, yes. And to be honest I don't have the knowledge to answer your question, I have no idea why level-triggered performs much better, but for some reason when I test it with a simple hello world response and wrk, there's a huge difference.
Thank you for the help though! I guess channel is the way to go.
Sorry to bother again, but I just completely missed what you said before: basically with the seastar model I could potentially save a round-trip between threads, right? Obviously the more threads I use, the less likely. However, it still sounds nice, but how could I actually implement this? Since all threads would have an event loop, do I have to use some kind of async io (futures, tokio, I don't know) for this? How could one thread in the same time listen for incoming requests from the other threads and poll the event loop?
Does mio support SO_REUSEPORT? If so, you can have multiple event loops (each on its own thread) accepting connections on the same listen port - kernel then picks a single loop to hand off connections to.
Alternatively, you can set up a normal blocking socket to listen on; when a connection is accepted, you ship it to a thread of your choosing that's running a loop, and then attach the socket to that loop. A tokio example of doing this is https://github.com/tokio-rs/tokio-core/blob/master/examples/echo-threads.rs - you should be able to do the same thing with mio.
Not sure if it supports natively (the docs say it does), but it didn't work for me reliably, so I'm using the nix package to solve that, it works fine. However, what if thread #1 accepts a connection and it needs data from thread #2? Since both thread would run an event loop, thread #2 (or any other thread) can't really recv() the channel, where would they?
Thread 1 sends the socket to thread 2. Thread 2 then attaches that socket to its loop, and handles the interaction with that socket (peer) for its lifetime.
Thread 1 forwards a request for data to thread 2 over a channel, and thread 2 responds with data over a (separate) channel. Thread 1 then writes the response back to the peer.
I'm not sure offhand how you'd set up the multiplexing in mio but it should be possible - Poll would be monitoring the listening socket and a channel; they'd be 2 separate sources of (notification) data. In the tokio world, one could spawn() handling of the channel onto the reactor (event loop), and this would be a monitored ("evented") source.
