Pipe `Write` to `AsyncRead`

help
1

Title says it all. I have a std::io::Write and I need a futures::AsyncRead.

If I'd build this myself, I'd probably have a ring buffer and write on one side and read on the other side. But I don't know if there are any ring buffers with the respective read+write implementations and that also are thread safe.

My second best guess would be to send over buffers using a channel (I know there are some sync-async-compat libraries for it) and somehow wrap the traits on the ends.

But let me know what you think.

2

I don’t feel like the title says it all. Or at least in my head this sounds a lot different from what the title sounds like. I would’ve thought (from the title) that you would have both a std::io::Write and a futures::AsyncRead and want to connect the two, i.e. pipe the contents of the AsyncRead to the Write. Although actually the title has it the other way around, hmm… anyways: Your post then sounds more like you’d want to convert a std::io::Write into a futures:.AsyncRead which sounds like a weird request to me.

Unless I’m missing or misunderstanding something, this actually sounds almost nonsensical: an object of a type that implements Write is something you can write to and a thing with a type that implements AsyncRead is something you can read from (and that can utilize asynchronous operations internally). How can I somehow gain the ability to read from something about which I, previously, only know that I can write to it?

3

Thanks for the input, I'll have to think that through again. I initially thought about something like std::io::copy but indeed it has the order reversed, so what I need is more something like a pipe. I also found an async version of that crate, but what I'd need would be one with one side sync and one side async.

But now that I think of it, this brings me down to simply needing either a Write->AsyncWrite or AsyncRead->Read adapter.

4

Okay, so all you need might be futures::io::AllowStdIo::new(writer))?

In case that the Write you have is something like a Vec<u8> or so that doesn’t block for too long, it might even be sensible to execute the copy operation still in an async context. Well even if that isn’t the case blocking execution is still an option.

Something like

futures::io::copy(async_reader, &mut futures::io::AllowStdIo::new(writer)).await
1 Like
5

Why not use a pipe that works with async in the first place? There is one in tokio::io::duplex.

6

AllowStdIo looks really helpful, thanks! I'll try to either create a synchronous Write->Read pipe and then wrap the Read in AllowStdIo, or to create an AsyncWrite->AsyncRead pipe and then wrap the write side in a blocking Write.

Do you know a reverse helper that does the opposite of AllowStdIo? It sounds really easy to implement, but I could I couldn't find any existing implementations.

Because one half of it must be passed to a sync library.

7

This is the part that confuses me. How does the writer act as input and the reader as output? I'm guessing the scenario is more complex and involves side-effects or something but I can't see the analogy of a pipe. By definition we can't use a writer to fill a reader.

8

That's how a pipe works. And yes, sometimes it's a bit confusing. You have a writer end and a reader end and the reader will read all bytes that have been written by the writer. This is not totally uncommon, in Java for example it's called PipedInputStream.

I noticed that "ring buffer" libraries often have an API that gives exactly this.

9

A pipe would be Read -> Write. If you're reading from input source and it's a writer, then the fact that it's a writer is a coincidence, you need to specify the input source it as Read in order to call the read method. Then the output data will be fed to a Write type.

10

The duplex pipe that I linked is Write -> Read. You write something in one end, then you can read what you wrote from the other end.

11

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.