Better way to work with tokio_fs

help
1

Hi,

I've been playing with new hyper and as small exercise I wanted to change file sending example with tokio-fs - so the data are send as they are read. I also wanted to read file size asynchronously.
As a first attempt I ended up with this function, which does nod compile due to borrow checker ( either file does not live long enough or if moved cannot be used later):

fn streamed_file_send(f: &str) -> ResponseFuture {
    // Serve a file by asynchronously by chunks
    let filename = f.to_string(); // we need to copy for lifetime issues
    Box::new(tokio_fs::file::File::open(filename)
        .and_then(|mut file| {
            let meta_future = poll_fn(|| file.poll_metadata());
            meta_future.and_then(|meta| {

                let stream = file.framed(ChunkCodec()); 
            
                let res = Response::builder()
                .body(Body::wrap_stream(stream))
                .unwrap();
                Ok(res)

            })
            
        })
        .or_else(|_| {
            Ok(Response::builder()
                .status(StatusCode::NOT_FOUND)
                .body(NOTFOUND.into())
                .unwrap())
        }))
}

The only way how to fix this I can think of is to create custom Future that first consumes and then returns file like this:

struct MetaFuture(Option<tokio_fs::File>);

impl Future for MetaFuture {
    type Item = (tokio_fs::File, std::fs::Metadata);
    type Error = io::Error;
    fn poll(&mut self) -> Result<Async<Self::Item>, Self::Error> {
        if self.0.is_none() {
             panic!("Invalid state - called after resolution")
        }
        let meta = try_ready!{self.0.as_mut().unwrap().poll_metadata()};
        let file = self.0.take();
        Ok(Async::Ready((file.unwrap(), meta)))   
        
    }
}

impl MetaFuture {
    fn new(f:tokio_fs::File) -> Self {
        MetaFuture(Some(f))
    }
}

But that's quite a bit of code for such common use case - isn't there some better. easier more elegant solution?

Thanks

2

Why are you using poll_fn + poll_metadata() to get the metadata? You should use https://tokio-rs.github.io/tokio/tokio/fs/struct.File.html#method.metadata, which gives you a future yielding the file and metadata, which is basically like the custom future you wrote :slight_smile:.

1 Like
3

I added File::metadata and Metadata for exactly this reason: using the poll_* functions are overly painful and aren't intended for most end users.

Of interest is that whenever async / await become "real enough", most of these functions can be rewritten to take &mut self instead of self and should be even easier to use.

1 Like
4

Indeed. I’m eagerly awaiting (no pun intended) async/await so that the APIs can adjust rather than going through the self -> &mut self breaking change churn. I suspect others are waiting as well to avoid living in somewhat of a bifurcated tokio/futures lib ecosystem.

5

@vitalyd - thanks, I somehow missed that method, not sure why. But at least I trained myself again in creating futures:-) What would be best resource to learn about coming async/await in Rust? I used them in Python, so wondering how it'll work in Rust.

6

I have to use version from git for file.metatada() to compile - in version on crates.io it's not yet available there - so that's why I initially took the detour.

But for latest tokio_fs from git I cannot make it work with hyper 0.12 due to this error: Error opening file: blocking annotated I/O must be called from the context of the Tokio runtime.

7

You need to use the (default) ThreadPool and Runtime, not the current_thread::Runtime, with tokio-fs.

(I'm trying to improve docs related to that, see PRs.)

8

Which PRs? The documentation is still not very good in this respect.