Crate providing readers/writers in background thread


#1

I’d like to ask for feedback on this small crate, whose purpose is to allow putting io::Read and io::Write instances to a background thread.


For me, this is mainly useful for reading from / writing to compressed files in the background, which is typically expensive CPU-wise. I’ve tested the code successfully as part of this tool, and found nice performance improvements with compressed input/output, if there was some expensive computation happening in the main thread. I’ve tested the crate quite extensively and tried to make sure that errors are handled well. Still I’d like to be sure that there isn’t another, maybe easier way of solving this problem using existing crates.
Note: I should credit the nice fastq-rs library, which implements a background reader, and from which I got the original idea. The implementation in my library is slightly different and is not restricted to using readers with static lifetimes.

I’d be very interested on feedback, and whether it could be useful to publish this crate on crates.io. Thank you very much!


#2

I would be very interested in using it if available on crates.io

(I also didn’t know about fastq-rs, thanks for showing it!)


#3

Thanks for the comment. I have been working on improving code quality and testing, and finally released the library on crates.io. I hope that it will be useful. Don’t hesitate to file an issue or post comments on this thread.

Regarding the API, I’m a bit unhappy about the need for adding type annotations (explained in README) in order to allow for custom error types to be returned. However, I would like to keep that flexibility and I don’t know whether there currently is a possibility to work around this :frowning: