I’m trying to make body-image fully asynchronous-friendly for use with tokio. Tokio reform and hyper 0.12 (initially, the master branch) offer more of what I need to do this. Below I provide some rough notes on progress with some embedded questions. Warning: long.
In most cases the byte-oriented http crate
HeaderMap is fine for body-image, but it does need to parse
Transfer-Encoding headers, for which I was relying on the typed
hyper::header module as of hyper 0.11. The entire module was removed in hyper 0.12. There was a promise made to split it out as part of the next release, and then a redaction. I asked about this again two weeks ago but didn’t hear back. So, to ease my migration and testing, I forked the header module into a new hyperx crate:
For extra-snarky humor value, pronounce it /hī ˈpərks/.
Its released on crates.io and you may find it helpful if you are in a similar situation.
This was/is tokio-core’s run method:
pub fn run<F>(&mut self, f: F) -> Result<F::Item, F::Error> where F: Future
The signature losses a return value and the
Future associated types in tokio reform, at least with the default runtime and executor:
pub fn run<F>(future: F) where F: Future<Item = (), Error = ()> + Send + 'static
I’m sure there must be good reason for this. Anyone know what it is? The Tokio reform RFC doesn’t mention it, and I can’t find any other conversation about it. (See below, I later discovered the more convenient
hyper 0.12 client
I’d like to point out one thing not explained in this terse example and also here. If the Client::new() line is moved out of the closure, up to the beginning of
main (like anyone would try in order to reuse it), the
run method never returns. I suspect I will not be the last one to burn some time on that.
Here is my first reasonable working version of the upgrade, using a one-shot channel, and also with my first public use of stable
impl Future. Regarding the later, I have a private function that remains returning a
Box<Future> as apparently
impl Future can currently only hide one type at a time?
Later I found it was even cleaner to use the
current_thread::Runtime and its
block_on method, at least in my tests. However, I suspect I will need to revert or replace this for cases where
BodyImage is using tokio annotated
blocking I/O. (A cool new method of annotating blocking I/O or slow computations without blocking the tokio event loop, by simply moving the event handling to another thread.)
This area seems sketchy with hyper 0.11, so I left it as a
FIXME. Here I was able to add timeouts using tokio-timer, which appears to work as expected and is much cleaner then the example originally given for the hyper 0.11 guide. I have two tunable timeout intervals, one for any response and another for a complete body. Any suggestions for improving this?
cargo tree --all-features -d shows I now have some duplication of dependencies via openssl(-sys) and (hyper(tokio(native)))-tls.
bitflags v0.9.1 └── openssl v0.9.24 └── native-tls v0.1.5 ├── hyper-tls v0.2.0 │ └── body-image v0.3.0 (file:///home/david/src/body-image) └── tokio-tls v0.1.4 └── hyper-tls v0.2.0 (*) bitflags v1.0.3 └── clap v2.31.2 └── body-image v0.3.0 (file:///home/david/src/body-image) lazy_static v0.2.11 └── native-tls v0.1.5 ├── hyper-tls v0.2.0 │ └── body-image v0.3.0 (file:///home/david/src/body-image) └── tokio-tls v0.1.4 └── hyper-tls v0.2.0 (*) lazy_static v1.0.1 ├── crossbeam-epoch v0.4.1 │ └── crossbeam-deque v0.3.1 │ └── tokio-threadpool v0.1.3 │ ├── tokio v0.1.6 │ │ ├── body-image v0.3.0 (file:///home/david/src/body-image) │ │ ├── hyper v0.12.1 │ │ │ ├── body-image v0.3.0 (file:///home/david/src/body-image) (*) │ │ │ └── hyper-tls v0.2.0 │ │ │ └── body-image v0.3.0 (file:///home/david/src/body-image) (*) │ │ └── tokio-core v0.1.17 │ │ └── tokio-tls v0.1.4 │ │ └── hyper-tls v0.2.0 (*) │ └── tokio-fs v0.1.0 │ └── tokio v0.1.6 (*) └── openssl v0.9.24 └── native-tls v0.1.5 ├── hyper-tls v0.2.0 (*) └── tokio-tls v0.1.4 (*)
cargo tree --all-features -r hyper-tls is (elided) below. Am I using the wrong crate(s) for HTTPS at this point? Why do I still have a dependency on tokio-core and older versions of lazy_static and openssl? Where and what PR’s should I submit to help clean this up?
hyper-tls v0.2.0 ├── native-tls v0.1.5 │ ├── lazy_static v0.2.11 │ └── openssl v0.9.24 │ ├── bitflags v0.9.1 │ ├── lazy_static v1.0.1 (*) │ └── openssl-sys v0.9.32 └── tokio-tls v0.1.4 ├── native-tls v0.1.5 (*) ├── tokio-core v0.1.17 │ ├── scoped-tls v0.1.2 │ ├── tokio v0.1.6 (*) │ ├── tokio-executor v0.1.2 (*) │ ├── tokio-io v0.1.6 (*) │ ├── tokio-reactor v0.1.1 (*) │ └── tokio-timer v0.2.3 (*) └── tokio-io v0.1.6 (*)
As this post might appear a bit critical: In general, the crates involved appear to be rapidly maturing and I’m excited about tokio reform, the hyper 0.12 release, and to have gotten this far with my project. Thanks for all this work!