Convert hyper::Body to String

I just added a commit to add the Cargo.lock to the repo !
I was wrong, I am not using the lastest version of hyper :

name = "hyper" 
version = "0.12.11

Okay, looking through the Hyper docs, it looks like you probably want to use the poll_data method to extract the underlying data which are Chunks. From here, you can convert the chunk into Bytes with into_bytes, which can be converted to a String with String::from(bytes) (see shepmaster's post below).

N.B. there is likely a better way to do this. I am relatively unfamiliar with Hyper, but this should work to extract a string from the response body.

Wow, I would have never figured that out my self, I will give it a spin, thanks!
Side question spawned by yours, would you have a way to simplify a chain of maps (like this one:

... 
.map(hyper::poll_data)
.map(hyper::into_bytes)
.map(String::from)

) ?

P.S. : I am open to more simple answers

In general, sequences of maps may be combined using a closure

...
.map(f)
.map(g)
.map(h)

can be converted to

...
.map(|x| h(g(f(x))))

However, in this specific case the closure must be more complicated. The poll_data method doesn't actually return the data directly, it returns a Poll<Option<Self::Data>, Self::Error>. To actually extract the data from this, you must pattern match on Poll to check for the Ok(Async::ready(data)). The most convenient way to do this is using if let notation.

The resulting code might look something like this (I haven't tried to compile this):

...
.map(|body| if let Ok(Async::ready(data)) = body.poll_data() {
    // This is incorrect, see `https://users.rust-lang.org/t/convert-hyper-body-to-string/23179/12`
    // String::from(data.into_bytes())
} else {
    panic!("The data wasn't ready!")
})

In a production environment, you would want cleaner error handling than an explicit panic. The panic may be unreachable if you call .wait() before .map() but I haven't worked with Futures enough to know for sure.

Yeah that's what I thought.

Where does this comes from ? I just tried to compile it and it sends back a wonderful use of undeclared type or module Async`.

Async comes from the futures crate.

That's exactly what I thought...and that sends back an awful compilation error :

error[E0191]: the value of the associated type `Item` (from the trait `page::futures::Future`) must be specified
  --> src/page.rs:61:27
   |
61 |                 if let Ok(Future::Async::Ready(data)) = body.poll_data() {
   |                           ^^^^^^^^^^^^^^^^^^^^ missing associated type `Item` value

error[E0191]: the value of the associated type `Error` (from the trait `page::futures::Future`) must be specified
  --> src/page.rs:61:27
   |
61 |                 if let Ok(Future::Async::Ready(data)) = body.poll_data() {
   |                           ^^^^^^^^^^^^^^^^^^^^ missing associated type `Error` value

error[E0223]: ambiguous associated type
  --> src/page.rs:61:27
   |
61 |                 if let Ok(Future::Async::Ready(data)) = body.poll_data() {
   |                           ^^^^^^^^^^^^^^^^^^^^ help: use fully-qualified syntax: `<dyn page::futures::Future as Trait>::Async`

I just tried the kind help above given by the compiler without any luck.

For info, by replacing this in the code :

if let Ok(<dyn page::futures::Future as Trait>::Async::Ready(data)) = body.poll_data() { ... }

One can get this error message :

   |                 if let Ok(<dyn page::futures::Future as Trait>::Async::Ready(data)) = body.poll_data() {
   |                                                                             ^ unexpected `(` after qualified path

https://stackoverflow.com/q/43419974/155423


I'd probably do

use futures::{Future, Stream}; // 0.1.25
use hyper; // 0.12.14 
use std::str;

type Error = Box<dyn std::error::Error>;

fn example(body: hyper::Body) -> impl Future<Item = String, Error = Error> {
    body.map_err(Error::from)
        .concat2()
        .and_then(|c| {
            str::from_utf8(&c).map(str::to_owned).map_err(Error::from)
        })
}

Can you clarify why you believe that to be possible? I don't see an implementation of From<Bytes> for String although I do see one the other way (impl From<String> for Bytes). I don't see how you could do the former as the bytes might not be UTF-8...

Ah, I think I saw impl From<String> for Bytes and erroneously read impl From<Bytes> for String. You are correct.

I was really happy when I found this snippet... and even more sad when it did not work :

I tried to adapt the code in this way sending back :

error[E0599]: no method named `concat2` found for type `std::result::Result<page::hyper::Body, std::boxed::Box<dyn std::error::Error>>`
 in the current scope
  --> src/page.rs:60:14
   |
60 |             .concat2()
   |              ^^^^^^^
   |
   = note: the method `concat2` exists but the following trait bounds were not satisfied:
           `&mut std::result::Result<page::hyper::Body, std::boxed::Box<dyn std::error::Error>> : page::futures::Stream`

error[E0277]: the size for values of type `[u8]` cannot be known at compilation time
  --> src/page.rs:61:24
   |
61 |             .and_then(|c| str::from_utf8(&c).map(str::to_owned).map_err(Error::from))
   |                        ^ doesn't have a size known at compile-time
   |
   = help: the trait `std::marker::Sized` is not implemented for `[u8]` 

Obviously, unwraping before concat2 makes things so worse that I don't even include it in this message.

It works just fine.


no method named concat2 found for type std::result::Result

Yes, there is no method concat2 on Result. In my original code, that's called directly on the body Stream. It appears you are attempting to exit out of the async world too early. Move your wait to the end of the chain.

You mean like this :

(*self)
            .request(
                request
                    .body(hyper::Body::empty())
                    .expect("The body of the request could not be consumed"),
            )
            .map(hyper::Response::into_body)
            .map_err(Error::from)
            .concat2()
            .and_then(|c| str::from_utf8(&c).map(str::to_owned).map_err(Error::from))
            .wait()

I tried and that sends back :

error[E0599]: no method named `concat2` found for type `page::futures::MapErr<page::futures::Map<page::hyper::client::ResponseFuture, f
n(page::hyper::Response<page::hyper::Body>) -> page::hyper::Body {<page::hyper::Response<T>><page::hyper::Body>::into_body}>, fn(page::
hyper::Error) -> std::boxed::Box<dyn std::error::Error> {<std::boxed::Box<dyn std::error::Error> as std::convert::From<page::hyper::Err
or>>::from}>` in the current scope
  --> src/page.rs:59:14
   |
59 |             .concat2()
   |              ^^^^^^^
   |
   = note: the method `concat2` exists but the following trait bounds were not satisfied:
           `&mut page::futures::MapErr<page::futures::Map<page::hyper::client::ResponseFuture, fn(page::hyper::Response<page::hyper::Bo
dy>) -> page::hyper::Body {<page::hyper::Response<T>><page::hyper::Body>::into_body}>, fn(page::hyper::Error) -> std::boxed::Box<dyn st
d::error::Error> {<std::boxed::Box<dyn std::error::Error> as std::convert::From<page::hyper::Error>>::from}> : page::futures::Stream`

error[E0277]: the size for values of type `[u8]` cannot be known at compilation time
  --> src/page.rs:60:24
   |
60 |             .and_then(|c| str::from_utf8(&c).map(str::to_owned).map_err(Error::from))
   |                        ^ doesn't have a size known at compile-time
   |
   = help: the trait `std::marker::Sized` is not implemented for `[u8]`
   = note: to learn more, visit <https://doc.rust-lang.org/book/second-edition/ch19-04-advanced-types.html#dynamically-sized-types-and-
the-sized-trait>
   = note: all local variables must have a statically known size
   = help: unsized locals are gated as an unstable feature

You'll need to put the entire code I posted inside of a map or and_then call. Something like

.map(hyper::Response::into_body)
.and_then(example)

I am so sorry but I tried this

    (*self)
        .request(
            request
                .body(hyper::Body::empty())
                .expect("The body of the request could not be consumed"),
        )
        .map(hyper::Response::into_body)
        .and_then(|b| {
            b.map_err(Error::from)
                .concat2()
                .and_then(|c| str::from_utf8(&c).map(str::to_owned).map_err(Error::from))
        })
        .wait()

and still fails :sob:

Error message
error[E0271]: type mismatch resolving `<page::futures::AndThen<page::futures::stream::Concat2<page::futures::stream::MapErr<page::hyper
::Body, fn(page::hyper::Error) -> std::boxed::Box<dyn std::error::Error> {<std::boxed::Box<dyn std::error::Error> as std::convert::From
<page::hyper::Error>>::from}>>, std::result::Result<std::string::String, std::boxed::Box<dyn std::error::Error>>, [closure@src/page.rs:
61:31: 61:93]> as page::futures::IntoFuture>::Error == page::hyper::Error`
  --> src/page.rs:58:14
   |
58 |             .and_then(|b| {
   |              ^^^^^^^^ expected struct `std::boxed::Box`, found struct `page::hyper::Error`
   |
   = note: expected type `std::boxed::Box<dyn std::error::Error>`
              found type `page::hyper::Error`

error[E0599]: no method named `wait` found for type `page::futures::AndThen<page::futures::Map<page::hyper::client::ResponseFuture, fn(
page::hyper::Response<page::hyper::Body>) -> page::hyper::Body {<page::hyper::Response<T>><page::hyper::Body>::into_body}>, page::futur
es::AndThen<page::futures::stream::Concat2<page::futures::stream::MapErr<page::hyper::Body, fn(page::hyper::Error) -> std::boxed::Box<d
yn std::error::Error> {<std::boxed::Box<dyn std::error::Error> as std::convert::From<page::hyper::Error>>::from}>>, std::result::Result
<std::string::String, std::boxed::Box<dyn std::error::Error>>, [closure@src/page.rs:61:31: 61:93]>, [closure@src/page.rs:58:23: 62:14]>
` in the current scope
  --> src/page.rs:63:14
   |
63 |             .wait()
   |              ^^^^
   |
   = note: the method `wait` exists but the following trait bounds were not satisfied:
           `page::futures::AndThen<page::futures::Map<page::hyper::client::ResponseFuture, fn(page::hyper::Response<page::hyper::Body>)
 -> page::hyper::Body {<page::hyper::Response<T>><page::hyper::Body>::into_body}>, page::futures::AndThen<page::futures::stream::Concat

The error message keeps getting bigger and bigger by each iteration...

        self
            .request(
                request
                    .body(hyper::Body::empty())
                    .expect("The body of the request could not be consumed"),
            )
            .map(hyper::Response::into_body)
            .map_err(Error::from)
            .and_then(example)
            .wait()

This works except that your error types don't match. You need to return some kind of error that can handle hyper errors and UTF-8 errors. The laziest solution is to just panic for any error:

    fn get_http_response(
        &self,
        url: Uri,
        cookie: hyper::header::HeaderValue,
    ) -> Result<String, hyper::Error> {
        let request = Request::get(url).header("Cookie", cookie);
        self
            .request(
                request
                    .body(hyper::Body::empty())
                    .expect("The body of the request could not be consumed"),
            )
            .map(hyper::Response::into_body)
            .and_then(example)
            .wait()
    }
}

fn example(body: hyper::Body) -> impl Future<Item = String, Error = hyper::Error> {
    use std::str;

    body.concat2()
        .and_then(|c| {
            str::from_utf8(&c).map(str::to_owned).map_err(|x| panic!("Not a string: {}", x))
        })
}

The next solution is to use a Box<dyn Error> or your own custom type.

Wow...that works... THANK YOU A LOT !

I am interested. How would I do it ?

It would by using the failure crate right ?

Error handling is well covered in The Rust Programming Language. The original code I posted uses Box<dyn Error> as an example.

I'm personally not a huge fan of that crate, but I also haven't used it extensively. I've listed a few crates that help with defining your own error type:

https://stackoverflow.com/q/42584368/155423

I personally like quick-error.

1 Like

Thanks again for the time and patience you put to solve my problem with your knowledge !