Box<dyn BufRead> -> BufRead giving compiler errors in certain scenarios


use std::io::{self, BufRead, BufReader};
use std::fs::File;

fn main() {
    let reader2 = get_reader(true);
    do_stuff(reader2);
    let reader1 = BufReader::new(File::open("test1.txt").unwrap());
    do_stuff(reader1);

    do_stuff_2(reader1, reader2);
}

fn get_reader(use_stdin: bool) -> Box<dyn BufRead> {

    if use_stdin {
        Box::new(io::stdin().lock())
    }
    else {
        let file = File::open("test.txt").unwrap();
        Box::new(BufReader::new(file))
    }
}

fn do_stuff<R>(reader :R) where R: BufRead {
    for b in reader.bytes() {
        println!("{}", b.unwrap());
    }
}

fn do_stuff_2<R>(reader1: R, reader2 : R) where R: BufRead {
    for c in reader1.bytes().zip(reader2.bytes()) {
        println!("{} {}", c.0.unwrap(), c.1.unwrap());
    }

}
  --> src/main.rs:10:25
   |
10 |     do_stuff_2(reader1, reader2);
   |     ----------          ^^^^^^^ expected struct `BufReader`, found struct `Box`
   |     |
   |     arguments to this function are incorrect
   |
   = note: expected struct `BufReader<File>`
              found struct `Box<dyn BufRead>`
note: function defined here
  --> src/main.rs:30:4
   |
30 | fn do_stuff_2<R>(reader1: R, reader2 : R) where R: BufRead {
   |    ^^^^^^^^^^                -----------

do_stuff() works fine with different objects which implement BufRead but what is up with do_stuff_2() which to my beginner level Rust eye looks just as good.

TIA for your helpful answers.

Each type variable stands for exactly one type per invocation, so do_stuff_2 requires both of the arguments to be the same type. This is useful to be able to express so that you can e.g. put them both in a vector together.

As your code isn't doing anything like that, you probably want to define it like this instead:

fn do_stuff2<R1,R2>(reader1: R1, reader2: R2)
    where R1: BufRead, R2: BufRead
{ /* ... */ }
6 Likes

Thank you that works and now have my full application working :smiley:

So the compiler must be doing something at the function call to 'unbox' the Box item. But does not like it when one has the same trait bound for both (for some reason).

// fn do_stuff_2<R1,R2>(reader1: R1, reader2 : R2) where R1: BufRead, R2: BufRead {
fn do_stuff_2<R>(reader1: R, reader2 : R) where R: BufRead {

This works with the original signature:

    let reader1 = get_reader(false);
    let reader2 = get_reader(true);
    do_stuff_2(reader1, reader2);

    let reader1 = BufReader::new(File::open("test1.txt").unwrap());
    let reader2 = BufReader::new(File::open("test1.txt").unwrap());
    do_stuff_2(reader1, reader2);

Sneaking in a follow on question. How do you manually unbox something?

    let box_reader1 = get_reader(false);
    let reader = box_reader1::???();
    for byte in reader.bytes() {
...

Thanks,

So the compiler must be doing something at the function call to 'unbox' the Box item. But does not like it when one has the same trait bound for both (for some reason).

Let me address the second sentence first. In the OP you had

let reader2 /* : Box<dyn BufRead> */ = get_reader(true);
let reader1 /* : BufReader<File> */ = BufReader::new(File::open("test1.txt").unwrap());
do_stuff_2(reader1, reader2);

And these are two different types -- and Rust doesn't do autoboxing. The problem wasn't the trait bound -- both implement the trait -- the problem was that you used a single type parameter, so the values you pass had to be the same type.


Now back to

So the compiler must be doing something at the function call to 'unbox' the Box item.

It's not unboxing anything here. When you call do_stuff_2, you're just pasing the Box<dyn BufRead> by value. Next, we can see how Box<dyn BufReader> implements BufRead:

impl<B: BufRead + ?Sized> BufRead for Box<B> {
    #[inline]
    fn fill_buf(&mut self) -> io::Result<&[u8]> {
        (**self).fill_buf()
    }
   // ...

It's just calling dyn BufRead's implementation of BufRead. The method calls will take a reference of the **self (i.e. the dyn BufRead) and nothing is removed from the Box.

Going deeper, there's also no need to recover the erased base type

At this point, it's important to understand a bit more about dyn Trait, aka trait objects. Trait objects are

  • dynamically sized (do not implement Sized)
  • able to dynamically dispatch to methods via a vtable
  • but not dynamically typed

That is, a trait object is a statically known type. The compiler supplies the implementation of Trait for the type dyn Trait.

What does that implementation look like? Basically, every time you interact with dyn Trait, you're dealing with some sort of pointer like &dyn Trait or Box<dyn Trait>. Those pointers are wide pointers, containing one pointer to the data and another pointer to a vtable. The vtable holds type-erased versions of the methods that take a type-erased pointer.

Here's an article with more details. Heads-up, it was written before the dyn keyword was used for trait objects, so it uses Foo instead of dyn Foo, for example.

Note that at no point does dyn Trait implementation need to know what the erased base type was, so along with no unboxing, there is no (emulated) downcasting.


Box has a special ability -- the compiler recognizes it's the unique owner of its contents, and lets you move the contents out via a dereference, even if the contents are not Copy. (If they're not Copy this consumes the Box, naturally.) So the literal answer to your question is, you dereference it.

let unboxed = *bx;

...however that's not really what you're asking I imagine.

  • You can't moved non-Sized things like a dyn BufRead
  • You probably meant how do I get the unerased base type out

And in order to do that, you emulate dynamic typing by downcasting with the Any trait.

You have a dyn BufRead and not a dyn Any, so this can't work without creating your own trait to wrap up the concept of BufRead + Any or similar. That's it's own little topic, so I'll stop here.

3 Likes

My own recommendation would be to just abandon that attempt and try to actually learn Rust. Almost half of the How not to learn Rust is dedicated to what you shouldn't do when learning Rust. And trying to “unbox” things is something that rarely works.

People are very creative creatures and they managed to invent ways to write Java in Rust, JavaScript in Rust, Haskell in Rust and many other languages in Rust.

That's not impossible, but hard… and #1 reason for why Rust have that infamous “step learning curve”.

It's not too hard to learn to write code in Rust, but if you try to write some other language in Rust… before you have learned to use Rust as Rust… it's both incredibly hard and incredibly frustrating.

Yet almost everyone does that (me, of course, too… in the beginning).

The sooner you realize that many high-level tasks are solved in entirely different way than you were teached to solve them and would stop trying to translate idioms from $YOU_FAVORITE_LANGUAGE to Rust… the sooner you would overcome that hurdle.

P.S. Surprisingly, many Rust idioms can be easily implemented in the other languages and work well there. That's not a mistake, even.

So even although both parameters share the same trait, that they do represent different types as detected by the compiler, I need to use the extended <R1,R2> syntax because it need to handle them differently.

Thanks for showing me what happens under the covers. When I could not get this to compile I tried all sorts of stuff without success.

Thanks for the link that is very interesting.

I've been reading the Rust book and Rust by Example book Using Trait Objects That Allow for Values of Different Types - The Rust Programming Language. This article helped me get so far with what I wanted to do but since I did not know I had to do <R1,R2> I just ended up trying all sorts of stuff to try and get it to work. I'm curious if this little rule is documented somewhere or just something you learn by experience.

There always seems to be more than one way to do things and I was curious if there was other options. I'm very happy with the solution provided and happy for the compiler to do all the work, once I've learned the right way to do it. There does seem to be lots of syntax tricks to shorten things but then there is also the long way as well.

Yes, and another way to help strengthen your mental model here is to think of generic types in terms of the type system. do_stuff<R> is generic over a single type, whose human-readable alias is R.

You can ignore that the type has a trait bound in this case, to reason about the error message: "expected struct BufReader, found struct Box". Clearly the two parameters passed to the function are not the same type (whose alias is R). The two different types need two different aliases (R1 and R2).

1 Like

In my original example, because I had two calls to do_stuff() with two different types and I just assumed the where R: BufRead was just a nice short cut for aliasing traits, there was no way the error messages helped me make the leap from <R> to <R1,R2>. I did try things like
do_stuff2(reader1: BufRead, reader2: BufRead) and variations but to no avail.

In a previous little program I was doing things like

   if stdin  {
      do_stuff(io:stdin()::lock();
   } else {
      do_stuff(BufReader::new(file));
}

and since I have a few do_stuff_X() in my new little program, I wanted to see if I could avoid all the if statements.

I see there is something -> imp { ... } as a way to avoid -> Box<dyn ...> . I shall read up on that and see if it help me do things in a better Rust way.

Thanks for all the replies and helping me move forward with learning Rust.

fn do_stuff_2<R>(reader1: R, reader2: impl BufRead ) where R: BufRead {
    for c in reader1.bytes().zip(reader2.bytes()) {
        println!("{} {}", c.0.unwrap(), c.1.unwrap());
    }

:star_struck:

And that was your problem. R: BufRead quite literally say “there's certain type R which implements trait BufRead and we are expecting variable of that specific type here”.

If, in your original example, make two arguments to make the same time everything works (at least compiles):

fn main() {
    let reader2 = get_reader(true);
    let reader1: Box<dyn BufRead> = Box::new(BufReader::new(File::open("test1.txt").unwrap()));

    do_stuff_2(reader1, reader2);
}

And error message talks precisely about that, too.

What rule? The fact that if you promised that you would have two arguments of the same type then they, indeed, have to be of the same type? Does that rule even need an explanation?

Let's forget about our buffered readers, traits and other such things, let's look on the most trivial function imaginable:

fn foo<T>(flag: bool, t1: T, t2: T) -> T {
    if flag {
        t1
    } else {
        t2
    }
}

There are no traits, there are no boxing/unboxing, there are no dynamic dispatch, you just take two arguments and return one.

You can call it like foo(1, 2) or foo(1.0, 2.0), but if you try foo(1, 2.0) then compiler would become confused and would complain. But do you even need “special rule” to understand why? What that rule should even say? If you promised to have two objects of the same type then they, indeed, have to have the same type? Isn't that just logic?

impl Trait in argument position (APIT) is mostly sugar for generics (only you can't name or turbofish the type).

So these are the same for most purposes.

fn do_stuff_2a<R1, R2>(reader1: R1, reader2: R2) where R1: BufRead, R2: BufRead {}
fn do_stuff_2b<R>(reader1: R, reader2: impl BufRead ) where R: BufRead {}
fn do_stuff_2c<R>(reader1: impl BufRead, reader2: R ) where R: BufRead {}
fn do_stuff_2d(reader1: impl BufRead, reader2: impl BufRead) {}
3 Likes

My reading of

fn do_stuff_2<R>(reader1: R, reader2: R) where R is BufRead {
}

Has been to assume the parameters need to be of the same trait. But clearly it is not like that, they have to be of the same trait and same type.

fn do_stuff_2<R1,R2>(reader1: R, reader2: R) where R is BufRead {
}

So now we can have the same trait for our parameters and different types. Sorry to say it was not at all obvious to me. Perhaps my Python and Java background.

Avoiding the short cut which I've just read from: impl Trait - Rust By Example. May have helped me avoid lots of confusion and probably similar to what I'd do in other languages.

fn do_stuff_2(reader1: impl; BufRead, reader2: impl BufRead) {
}

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.