How to use BufReader with File?

Hello,

I don't understand how to use BufReader to read a file.
I use read with slice of u8, it work at start, but when the end of the buffer is reach, the read call return with only a part of wanted content.
I expected that when the end of the buffer is reached, the buffer will be filled with the following part of the file.
It's how read_until works, why read don't work like this ?
And/Or what can I use as an alternative, to read the file with buffer intermediate by determinate size, not until marker ?

I also tested with consume, but don't seems to activate buffer update.

Thank you

The read() function is fairly low level and may read any amount of bytes up to what is needed to fill up the passed in &mut [u8]. I suppose BufReader stops before filling the buffer to avoid an expensive buffer filling operation if all you want to do is pass the read bytes on to something else.

For reading a specific number of bytes, you want read_exact().

3 Likes

Thank you for response.
I have tried read_exact, and as far as I remember read_exact have the same limitation (add the source code seems to confirm).

Please show code. You must be doing something wrong, because read_exact() is guaranteed to fill the entire buffer (as also evidenced by this Playground).

4 Likes

Thank's for your responds.
You Playground don't read all the file, and only read one buffer.
My problem was, than the buffer wasn't updated to read next part of the file.

But I have adapted your playground and it's seems to work : new Playground
I'm going to look at my use case, to update it.

The point of that example was that it reads into a buffer larger than the BufReader's capacity (which is 8192 bytes by default), and it shows that the BufReader will refill its internal buffer as much as needed to read the requested number of bytes when using read_exact() (while read() may stop after exhausting the internal buffer and only refill on the next call).

Your example uses a BufReader capacity the same size as the buffer you are reading into (and it is a quite small size for buffering a file), so there isn't much point in using a BufReader (it will still have to do a syscall to read the file on every read_exact() call). It also errors out at the end before reading the whole file, because the number of bytes in the file isn't a multiple of 35, so you can't read exactly 35 bytes at the end. Here's a fixed playground that uses the default BufReader capacity, reads the file in chunks that divide the file size, and stops at the end of the file. For your use case, differing sizes of reads may be required, and you will need a different buffer for each (the buffer passed to read_exact(), not BufReader's internal buffer).

1 Like

Yes but that wasn't my problem, the point of my playground was than the buffer (of BufReader) is updated to read all file.

I have fixed my problem with read_exact, and I don't know why my first test with it was failing. Probably something else misled me.

And Thank you for your answer jameseb7 and H2CO3.