Default buffer capacity of BufReader?

help
1

Hey there :hugs:

What is the default buffer capacity of BufReader::new?
How many Bytes? :thinking:

Greetings :hugs:

2

tl;dr 8k

source:

https://github.com/rust-lang/rust/blob/ba2741594ba2d4de2ad8cea69def6408ff5409af/src/libstd/io/buffered.rs#L74

uses

https://github.com/rust-lang/rust/blob/6ccfe68076abc78392ab9e1d81b5c1a2123af657/src/libstd/sys_common/io.rs#L10

2 Likes
3

Should I change this if I want to read files up to 20mb line by line? :thinking:

4

How long is each line? Is reading the file a performance bottleneck? You should benchmark it and see what works well for you.

8kB is probably a bit conservative. But: This is a buffered reader -- don't use it as "copy the whole file into memory." If reading the whole file up front is an option, you can of course try that as well (just read_to_end into a Vec<u8>), along with memmap.

1 Like
5

For a 20MB file Iā€™d just read the whole file in at once, as @killercup mentioned. Unless you have a lot of parallelism (ie multiple procs/threads doing this in parallel) or are in a constrained environment, 20MB is nothing to hold in memory all at once.

1 Like