Default buffer capacity of BufReader?


#1

Hey there :hugs:

What is the default buffer capacity of BufReader::new?
How many Bytes? :thinking:

Greetings :hugs:


#2

tl;dr 8k

source:

uses


#3

Should I change this if I want to read files up to 20mb line by line? :thinking:


#4

How long is each line? Is reading the file a performance bottleneck? You should benchmark it and see what works well for you.

8kB is probably a bit conservative. But: This is a buffered reader – don’t use it as β€œcopy the whole file into memory.” If reading the whole file up front is an option, you can of course try that as well (just read_to_end into a Vec<u8>), along with memmap.


#5

For a 20MB file I’d just read the whole file in at once, as @killercup mentioned. Unless you have a lot of parallelism (ie multiple procs/threads doing this in parallel) or are in a constrained environment, 20MB is nothing to hold in memory all at once.