`Read`ing: whose job is it to buffer?

I'm currently using @BurntSushi's ace csv library. All the examples I have read don't mention buffering the input, so I assume that any buffering that needs to take place for performance happens inside the library. However, this kinda clashes with my intuition that it's the responsibility of the user of the library to handle performance concerns like buffering.

So my question is: whose job is it to buffer, is it:

  • the library that will be calling read often,
  • the library consumer,
  • sometimes one, sometimes the other?

EDIT found a quote in the library docs:

Note that we do not wrap the File in a buffer. The CSV reader does buffering internally, so there's no need for the caller to do it.

It's plausible that csv probably should be using io::BufRead instead of io::BufReader. IIRC, a while back, it used to use its own custom buffering internally, but at some point it got simplified such that it could just use the standard io::BufReader, and I probably never re-thought the API.

Accepting an io::Read is more convenient, so I think those APIs can probably remain the same. But adding an alternative constructor that accepts an io::BufRead instead would be useful. I think this could be done backwards compatibly, but I'm not 100% sure.

The other part of this is that you probably also need to provide APIs for getting the buffer back out of the reader, like Reader::into_inner, so that the caller can reuse the buffer somewhere else. That's probably also where the compatibility hazard for csv would lie. I don't think into_inner could be made to continue to return the underlying io::Read implementation. Instead, it would need to return its explicit io::BufReader instead.

In the grand scheme of things, this probably doesn't matter too much.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.