Getting stream_position with BufWriter flushes buffer

help
1

Hi!

I’m having a similar issue then the one reported on `BufReader::Seek` should not flush the buffer when seeking to `SeekFrom::Current(0)` · Issue #86832 · rust-lang/rust · GitHub, but with BufWriter instead of BufReader.

It looks like that looking at the stream position flushes the buffer, which doesn’t seem necessary. I wanted to use f.stream_position() to check if a seek is necessary or not when writing bytes at different locations.

An example:

use std::error::Error;
use std::io::{BufWriter, Write, Seek};
use std::fs::OpenOptions;

fn main() -> Result<(), Box<dyn Error>> {
    let mut file_options = OpenOptions::new();
    file_options.read(true).write(true).create(true);
    let f = file_options.open("test.out")?;
    let mut f = BufWriter::new(f);

    let buffer = [61u8; 1];

    f.write(&buffer)?;
    println!("buffer len: {}", f.buffer().len());
    f.write(&buffer)?;
    println!("buffer len: {}", f.buffer().len());
    f.write(&buffer)?;
    println!("buffer len: {}", f.buffer().len());

    let _ = f.stream_position();
    f.write(&buffer)?;
    println!("buffer len: {}", f.buffer().len());
    let _ = f.stream_position();
    f.write(&buffer)?;
    println!("buffer len: {}", f.buffer().len());

    Ok(())
}

It prints

buffer len: 1
buffer len: 2
buffer len: 3
buffer len: 1
buffer len: 1

But I would have expected the last ones to be 4 and 5, as we are still writing at consecutive locations.

2

This comment seems relevant:

I wonder if that’s still an option (I wonder if anyone is relying on the implicit flushing of this).

The documentation is somewhat contradictory anyway, depending on your understanding of “equivalent” in the docs here

Returns the current seek position from the start of the stream.

This is equivalent to self.seek(SeekFrom::Current(0)).

because of course, the “optimization” in BufReader’s implementation does change the behavior slightly (and documents it in the specific trait impl), so it’s not fully equivalent anymore.

3

Thanks for the link.

I was able to find a better solution by reading the comments: if you keep a reference to the original file, and call stream_position() on it, it doesn’t flush the buffer. It is still a bit strange, but it will work for my use case.

use std::error::Error;
use std::io::{BufWriter, Write, Seek};
use std::fs::OpenOptions;

fn main() -> Result<(), Box<dyn Error>> {
    let mut file_options = OpenOptions::new();
    file_options.read(true).write(true).create(true);
    let mut f_origin = file_options.open("test.out")?;
    let mut f_buffered = BufWriter::new(f_origin.try_clone()?);

    let buffer = [61u8; 1];
    
    f_buffered.write(&buffer)?;
    println!("buffer len: {}", f_buffered.buffer().len());
    f_buffered.write(&buffer)?;
    println!("buffer len: {}", f_buffered.buffer().len());
    f_buffered.write(&buffer)?;
    println!("buffer len: {}", f_buffered.buffer().len());

    let _ = f_origin.stream_position();
    f_buffered.write(&buffer)?;
    println!("buffer len: {}", f_buffered.buffer().len());
    let _ = f_origin.stream_position();
    f_buffered.write(&buffer)?;
    println!("buffer len: {}", f_buffered.buffer().len());

    Ok(())
}

gives

buffer len: 1
buffer len: 2
buffer len: 3
buffer len: 4
buffer len: 5
4

If you want to consider the buffer contents for a more accurate stream_position, you can probably just mirror the current BufReader implementation, e.g. as follows

use std::fs::File;
use std::io;
use std::io::BufWriter;
use std::io::Seek;

fn my_stream_position(x: &mut BufWriter<File>) -> io::Result<u64> {
    let buffer_len = x.buffer().len();
    x.get_mut().stream_position().map(|pos| {
        pos.checked_add(buffer_len as u64)
            .expect("overflow when adding buffer size to inner stream position")
    })
}
5

Ah yes, I had forgotten that the stream_position() from the original file doesn’t count the bytes not written yet, and still in BufWriter.

Thanks for the function, it will be very helpful.

6

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.