Overwriting entire file Cross platform

help
1

So I had some simple logic for overwriting a file with random bytes


let file_size = std::fs::metadata("file_path")?.len();

// create random number generator
let mut rng = rand::rng();

//open the file and give write permissions
let mut f = std::fs::OpenOptions::new().write(true).open(path_as_str)?;

//for each byte in the file, overwrite it with a random byte
for i in 0..file_size {
    let rand_byte: u8 = rng.random();
    f.write_at(&[rand_byte], i)?;
}

f.flush();

So this works great as long as the program is for unix, however this won't work for windows.
I've gathered that the correct way of making this cross platform is by using std::io::write::write_all

simplest example i could find:

let mut f = std::fs::OpenOptions::new().write(true).open("./file")?;
f.write_all(b"XXX")?;
f.flush()?;

This works all fine, but i'm having trouble finding out how i could overwrite the entire file length with random bytes using write_all, to be clear I don't want the file to be truncated, I want the entire file to be overwritten so it preserves the filesize.

Thank you, I hope it's clear.

2

Looks like you are trying to implement shred

Please note this information from the shred manpage.

CAUTION: Note that shred relies on a very important assumption: that the file system overwrites data in place. This is the traditional way to do things, but many modern file system designs do not satisfy this assumption.
1 Like
3

Aside from the potential for efficiency improvements, is there any reason that something like this wouldn't meet your needs?

let file_size = std::fs::metadata("file_path")?.len();
let mut rng = rand::rng();
let mut f = std::fs::OpenOptions::new().write(true).open("file_path")?;
for i in 0..file_size {
    let rand_byte: u8 = rng.random();
    f.write_all(&[rand_byte])?;
}
f.flush()?;
1 Like
4

Writing one byte at a time is very slow. You should write a whole buffer at a time.

Something like this (untested):

fn overwrite_internal(
    path: &Path,
    buf: &mut [u8],
    mut fill: impl FnMut(&mut [u8]),
) -> io::Result<()> {
    let mut file = std::fs::OpenOptions::new().write(true).open(path)?;
    let file_size = file.metadata()?.len();
    let mut bytes_written = 0;
    while bytes_written < file_size {
        let to_write = usize::try_from(min(
            u64::try_from(buf.len()).unwrap(),
            file_size - bytes_written,
        ))
        .unwrap();
        fill(&mut buf[..to_write]);
        file.write_all(&buf[..to_write])?;
        bytes_written += u64::try_from(to_write).unwrap();
    }
    file.flush()?;
    Ok(())
}

const BUF_SIZE: usize = 4 * 1024;

pub fn overwrite_random(path: &Path) -> io::Result<()> {
    let mut buf = [0; BUF_SIZE];
    let mut rng = rand::rng();
    overwrite_internal(path, &mut buf, |b| rng.fill(b))
}

pub fn overwrite_pattern(path: &Path, pattern: u8) -> io::Result<()> {
    let mut buf = [pattern; BUF_SIZE];
    overwrite_internal(path, &mut buf, |_| {})
}
2 Likes
5

A reasonable minimum buffer size for file I/O is 64 KiB.

1 Like
6

@stonerfish I know of shred and it's rust implementation but this is a small thing i'm doing mostly for learning

That was what i was going for, thanks, got stuck!

I will probably implement @CodesInChaos buffer solution though.

1 Like