Single writer multi reader file

Hi,

I am new to rust and learning its concurrency. I am trying to implement single writer multi reader with 2 open files, one for writing strictly single thread writer as to guarantee sequence and thread safe reader. Following code is my poc. Please note that RwLock is not what i am after as it blocks all read on write. I would really appreciate if anyone could point a better or faster way to accomplish such requirement.
Thanks in advance.

Cheers

use std::io;
use std::io::prelude::*;
use std::fs::File;
use std::fs;
use std::io::SeekFrom;
use std::str;
use std::thread;
use std::sync::{Mutex, Arc};

fn main() -> io::Result<()> {
    // open file for reading
    let mut f = File::create("text.txt")?;
    f.write(b"initial data inside the file")?;
    // read file concurrently
    let file = File::open("text.txt")?;
    let file_mutex = Arc::new(Mutex::new(file));
    let mut threads = vec![];
    for i in 0..30 {
        // let handle =Arc::clone(&file_mutex);
        let handle = file_mutex.clone();
        let thread = thread::spawn( move || {
            let mut buffer = vec![];
            let mut fs = handle.lock().unwrap();
            fs.seek(SeekFrom::Start(0)).unwrap();
            fs.read_to_end(&mut buffer).unwrap();
            println!("from thread {} {}",i,str::from_utf8(&buffer).unwrap());
        });
        threads.push(thread);
    }
    // write to file sequentially
    for i in 0..1000 {
        let x = format!(" {}",&i);
        f.write(x.as_bytes())?;
    }
    // wait for threads to finish
    for thread in threads {
        thread.join().unwrap();
    }
    fs::remove_file("text.txt")?;
    Ok(())
}

(Playground)

Why is RwLock not what you are after? It will only block if you need to write while there is another writer or other readers, or if you try to read while there is an active writer. You can do many concurrent readers. If you don’t want to block, then you can use RwLock::try_* methods to get a non-blocking version.

Exactly what behaviour do you expect?

Hi Krishna,

Thanks for your reply and your pointer to RwLock::try_* the above code already delivers what i expected as no reader can be blocked by anything and reader give a “point in time” view of the file, but write is strictly single threaded, i will implement blocking queue or multiqueue for write. Question was is there a better way? better in more concise or faster or anything for that matter

Cheers

Note that Read and Write are implemented for &File, which means that you don’t need to use a Mutex to syncronize files.

Hi Krishna,

you mean like
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=95bf35bf732ec1a0d4306feca058545d
? but the result is like interleaved amongst threads, like other thread continuing reading from another, even tough i did seek to 0 (run a few times and your will see what i mean)

Cheers

The reason you are seeing tearing is because you are sharing a single handle, and that handle is being changed across multiple threads. I am a bit confused by your requirements because you are blocking when you use a Mutex. Another reason why the behviour is different is because there is more contention between the threads in the Mutex case, so it slows down the mult-threaded part enough for the writer to write out most of the data. But in the case of no-Mutex, there is very little contention, so the threads don’t read all of the data before ending.

Another option is to open the file inside each thread like so, this prevents the threads from messing with file handles that aren’t theirs
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=7bcd7b9edc7963d40eaf77bfa1c6a428

Thanks. what i meant by blocking was more about multi thread safety, not actual the actual io blocking

Cheers

Mutex is multi thread blocking

Yup.

Coming from Java and being spoiled by filechannel, mutex could simulate the thread safety of synchronized methods that filechannel provides.
Thanks mate.

Cheers

1 Like