I am new to rust and learning its concurrency. I am trying to implement single writer multi reader with 2 open files, one for writing strictly single thread writer as to guarantee sequence and thread safe reader. Following code is my poc. Please note that RwLock is not what i am after as it blocks all read on write. I would really appreciate if anyone could point a better or faster way to accomplish such requirement.
Thanks in advance.
Cheers
use std::io;
use std::io::prelude::*;
use std::fs::File;
use std::fs;
use std::io::SeekFrom;
use std::str;
use std::thread;
use std::sync::{Mutex, Arc};
fn main() -> io::Result<()> {
// open file for reading
let mut f = File::create("text.txt")?;
f.write(b"initial data inside the file")?;
// read file concurrently
let file = File::open("text.txt")?;
let file_mutex = Arc::new(Mutex::new(file));
let mut threads = vec![];
for i in 0..30 {
// let handle =Arc::clone(&file_mutex);
let handle = file_mutex.clone();
let thread = thread::spawn( move || {
let mut buffer = vec![];
let mut fs = handle.lock().unwrap();
fs.seek(SeekFrom::Start(0)).unwrap();
fs.read_to_end(&mut buffer).unwrap();
println!("from thread {} {}",i,str::from_utf8(&buffer).unwrap());
});
threads.push(thread);
}
// write to file sequentially
for i in 0..1000 {
let x = format!(" {}",&i);
f.write(x.as_bytes())?;
}
// wait for threads to finish
for thread in threads {
thread.join().unwrap();
}
fs::remove_file("text.txt")?;
Ok(())
}
Why is RwLock not what you are after? It will only block if you need to write while there is another writer or other readers, or if you try to read while there is an active writer. You can do many concurrent readers. If you don't want to block, then you can use RwLock::try_* methods to get a non-blocking version.
Thanks for your reply and your pointer to RwLock::try_* the above code already delivers what i expected as no reader can be blocked by anything and reader give a "point in time" view of the file, but write is strictly single threaded, i will implement blocking queue or multiqueue for write. Question was is there a better way? better in more concise or faster or anything for that matter
? but the result is like interleaved amongst threads, like other thread continuing reading from another, even tough i did seek to 0 (run a few times and your will see what i mean)
The reason you are seeing tearing is because you are sharing a single handle, and that handle is being changed across multiple threads. I am a bit confused by your requirements because you are blocking when you use a Mutex. Another reason why the behviour is different is because there is more contention between the threads in the Mutex case, so it slows down the mult-threaded part enough for the writer to write out most of the data. But in the case of no-Mutex, there is very little contention, so the threads don't read all of the data before ending.
Another option is to open the file inside each thread like so, this prevents the threads from messing with file handles that aren't theirs
Coming from Java and being spoiled by filechannel, mutex could simulate the thread safety of synchronized methods that filechannel provides.
Thanks mate.