Hi,
I try to parallelize writing a CSV. For that I loop through the original CSV and spawn threads to write into multiple output CSVs. Essentially the traget file is randomly chosen for now.
In the function that writes to the CSVs I always opened the writer and then wrote a new row. However this is incredibly slow. Thus I wanted to first open all writers to the different output files and just borrow them into the writer functions so they just stay open and safe precious time.
So far I could not figure out which type hint I need.
I started with a simple:
Writer <W>
Here Rust returns "not found in this scope". I then checked the CSV writer documentation. So I tried this based on what I saw:
Writer<W: io::Write>
However then the compiler complains that assoociated types are not allowed in the function definition and that it expected 1 generic argument.
This is how the definitions look like:
fn write_to_file_header(result_filename: &str, groupby_col: &str, count_col: &str, cpu_core: u32) -> Writer<W: io::Write>
--> Creates the writer and writes the header.
and:
fn write_subset_to_csv(filename: &str, groupby_col: &String, count_col: &String, row_idx: &u32, record: &Vec<&str>, cpu_core: u32, total_cores: u32,
counts: &HashMap<String, (i32, Decimal)>, col_indices: &HashMap<String, usize>, stds: &HashMap<String, f64>,
result_filename: &str, writer: Writer<W: io::Write>)
--> Take the writer object and writes a row.
How can the writer be passed as an argument? Or is this an anti-pattern / not possible maybe?