I have written a blog post about something that I learned, maybe can be useful to someone else too.
This Is completely off-topic, I love the font you used for the titles. <3
You could have called Stdout::lock inside the rayon closure, and written to the returned guard.
Thanks @RustyYato ! .The fact is that I need to handle the initialization formatting, i support csv and json at the moment (pardi/src/presenter.rs at main · edap/pardi · GitHub).
Probably I need to specify this in the post that I have this special need
If you mean writing the header, then you could also use std::sync::Once for that.
The header, the footer, and I need to skip the trailing comma for the first record.
I think you definitely oversimplified the blogpost then ![]()
The article still has some very valid points like the explanation of how closures capture values, but using an additional wrapper struct doesn't make much sense in the context of that small example.
I also want to note that if your goal is to just make the code compile there's a one-character wrong fix: just do writeln!(&stdout instead of writeln!(stdout. This is wrong because it might output lines from different threads mixed together, but it might be interesting for you to understand why/how it compiles.
Why? because of two problems, the first one is the more obvious one, we are breaking the Exclusive Mutability Rule. That is, for any given value, you can have either one mutable reference
(&mut T)or any number of shared references(&T)at a time, but never both simultaneously. the macrowriteln!requires a mutable reference&mut stdoutto write data, meaning only one thing can write to it at a exact time.The second problem is more complex and it is related to how this closure captures the
stdoutvariable declared outside of its scope.
This is actually the same problem. The closure has to capture stdout by mutable reference, but it is forced to be a Fn by rayon exactly to avoid the Exclusive Mutability Rule.
Many thanks @SkiFire13 , you are probably right as @RustyYato was pointing out, if I omit that part about the formatting, there is an easier path. I will add it, at least a simplified version.
Regarding the "two" problems, in my way of understanding things, they were separated at the beginning, they get a "unified theory" at the end. Exclusive mutability and how closures works are related, but I need to understand both to get a full picture. It took me a bit to understand the error message, as it sounded like two different things. Rayon wanted Fn and therefore a shared ref, but writeln! wanted a mutable reference. Conceptually to me, they were two different problems, it took me a moment to get them under the same umbrella (the table in the post). Maybe I could write that part better.
i am not entirely sure what's the point of parallelizing the access to standard output tho my personal approach would be to have multiple threads generate the content to output and then share them with a thread that is dedicated to output it. this way you can severely reduce contention. otherwise contending on standard output will mean that you have content generation stuck waiting for stdout and stout waiting for content generation all the time
My application spends most of the time parsing DICOM files. Writing a single JSON line or CSV row takes microseconds, while parsing takes milliseconds. Because worker threads spend almost all their time parsing independently and only fractions of a microsecond holding the lock while writing, I don't think this applies in my case, but correct me if I am wrong.
yeah if your io is really minor compared to compute you don't see that much difference. tho i still consider it a bit better to have the parsing threads dedicate 100% of the time to parsing and just push strings to and output thread. only drawback migt be memory usage if you end up holding a lot of data before being able to push it out