Scoped threadpool + crossbeam channel program does not execute in parallel

[smukherjee17]

Hello all,

I am trying to multithread my program using scoped threadpools and crossbeam channels. I have my things set up as follows:

src/directlighting.rs:
 pub fn integrate(scene: &SceneConfig, start_position_in_film: i32, samples_count: u32, bounces_count: u32) -> Tile;

src/baseintegrator.rs:
use crossbeam::crossbeam_channel::unbounded;
use scoped_pool::Pool;

fn render(scene: &SceneConfig, samples_count: u32, bounces_count: u32) -> Vec<Tile> {
        let mut tiles: Vec<Tile> = vec![];
        let cpus = num_cpus::get();
        info!("Trying with {} cpus", cpus);
        let pool = Pool::new(cpus);

        let film = scene.film.borrow();
        info!(
            "Beginning rendering with {} spp and {} bounces",
            samples_count, bounces_count
        );
        let (s, r) = unbounded();

        let pixel_numbers = 0..(film.height * film.width);
        pool.scoped(|scope| {
            for i in pixel_numbers.step_by(TILE_SIZE) {
                let sender = s.clone();
                scope.execute(move ||
                 match scene.integrator {
                    Integrators::DirectLighting => {
                        let tile: Tile = DirectLightingIntegrator::integrate(
                            scene,
                            i,
                            samples_count,
                            bounces_count,
                        );
                        sender.send(tile).unwrap();
                    }
                   _ => {}
                }
                );

                tiles.push(r.recv().unwrap());
                //warn!("{:?}", tile.pixels);
            }
        });
        tiles
    }

So far, I have modified the render function so that it does not modify anything inside scene so I don't have to take a mutable borrow. However, at this stage, while the code compiles and runs without errors and gives the correct answer, it does not run in parallel. Instead, it seems to do something like a sequential execution of one tile in one thread, followed by another tile in another thread.

It would be great if someone could point me where I'm doing wrong. Thanks in advance!

[cuviper]

You are calling recv in the same loop that queues jobs to execute, which makes you wait for each one before continuing to the next. Try moving that to a separate loop, or even just extend:

for ... {
    let sender = s.clone();
    scope.execute(...);
}
drop(s); // so we don't wait for our own handle
tiles.extend(r);

The channel will receive items in the order they are produced though, which won't necessarily be the natural order when you're processing in parallel. You may want to send i with each tile so you can sort them later.

[smukherjee17]

That was really silly of me, haha. Thank you so much for pointing this out! And yes, I also pass in i in the Tile struct.

It works now, and wow, I am free from headaches of data races just like that. Thanks again!