Threads and static lifetime

The below code fails to compile with the following error:

  --> src/lib.rs:33:32
   |
33 |             let worker_jobs = &jobs[(i - 1) * chunksize..i * chunksize - 1];
   |                                ^^^^ borrowed value does not live long enough
34 |             let handler = thread::spawn(move || for worker_job in worker_jobs {});
   |                           ------------------------------------------------------- argument requires that jobs is borrowed for 'static
...
38 |     }
   |     - `jobs` dropped here while still borrowed`

I understand why Rust is complaining here. But how can I cleanly solve this issue?

use std::cmp;
use std::thread::{self, JoinHandle};

use pyo3::prelude::*;
use pyo3::Python;

#[pyclass]
pub struct Pool {
    children: Vec<JoinHandle<()>>,
    max_children: usize,
}

#[pymethods]
impl Pool {
    #[new]
    fn __new__(max_children: Option<usize>) -> PyResult<Self> {
        Ok(Pool {
            children: Vec::new(),
            max_children: match max_children {
                Some(v) => cmp::max(v, 1),
                None => usize::MAX,
            },
        })
    }

    fn map(&mut self, jobs: Vec<u8>) -> PyResult<bool> {
        let job_cnt = jobs.len();
        if job_cnt == 0 {
            return Ok(true);
        }
        let chunksize = cmp::max(job_cnt / self.max_children, 1);
        for i in 1..self.max_children {
            let worker_jobs = &jobs[(i - 1) * chunksize..i * chunksize - 1];
            let handler = thread::spawn(move || for worker_job in worker_jobs {});
            self.children.push(handler);
        }
        Ok(true)
    }
}

#[pymodule(foo)]
fn init(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add("__version__", env!("CARGO_PKG_VERSION"))?;
    m.add_class::<Pool>()?;
    Ok(())
}

The crossbeam crate has some tools for sending jobs to threads.

I'd just use rayon and ignore all those threads and synchronizations.

https://docs.rs/rayon/1.3.1/rayon/

A fix for the issue with minimal changes would be

        let chunksize = cmp::max(job_cnt / self.max_children, 1);
        let mut jobs = jobs.into_iter();
        for i in 1..self.max_children {
            let worker_jobs: Vec<_> = jobs.take(chunksize).collect();
            let handler = thread::spawn(move || for worker_job in worker_jobs {});
            self.children.push(handler);
        }

Agreed that rayon will likely be easier though.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.