How to test functions in a multi-processed way (like `std::thread⁠::spawn`)?

I'm developing a program that uses shared memory as the way of inter-process communications. But I cannot find a good way to test.

Imagine there are two functions (that should be run spontaneously in two individual processes) like this:

// These are the entries of the two subprocess.
fn process1() {}
fn process2() {}

#[test]
fn interprocess_test() {
    // Here `std::process::spawn` is an imaginary function which 
    // creates a subprocess like `std::thread::spawn`.
    let handle1 = std::process::spawn(process1);
    let handle2 = std::process::spawn(process2);
    handle1.join().unwrap();
    handle2.join().unwrap();
}

On Unix, it can be done by fork, like this:

use nix::sys::wait::waitpid;
use nix::unistd::{fork, ForkResult};

fn process1() {}
fn process2() {}

fn spawn_process(f: impl FnOnce()) -> Pid {
    match unsafe { fork() }.expect("Cannot create subprocess") {
        ForkResult::Child => {
            f();
            std::process::exit(0);
        }
        ForkResult::Parent { child } => child,
    }
}

fn join(child: Pid) {
    match waitpid(child, None) {
        Ok(_) => {}
        Err(nix::errno::Errno::ECHILD) => {}
        Err(e) => {
            panic!("Child process exit abnormally: {:?}", e)
        }
    }
}

#[test]
fn interprocess_test() {
    let pid1 = spawn_process(process1);
    let pid2 = spawn_process(process2);
    join(pid1);
    join(pid2);
}

Are there any better ways to do this?

use std::process::Command;

Command::new("sh")
        .spawn()
        .expect("sh command failed to start");

I don't think using std::process::Command is right way.

In each test case, there are only two functions (which is tiny small) to run. But I have to create individual small binaries (in crates or examples?) and then pass the paths to Command::new.

I don't understand stand what you are trying to do. If you want to run a command from a path why not pass it :

Command::new("/path/to/my_program")
        .spawn()
        .expect("sh command failed to start");

I am trying to "spawn" a process in a #[test] function, just like what std::thread::spawn does to spawn a thread.

There is a Unix function fork that does the expected thing, and that's what I do when writing C/C++. I'm here to ask for a Rusty alternative way.

std::process::Command is a feasible way, but not straight-forward enough. And it is not convenient if I'd like to test the same function both in a multi-threaded way and in a multi-processed way.


EDIT: I mean this

// Test in a multi-processed way.
#[test]
fn test_multiprocess() {
    let pid1 = spawn_process(process1);
    let pid2 = spawn_process(process2);
    join(pid1);
    join(pid2);
}

// Test in a multi-threaded way.
#[test]
fn test_multithread() {
    let handle1 = std::thread::spawn(process1);
    let handle2 = std::thread::spawn(process2);
    handle1.join().unwrap();
    handle2.join().unwrap();
}

fork() cannot be soundly used in a multithreaded program (because the child has a single thread, as if all other threads running in the parent were deleted without any cleanup, so locks and other resources might be in inappropriate states), so Rust does not offer fork() in any convenient fashion. (This isn't a rule Rust made up; POSIX says so.)

This isn't a theoretical concern: the Rust test harness uses threads to run your tests in parallel.

Using Command to spawn a known binary is not as simple-looking as fork(), but it is sound. You can use the CARGO_BIN_EXE_<name-of-binary> environment variable to easily obtain the path of any binary defined in your Cargo package, so you don't need any more test setup than actually writing the code for that binary.

I used to program with MPI, so what I think of multi-processing programs might take after the MPI models.

The intention to test by multiprocessing instead of multithreading is not just to run tests in parallel, but that the inter-processing communication (via shared memory on Unix) functionality must be verified to work.

In addition, because I have also to make sure the multiprocessed stuffs also work well in multi-threaded contexts, so I think of fork, which can "spawn" processes similar like what std::thread::spawn to spawn threads. It is the most convenient way to test both multiprocessing and multithreading.

Actually, I only use fork to create the subprocess at the very beginning of the tests, where there have been no resources acquired. Only some configures might be initialized by hard code, or by environment variables. I don't think it is a soundness problem, except that the soundness is guaranteed by human, not by the compiler. Indeed, it is unsafe.

You can fork, but you will have to step outside of the portable functionality of the standard library, and call the fork system call directly. (I believe libc::fork() is the way to do that, but I haven't ever written such a program myself, so there may be better ways.)

And, you must not use Rust's test harness (that calls #[test] functions) because, as I mentioned, it creates threads before your tests start. To write a test without the test harness:

  1. Create a file tests/multiprocess.rs (or whatever file name, as long as it is in the tests/ directory, not src/). This file will be compiled as a separate test binary. It must have an ordinary fn main() {...} function that performs your test; you can still use assert_eq! within it, and other things you might write in a test function.

  2. Put this test target configuration in your Cargo.toml to request disabling the test harness:

    [[test]]
    name = "multiprocess"
    harness = false
    

    (This disables the default test compilation mode in which fn main() is ignored and the test harness is started instead.)

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.