Getting build.rs to re-run some code *only* when a file has changed

Hi there! I've got a build.rs that compiles some CUDA code with the cc crate, and right now it only gets compiled when it has changed, because I'm using

println!("cargo:rerun-if-changed=src_cuda/vis_gen.cu");

which is great. However, I'd like to use the built crate, which provides build-time variables, like the git hash and the time at which the compiler was invoked. I would like built to generate its variables for every compilation, but because of the above println!, if I haven't edited the CUDA source file recently, the built variables are potentially much older than the Rust code in the crate. (At least, this is my understanding. Clarification is welcome!)

Is it possible to have build.rs run for every compilation, but my CUDA code is compiled only when the specified source file has changed?

If it helps, my file is here: https://github.com/MWATelescope/mwa_hyperdrive/blob/devel/build.rs

And, I asked this question on Reddit, but wasn't able to get a solution: https://www.reddit.com/r/rust/comments/i6yqng/hey_rustaceans_got_an_easy_question_ask_here/g1z3xvu/?context=3

What do you mean by “every compilation”? This is the default for build scripts. If you didn't change any source files/rerun-if-changed files, the crate isn't actually compiled.

Do you mean “every time I run cargo build”?

When your build script runs, it could also check the source file's modification time and compare it to an output file's timestamp. If the output file exists and has been changed more recently than the source file, skip the compilation step.

Another option is to write a Makefile for the CUDA compilation task, and have the build script shell out to make. The advantage is that make handles the timestamp comparisons automatically. The disadvantage is that your crate can now build only on systems with make installed, which may be inconvenient for people building your code on non-Unixy platforms.

Thanks for the suggestion - this gets me close! My current problem seems to be that the modification time isn't correctly picked up (!?). I'm using the code below to get the modified times (std and filetime crate have the same behaviour AFAICT):

// fn get_modified_time<T: AsRef<Path>>(file: &T) -> SystemTime {
//     metadata(file).unwrap().modified().unwrap()
// }
fn get_modified_time<T: AsRef<Path>>(file: &T) -> FileTime {
    FileTime::from_last_modification_time(&metadata(file).unwrap())
}

Here's some compilation output, with cargo warnings littered around to help me debug:

CXX=/opt/cuda/bin/g++ cargo build --release
warning: /home/chj/Software/personal/mwa_hyperdrive/target/release/build/mwa_hyperdrive-d594a365b0d72114/out/libhyperdrive_cu.a
warning: cuda_library_mod_time: FileTime { seconds: 1598410021, nanos: 636274949 }
warning: src_cuda/vis_gen.cu FileTime { seconds: 1598410797, nanos: 634500914 }
warning: FileTime { seconds: 1598410021, nanos: 636274949 } FileTime { seconds: 1598410797, nanos: 634500914 }
warning: compiling cuda!
    Finished release [optimized] target(s) in 0.05s

Because the modification times aren't accurate, sometimes, I will always be "compiling cuda", or not. Is this related to filesystem journaling, or something? If I touch or modify the CUDA file, these mtimes don't update, either.

Aside from all this, it seems that the build-time variables aren't being generated for every invokation of cargo build, even after adding a recursive function:

fn rerun_directory<T: AsRef<Path> + ?Sized>(dir: &T) {
    println!("cargo:rerun-if-changed={}", dir.as_ref().to_str().unwrap());
    // Find any other directories in this one.
    for entry in read_dir(dir).unwrap() {
        let entry = entry.expect("Couldn't access file in src directory");
        let path = entry.path().to_path_buf();
        // Skip this entry if it isn't a directory.
        if !path.is_dir() {
            continue;
        }
        rerun_directory(&path);
    }
}

fn main() {
    rerun_directory("src");
    rerun_directory("src_cuda");
    // ...

For a seemingly small thing, this is quite bothersome! Any ideas? I'd rather not depend on make, but I guess that's looking good right now... I'm not worried about cross-platform compatibility, as this code is intended for linux-only desktops and supercomputers.

Sorry for the confusion - yes, I did mean I want the build.rs to be run every time I do a cargo build. I guess I don't mind if build.rs is not run if no source files have changed, but then I do want the CUDA compilation to be conditional on the CUDA source files having changed.

Can you separate out the CUDA code into another crate? That way you can just leverage Cargo's mechanics to rebuild it when needed.

If you really want to re-run a build script every time, you can do:

use std::{env, fs::File, path::Path};

fn main() {
    let out_dir = env::var("OUT_DIR").unwrap();
    let p = Path::new(&out_dir).join("rebuild_stamp");
    File::create(&p).unwrap();
    println!("cargo:rerun-if-changed={}", p.display());
}

Edit: actually, I can't quite get this to re-run reliably, but I think something like this should work.

Thanks. Yeah, I think it would be tidier overall to have the CUDA code inside a sub-crate or somesuch. I'll explore that, but I don't know how long I will be until I've identified that this has resolved my issue.

Well, I nerd sniped myself and excised the CUDA code from my main crate. Now my build.rs is very clean, and with this code you've posted, it seems to force build.rs to re-run every time! Thanks very much!

For posterity, my full build.rs is below:

// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.

use std::env;
use std::{
    fs::File,
    path::{Path, PathBuf},
};

fn bind_erfa(out_dir: &Path) {
    match pkg_config::probe_library("erfa") {
        Ok(lib) => {
            // Find erfa.h
            let mut erfa_include: Option<_> = None;
            for mut inc_path in lib.include_paths {
                inc_path.push("erfa.h");
                if inc_path.exists() {
                    erfa_include = Some(inc_path.to_str().unwrap().to_string());
                    break;
                }
            }

            bindgen::builder()
                .header(erfa_include.expect("Couldn't find erfa.h in pkg-config include dirs"))
                .whitelist_function("eraSeps")
                .whitelist_function("eraHd2ae")
                .whitelist_function("eraAe2hd")
                .generate()
                .expect("Unable to generate bindings")
                .write_to_file(&out_dir.join("erfa.rs"))
                .expect("Couldn't write bindings");
        }
        Err(_) => panic!("Couldn't find the ERFA library via pkg-config"),
    };
}

// Use the "built" crate to generate some useful build-time information,
// including the git hash and compiler version.
fn write_built(out_dir: &Path) {
    let mut opts = built::Options::default();
    opts.set_compiler(true)
        .set_git(true)
        .set_time(true)
        .set_ci(false)
        .set_env(false)
        .set_dependencies(false)
        .set_features(false)
        .set_cfg(false);
    built::write_built_file_with_opts(
        &opts,
        env::var("CARGO_MANIFEST_DIR").unwrap().as_ref(),
        &out_dir.join("built.rs"),
    )
    .expect("Failed to acquire build-time information");
}

fn main() {
    let out_dir = PathBuf::from(env::var("OUT_DIR").expect("OUT_DIR env. variable not defined!"));
    let p = Path::new(&out_dir).join("rebuild_stamp");
    File::create(&p).unwrap();
    println!("cargo:rerun-if-changed={}", p.display());

    write_built(&out_dir);
    bind_erfa(&out_dir);
}

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.