Using cargo chef in gitlab's CI/CD

help
1

I have the following Dockerfile to create my binary

# Build stage
FROM rust:slim-buster as builder

RUN apt-get update && \
  apt-get install -y pkg-config make g++ libssl-dev cmake libmariadb-dev-compat zlib1g && \
  rustup target add x86_64-unknown-linux-gnu

WORKDIR /var/www/app

COPY . .

# RUN cargo build --release --bin app # takes a longer compile time cause of optimization. This should be used for deployment time in 2025
RUN cargo build --release --bin app

# Prod stage, removing the Rust toolchain
FROM rust:slim-buster

COPY --from=builder /var/www/app/config /config
COPY --from=builder /var/www/app/.env /.env
COPY --from=builder /var/www/app/target/release/app /

EXPOSE 8088

CMD ["./app"]

How can I use cargo chef in order to cache the compiled crates

2

Anyone who can help me?

3

It's not clear to me what you're needing help with. The gitlab side, docker side, or cargo chef parts.

I don't use cargo chef personally, but I have a shell script which does something similar to speed up gitlab builds.

This also involves setting up a docker container registry and pushing the cached build image.

4

My goal is that I don't want to recompile all dependencies every time I build a Docker image in gitlab, as long as they don't change. Something like a cache that takes all pre-compiled dependencies compared to the Cargo.toml.

5

I make use of a self hosted gitlab instance with a container registry which I can upload base images used for building. Setting that up is a bit of work and might be project specific.

I can't walk you though it, but here are some resources I found helpful when working through this myself.

Most of the links above have examples for a single crate. Here's a starting point if you need to do something similar with a workspace. This script would need to be used in conjunction with a Dockerfile. For example in your where you do COPY . . you would instead do something like COPY fake-project-root .

#!/bin/bash
set -xe
FAKE_PROJECT_ROOT="${PWD}/ci/fake-project-root"

# reset the fake project root
rm -rf "${FAKE_PROJECT_ROOT}"
mkdir -p "${FAKE_PROJECT_ROOT}"

# this will break on file paths with spaces, but is more readable then the alternative
for CARGO_TOML in $(find . \( -name Cargo.toml -o -name Cargo.lock \) -not -path './ci/*'); do
  PACKAGE_DIR=$(dirname "${CARGO_TOML}")

  # create the parent directories
  mkdir -p "${FAKE_PROJECT_ROOT}/${PACKAGE_DIR}/src"

  # create lib.rs and main.rs so that cargo commands will succeed
  echo 'pub fn main() { }' >"${FAKE_PROJECT_ROOT}/${PACKAGE_DIR}/src/main.rs"
  touch "${FAKE_PROJECT_ROOT}/${PACKAGE_DIR}/src/lib.rs"

  # copy each Cargo.toml file
  cp "${CARGO_TOML}" "${FAKE_PROJECT_ROOT}/${CARGO_TOML}"
done
1 Like
6

I switched over from using my custom scripts to cargo-chef with gitlab so I thought I'd share a couple pointers in case you never resolved your issue.

The first thing I'd recommend being aware of is the environment variable CARGO_LOG=cargo::core::compiler::fingerprint=trace.

Setting this environment variable will be helpful in debugging why cargo caching is or isn't working properly. The output is a bit hard to understand, but reading this page may help.

Next thing to take note of is this limitation with cargo-chef noted in the github readme:

cargo chef cook and cargo build must be executed from the same working directory. If you examine the *.d files under target/debug/deps for one of your projects using cat you will notice that they contain absolute paths referring to the project target directory. If moved around, cargo will not leverage them as cached dependencies;

In the context of gitlab, this has the consequence that you need to play some tricks with how things are built if you want to exploit caching. At least in my environment the gitlab build directory is /builds/project/repository.

If you make a docker image which builds from this location and use it with the standard mechanism of setting the gitlab docker image in the .gitlab-ci.yaml you will also encounter the additional problem that the directory is removed before the repository is cloned to it.

One solution is to use cargo-chef in a docker file with the build directory / working directory set to /builds/project/repository. After building move the cached target directory to another location to avoid issues with it being cleaned by the gitlab environment. Finally, during your build process copy the directory from the cached location to the normal target directory.

If you're making use of docker-in-docker in gitlab ci you might not encounter the problems described because you won't be running from the build path gitlab has setup for you.

1 Like