GitLab CI cache problem

help
1

For some reason I can't make rust not recompile everything everytime with cache.

Here's my .gitlab-ci.yml:

image: rust

variables:
  CARGO_HOME: $CI_PROJECT_DIR/cargo

cache:
  paths:
    - target
    - cargo

test:
  script:
    - rustc --version && cargo --version
    - ./target/debug/rust-cache-test || true
    - cargo run --verbose

I've setup this simple repository: https://gitlab.com/kuviman/rust-cache-test

This job has artifacts from previous build, no changes made since, yet it still recompiles.

Does anyone have a working GitLab CI rust cache?

2

I have a similar problem with a project of a different language.

As far as I was able to analyze it, this is due to how the dates of the files are checked.

The unzipped files from the cache all have their proper timestamps, but the checked out files from the git repository all have a current creation timestamp and therefore appear to be newer than their cached artifacts to the compiler.

I have not found a way around this. At least for my project the cache worked for the artifcats of dependencies, which solved most of my compile time issues.

For my rust project I have not yet tried to set up caching.

3

In my real project I see external dependencies being recompiled often, not only code in the repository.

It works really strange, sometimes cache works for them, and sometimes it doesn't.

4

Sounds like what needs to happen is to tarball the artifacts after the build, and extract them at the start.

1 Like
5

That's what caching already does.

6

That caching isn't preserving file metadata, however. That's what tarballing can solve. We use the same approach for debian packaging.

1 Like
7

The documentation is quite clear that there no guarantee is made on cache availability (It surprised me at first too)

Caching is an optimization, but isn't guaranteed to always work, so you need to be prepared to regenerate any cached files in each job that needs them.

Assuming you have properly defined cache in .gitlab-ci.yml according to your workflow, the availability of the cache ultimately depends on how the Runner has been configured (the executor type and whether different Runners are used for passing the cache between jobs)

8

Cache was available during this job. You can see that cached artifacts from previous build are present, but rust still decides to recompile. Cache availability is not the problem here.