Announcing Cranko, a release automation tool

announcements
#1

I'm very happy to announce a beta release of Cranko, a release automation tool that implements a new “just-in-time versioning” workflow. It is cross-platform, installable as a single executable, supports multiple languages and packaging systems, and is designed from the ground up to work with monorepos. Quick links:

This announcement is going to be a bit lengthy because Cranko introduces some new ideas (at least, new compared to the prior art I've researched) and they're important for understanding how Cranko differs from other similar tools.

Motivation

For at least a decade, I've been unhappy with my software release workflows.

Sure, I've made releases and things have worked OK, but several of the basic mechanics of the release process have felt awkward to me. To make new releases, I'd generally make a commit bumping the version number in Cargo.toml (or equivalent), push it to CI, wait for verification that the commit builds, do my release activities, then make and push another commit advancing the version to some sort of beta/development value. Not the worst thing, but it's a very manual process, and if something goes wrong you have to go back and force-push branches, force-recreate Git tags, and so on. It feels like I'm using my tools wrong.

This year I got into a project (WorldWideTelescope/wwt-webgl-engine) that I knew was going to be very monorepo-y. It was going to be a huge pain to work with this project without really good release automation, so I sat down and really thought about what was bothering me and how to fix it. That work has led to Cranko. Depending on your mindset, the new approach offered by Cranko might not sound like a big deal — but if you like rigor and have been bothered by your release workflows too, it might be just what you need. For me, it feels frankly revolutionary.

The problem and the solution

I claim that there is a fundamental issue that leads to awkward patterns such as the one I sketched above. The issue is that making a release is fundamentally an action that operates on a repository: I tag a certain commit as identifying version 1.0.17, and I create release artifacts by processing that commit. However, the existence of files like Cargo.toml forces you to embed this information in the repository, which leads to circularity: if I'm being rigorous, I can only know that commit Q "is" version X.Y.Z after running CI on it, but I can't run CI on it without creating and publishing a commit that labels the project files as having that version.

As another example, the practice of bumping versions to some "development number" exists because the existence of a Cargo.toml-like file forces you to assign a version number to every commit, whereas really only some of those commits correspond to versioned releases.

I further claim that there's really only one good way to avoid this circularity: if you need to create a commit that embeds some sort of repository metadata in the repository content (e.g., "this commit is version 1.0.17"), the way to do it is to have a main branch that does not include any of this metadata, plus a tool that mechanically inserts that metadata into a derived branch. Cranko is such a tool, specialized to release metadata.

Just-in-time versioning

What does this mean in practice? While Cranko provides some generalized release-automation tools, its raison d'être is a just-in-time versioning workflow derived from the above diagnosis. It works thusly:

  • Our unit of consideration is one repository, which contains one or more projects, where a “project” is something with version numbers and releases. A repo with more than one project is, er, a monorepo. Cranko knows about the projects in your repo and their release histories.
  • On the main development branch, all projects are assigned a fixed unchanging version number. Cranko uses 0.0.0-dev.0 for semver projects.
  • To make a release, you use Cranko to create a special “release request” commit on a branch called rc, which is then pushed to CI.
  • Your CI acquires a new first step: invoke Cranko to rewrite your metafiles with their actual version numbers.
  • If there is an update to the rc branch and CI passes, the release request has succeeded. The rewritten files are committed to a branch called release, which is pushed to the upstream repository along with Git tags, etc. Other release automation happens as needed. Cranko knows the release histories of your projects by analyzing the history of this release branch.

All of the above is written as if your repo contains exactly one project. For a monorepo, things get more complicated because projects have internal dependencies, and you need to be able to express the version requirements between different projects on the main branch, where no actual version information is available. Cranko handles this. Importantly, each release request on rc may request the release of some arbitrary subset of all of the projects in the repo, so long as their internal version requirements are self-consistent.

Cranko in practice

OK, what does this actually look like in practice? To make a release of one of the packages in WorldWideTelescope/wwt-webgl-engine, all I have to do is:

$ cranko stage @wwtelescope/engine-types
@wwtelescope/engine-types: 25 relevant commits

$ {edit engine-types/CHANGELOG.md to specify version 
   bump type and write up release notes}

$ cranko confirm
info: @wwtelescope/engine-types: micro bump (expected: 0.1.0 => 0.1.1)
info: staged rc commit to `rc` branch

$ git push origin rc

The CI processing runs and (if it succeeds) automatically creates:

  • Git tags
  • GitHub releases
  • NPM package uploads

Cranko's release automation tools are fairly standard, but build on Cranko's knowledge of the project graph and which projects are getting new releases.

Finally, the release branch is updated. Back in my developer tree:

$ git fetch origin
[...]
   9fa82ad..8be356d  release      -> origin/release
 * [new tag]         @wwtelescope/engine-types@0.1.1 -> @wwtelescope/engine-types@0.1.1

$ cranko status
@wwtelescope/astro: 1 relevant commit(s) since 0.0.5
@wwtelescope/engine-types@: 0 relevant commit(s) since 0.1.1
...

There is a cranko bootstrap command to help you start using Cranko in a preexisting repository.

Example Implementations

  • CI files and logs for elfx86exts, a simple Rust program without any monorepo fanciness
  • Cranko's CI files and logs
  • CI files and logs for Tectonic, showing multi-platform builds, GitHub releases with artifacts, and GitHub pages updates

(Even) more information

See the book:

Thanks for reading this far! So far I’m feeling as if Cranko is really transforming how I do releases, so I hope you find it useful too.

10 Likes
#2

"Cranko". Best. Name. EVER.

2 Likes
#3

I agree that versioning in both Cargo.toml and git seems redundant. I've used a similar approach where the Cargo.toml always contains 0.0.0, and the versioning is done exclusively through git tags. In that setup, the only thing I need to do to make a release is to push a new git tag, and the CI script will edit Cargo.toml and publish the crate with the version specified in the tag. The tags are placed on the commits from the main branch, so they still appear in git log of the main branch, but without redundant extra "version bump" commits.

In your implementation, it looks like releases are not visible on the main branch, which can be a downside. A typical use case is that I discover a bug in some crate, go to its repository and look what changed since the last release. Or if a particular version is bugged, I'd like to see what changed in that version compared to the previous version. In these cases, looking at the history of the main branch is usually the simplest thing to do.

There is another downside to having 0.0.0 in Cargo.toml: cargo patches won't work because cargo will detect a version mismatch. You can work around this by editing Cargo.toml of your patched version manually, so it's more of a minor inconvenience.

1 Like
#4

Yeah, I started out planning to do something along these lines. It ends up being a bit limited in the monorepo case where you have many independently versioned projects, though: the CI implementation complexity would get pretty high as the number of tag names and projects goes up, and there's no mechanism to release a group of projects all at once. Since Cranko models the project graph as being acyclic it is always possible to break a group release into a bunch of individual releases, but efficiency-wise it can be nice to batch things.

The other thing is that if releases are triggered with tag pushes, you can't really include any metadata in the release request. In Cranko, the proposed release notes / changelog entry are essentially part of the release request, which allows Cranko to do things like include them in the GitHub release description. You could probably use a heuristic for that in a tag-based model, though.

Finally, there's the issue that if you push a tag and the release CI fails, you have to go back and force-update the tag. I fully recognize that in practice people are happy to do this, but my intuition tells me that the right design is one in which this isn't necessary.

Yeah, it's not ideal. At the moment I've adopted a convention of having the main-branch changelog point people to the one on the release branch (example). Between that and the GitHub release history, my hope is that people won't have any trouble finding the info they need.

That's a good point! But yeah, I tend to think that it's not a huge problem.

1 Like