How to write/replace files atomically?

If you care (... and you may not! :), things are a lot more complicated than it first appears. While rename is operationally atomic, that doesn't necessarily result in atomicity in the face of system reboot or crash.

From this article:

Similarly, if you encounter a system failure (such as power loss, ENOSPC or an I/O error) while overwriting a file, it can result in the loss of existing data. To avoid this problem, it is common practice (and advisable) to write the updated data to a temporary file, ensure that it is safe on stable storage, then rename the temporary file to the original file name (thus replacing the contents). This ensures an atomic update of the file, so that other readers get one copy of the data or another. The following steps are required to perform this type of update:

  • create a new temp file (on the same file system!)
  • write data to the temp file
  • fsync() the temp file
  • rename the temp file to the appropriate name
  • fsync() the containing directory

One of the exiting outcomes (that I debugged in the past year...) if you do not do this dance can be: the destructive rename succeeds, but the resulting file is zero-length.

See also: clarification regarding the robustness of `persist()` · Issue #110 · Stebalien/tempfile · GitHub

Edit: I ran across this bug in ldconfig of all places. Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=18093 ... and the fix: ldconfig: Sync temporary files to disk before renaming them [BZ #20890] · bminor/glibc@999a6da · GitHub

:exploding_head:

5 Likes