Best (efficient) way to join Paths

1

Given a set of path components, PathBuf potentially provides a couple ways to join them.

Method 1. Use PathBuf::push with a mut variable:

fn method_1(root: impl AsRef<Path>) -> PathBuf {
  let mut path = root.as_ref().to_owned();
  path.push("directory_a");
  path.push("file_b.whatever");
  path
}

Method 2. Use PathBuf::join with an immutable variable:

fn method_1(root: impl AsRef<Path>) -> PathBuf {
  let path = root.as_ref().join("directory_a").join("file_b.whatever");
  path
}

Method 3. Use an iterator. Actually, I'm not sure how to write this in my example given the heterogeneous types of the components, but I see examples like node.js - How to get path.join in Rust? - Stack Overflow on Stack Overflow.

I'm a bit reluctant to use Method 2 because I'm not sure the compiler will optimize the allocation away. Strictly speaking, it creates a new PathBuf for each join call. If I were to make this longer by adding more directory components, then the cost could go up substantially.

Is there a best practice on how to do this? It just seem a bit unfortunate that I need to use mutable APIs to avoid wasteful allocations. If this were C++, with the right set of move constructors and so on, I think it would actually be possible to do this in a zero-cost way, but I'm not sure Rust does that (ironically).

Thanks in advance.

1 Like
2

It's impl Add for String. I don't know why that doesn't exist for PathBuf. Write a trivial extension trait:

trait PathBufExt {
    fn joined(self, suffix: &Path) -> PathBuf;
}

impl PathBufExt for PathBuf {
    fn joined(mut self, suffix: &Path) -> PathBuf {
        self.push(suffix);
        self
    }
}
4 Likes
3

Nice example

4

Strictly speaking, the mutable case can reallocate on each push call, and the allocation is probably by far the most expensive part of the PathBuf creation anyway, so it's likely that you won't see any actual difference. We can try and measure this, of course.

4 Likes
5

Are there other ways to do this, e.g. that result in a single allocation?

The two options I presented in my post are just that: two ways I know how to do it. They're not by any means intended to be definitive. If Rust has an idiomatic, zero-cost abstraction for efficiently concatenating paths, I'd love to hear about it.

6

Re-allocation, even for super long paths, from push operations should benefit from the same kind of exponential growing strategy that makes pushing to Vec (or String) efficient (amortized constant time complexity). Realistically, I cannot imagine how path concatenation could ever be in code performance critical enough so that this (constant factor) overhead over allocating the right amount to begin with would be relevant.

As for convenience and avoiding handling a mutable variable yourself, one option I haven’t seen mentioned yet, and which is equivalent to method_1 (as it’s literally implemented as .push in a loop) is to use from_iter:

fn method_1_alt(root: impl AsRef<Path>) -> PathBuf {
    PathBuf::from_iter([
        root.as_ref(),
        "directory_a".as_ref(),
        "file_b.whatever".as_ref(),
    ])
}

Of course, this still doesn’t support starting with any pre-existing PathBuf re-using potentially pre-existing spare capacity; but even then, for pushing multiple parts at once, .extend can be nicer to write than multiple single .pushes, e.g.:

fn another_method(root: impl Into<PathBuf>) -> PathBuf {
    let mut path = root.into();
    path.extend(["directory_a", "file_b.whatever"]);
    path
}
11 Likes
7

pathbuf has try_reserve(), which you can use to preallocate enough space for all path component (plus slashes). This will make push cheap.

Sadly, there isn't a standard generic trait for taking either &Path, or PathBuf and avoiding extra realloc when reserving capacity.

With impl AsRef you won't be able to reuse existing PathBuf. impl Into<PathBuf> is slightly better for reuse, but given &Path it will allocate, so you'll get alloc + realloc on reserving, but that still might be cheap with a right memory allocator.

1 Like
8

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.