Rust and strings: what is the deal with "temporary value dropped while borrowed"?

1

I've been trying to write a very small routine that gets data from the database and passes into a function, but can't really grasp the concept of borrow in here. The compiler refuses to compile the following code:

use std::thread::sleep;
use std::time::Duration;
use std::process::Command;
use std::process::Child;
use std::collections::HashMap;
use tokio_postgres::{NoTls, Error};

type AgentMap = HashMap<String, Child>;

fn spawn<S: AsRef<str>>(name: S) -> Child {
    Command::new("python")
        .arg("worker.py")
        .arg(name.as_ref())
        .spawn()
        .expect("worker failed to start")
}

#[tokio::main]
async fn main() -> Result<(), Error> {
    let (client, connection) = tokio_postgres::connect("host=localhost port=5432 user=pg password=pg dbname=test", NoTls).await?;

    tokio::spawn(async move {
        if let Err(e) = connection.await {
            eprintln!("connection error: {}", e);
        }
    });

    let mut agents = AgentMap::new();
    let rows = client
        .query("select id, user_id, cat_id created_at from my_table", &[])
        .await?;

    for row in rows {
        let id: i32 = row.get("id");
        let uid: i32 = row.get("user_id");
        let cid: i32 = row.get("cat_id");
        let name: &str = format!("line:{}:{}:{}", id, uid, cid).as_str();

        if agents.contains_key(name) {
            continue;
        }

        agents.insert(name.to_string(), spawn(name));
    }

    Ok(())
}

(Playground)

Errors:

   Compiling playground v0.0.1 (/playground)
warning: unused import: `std::thread::sleep`
 --> src/main.rs:1:5
  |
1 | use std::thread::sleep;
  |     ^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

warning: unused import: `std::time::Duration`
 --> src/main.rs:2:5
  |
2 | use std::time::Duration;
  |     ^^^^^^^^^^^^^^^^^^^

error[E0716]: temporary value dropped while borrowed
  --> src/main.rs:37:26
   |
37 |         let name: &str = format!("line:{}:{}:{}", id, uid, cid).as_str();
   |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^         - temporary value is freed at the end of this statement
   |                          |
   |                          creates a temporary value which is freed while still in use
38 |
39 |         if agents.contains_key(name) {
   |                                ---- borrow later used here
   |
   = note: consider using a `let` binding to create a longer lived value
   = note: this error originates in the macro `format` (in Nightly builds, run with -Z macro-backtrace for more info)

For more information about this error, try `rustc --explain E0716`.
warning: `playground` (bin "playground") generated 2 warnings
error: could not compile `playground` (bin "playground") due to previous error; 2 warnings emitted

What is wrong, though? I'm keeping everything as a &str to avoid allocating a big String and then just sending that to the HashMap.

2

The format!(…) macro creates a new String. If this string is not assigned to any variable that owns it, it will live in an implicit so-called „temporary variable“ which lives until the end of the

let name: &str = format!("line:{}:{}:{}", id, uid, cid).as_str();

statement. This temporary variable is being borrowed by the vall to as_str, the borrow is stored in name, then the statement ends and the temporary is freed, which means that name pretty much immediately no longer points to valid data and any attempt to use it will – rightfully – result in a compiler error.

Note that this is the most straightforward kind of borrow-checking error imaginable: point to thing, thing gets freed, then pointer is used after the free. It’s also the kind of error that might directly let you run into some use-after-free situations in less safe languages (like e.g. C++). Maybe the only not-straightforward thing here is that format! is a macro, so its type might not immediately be obvious (but it’s well-documented).

The way to fix the code is to make name a String, then potentially borrow (via as_str(), or implicit coercion) where you use it, though at the last use, you can simply move the String and need not copy it another time:

    for row in rows {
        let id: i32 = row.get("id");
        let uid: i32 = row.get("user_id");
        let cid: i32 = row.get("cat_id");
-       let name: &str   = format!("line:{}:{}:{}", id, uid, cid).as_str();
+       let name: String = format!("line:{}:{}:{}", id, uid, cid);

-       if agents.contains_key(name) {
+       if agents.contains_key(&name) {
            continue;
        }

-       agents.insert(name, spawn(name));
+       let child = spawn(&name); // doing this first means we don't
+                                 // need to clone `name` again
+       agents.insert(name, child);
    }

By the way, note that because of a somewhat niche language feature called ”temporary lifetime extension”, the minimal change

-       let name: &str = format!("line:{}:{}:{}", id, uid, cid).as_str();
+       let name: &str = &format!("line:{}:{}:{}", id, uid, cid);

also makes the error go away. “Temporary lifetime extension” defines certain syntactical cases of let statements, where a temporary variable lives longer than to the end of the statement (instead it would live up until the end of the containing blocks). These are certain syntactical cases that are known to be relatively useless otherwise, causing a compiler error whenever used, in particular a statement of the form

let variable = &…;

always qualifies to temporary lifetime extension, if … is some expression evaluating to a value (as opposed to a “place”), but a method call such as

let variable = ….as_str();

wouldn’t qualify. (Yes, these rules are indeed relatively arbitrary.)

6 Likes
3

format! macro always allocates a String.

4

Thank you, that is a very detailed answer. Would you recommend a specific article or book on the subject of ownership, with examples like yours?

5

Of course, if you haven’t read it already, “the book” is always a decent place to start learning about Rust, and it does come with a chapter about ownership. I don’t remember whether it says anything about temporaries, though. (It probably doesn’t really cover that.)

There’s also this experimental modified version of the book with additional quizzes and visualizations (and apparently some changes in the contents in some of the chapters) from … I’m not actually sure, some researchers at Brown University, I suppose(?) … that has its own, (almost) completely different chapter about ownership. Haven’t actually personally ever read that, but their visualizations look cool.

1 Like
6

For the other question about allocating a string, you can avoid that by not making the key in your map a string.

type AgentMap = HashMap<(&'static str, i32, i32, i32), Child>;

Not sure what "line" represents here, so I just made it &'static str, but you may want to omit it entirely.

Then you can use this tuple for checking the map contents

agents.contains_key(&("line", id, uid, cid))

and create a string only when needed

spawn(format!("line:{}:{}:{}", id, uid, cid))

You can also replace the tuple with a proper struct for better organization, on which you could implement construction from row and conversion to string.

However, it's not really necessary to avoid the string, since allocating a couple dozen bytes is much faster than database queries and command spawning.

Another thing is that you can use entry to avoid doing two lookups.

2 Likes
7

You guys have all been extremely helpful. Thank you so much!

8

But how does that help if you are already allocating a string with format!()? You are probably completely misunderstanding how memory allocation and borrowing works.

It is not the case that if you somehow obtain a reference – any reference –, then you automatically and magically eliminated all overhead.

The reference you got must have already pointed to owned data. There's no way to make a &str without having some owned contents that it points to. And if you need to build that owned content dynamically, that is usually going to involve a dynamic allocation. Once you have a call to format!() that performs that dynamic allocation, you can't just undo it by taking a reference to the resulting String. References are not magic; all the &String -> &str conversion does is discard the capacity and return the same buffer pointer and length the String already had. You can't go back in time and un-spend the time you spent by allocating and constructing the String.

It is true that you can avoid copying in some cases by passing and taking references instead if owned values. But you are not copying anything here. You are constructing a single-use value once. That isn't going to be any faster by taking a reference after the fact.