Relationship between return, ret, and drop

new updates:

Except 2 questions below, discussion also covers the rule for final expression and temporaries.


  1. what's the relationship between drop and return? :white_check_mark:
  2. does ret in asm correspond to return in rust source code? :white_check_mark:

drop seems to be called after return code and asm. (probably I'm wrong, drop is called when the end of the scope is reached for a var. Usually, return triggers an end of scope)

pub fn demo2() -> usize {
    
    let x = "gui".to_owned();
    let y = &x;
    y.len() // x is dropped after this return
} // that is to say, x is dropped at here.

and

        mov     qword ptr [rsp + 8], rax
        jmp     .LBB84_2
.LBB84_2:
        lea     rdi, [rsp + 16]
        call    qword ptr [rip + core::ptr::drop_in_place@GOTPCREL]
        mov     rax, qword ptr [rsp + 8]
        add     rsp, 56
        **ret**

Then why the code in my previous question fails (shown below)? x should be dropped after the return clause so that x lives long enough than x.borrow().val

use std::cell::RefCell;

// Definition for a binary tree node.
#[derive(Debug, PartialEq, Eq)]
pub struct Foo {
    pub val: i32,
}

pub fn demo() -> i32 {
    let x = RefCell::new(Foo{ val: 10 });
    x.borrow().val
} 
// x should be dropped at here so that return is valid. 
// But compiler says no

However, from the asm, drop is called before return.

previous discussion, which is the original question:

follow up discussion, which is based on @steffahn's great answer:

Okay, I’m not entirely sure what you mean by “calling return” here. For sure, there’s the return keyword in Rust, there’s the ret instruction in (most) assembly languages and there’s the concept of return values and “returning from a function”. Then there’s also final expressions in blocks or functions in Rust that can be used to indicate return values. There’s also a distinction between dropping a value in Rust and “calling drop”, where the latter could mean specifically the drop function or the Drop::drop method (two different things yet again).


So, as you’ve asked before editing your question: Is there a relationship between “returning from a function” and the ret instruction? Yes, but not as straightforward. Returning from a function involves a bunch of steps dictated by the calling convention of Rust functions, which is currently not specified/stable as in: it might change between Rust versions, between functions, etc... A calling convention is part of a programming language’s ABI, so this fact might be pointed out by statements such as “Rust doesn’t have a stable ABI”.

So the calling convention describes how function calls get translated into assembly. Ordinarily, functions are implemented using a call stack, and returning from a function (think “function without return value” here, e.g. fn foo(bar: Bar) or more explicitly fn foo(bar: Bar) -> ()) involves multiple steps like

  • restoring overwritten values in registers that the caller needs
  • shrink the stack so that the current function’s stack frame gets “deallocated”
  • load a return address (from stack) and jump to it

That last step, “load a return address (from stack) and jump to it” is what the ret assembly instruction does and it is necessarily (or even: by definition) that a function does.

Similarly, “returning from the function” is, by definition, the last thing that a Rust function does. So dropping local variables happens before this. We haven’t considered “return values” yet though. And all these assembly steps listed above that are part of the whole “returning from the function” business are thus the last thing that happens, too. Dropping local variables always happens before returning.


So, return values... basically, to be honest you wouldn’t really need them, you could just pass pointers as function arguments and set up the convention that the return value is written to those pointers. But there’s a cheaper way to return small values: Leave them behind in a specific register. Or multiple registers for e.g. structs. Or have no return value at all at runtime, if the value is zero-sized. (So () is a special case of this.)

Now when is the return value returned? Hard to answer. What counts? The time when it’s written to the return register? Or just at the same time that the function “returns” (i.e. ret is called)? Up to interpretation; technically return values do not exist; as in: There’s nothing that distinguishes a return value from just a bunch of assembler instructions that turn out to behave as the Rust language specifies. Of course you’ll often be able to interpret “returning from a function” as something that your assembly does, especially if you know the calling convention, but it’s not set in stone.

In particular, as long as order doesn’t matter, the compiler, especially when optimizing, can do whatever it wants. First assign a return value to a register and then shrink the stack or the other way around? Who cares. You (the compiler) have figured out that calculating the return value and dropping some local variables doesn’t influence each other? Do it in the assembly in whatever order you like.


At last, on your Rust code examples. The only thing to consider here is actually the order in which local variables are dropped. For this, it is important to note that local variables include named local variables and temporary variables/values (aka “temporaries”).

The basic rules in Rust are:

  • local variables are dropped at the end of the block, in reverse order of declaration
  • temporaries are dropped at the end of the containing statement

now that latter part “the end of the containing statement” is not quite true, in actuality temporaries are dropped at the end of ...IT’S COMPLICATED... . And part of this “IT’S COMPLICATED” is this note

Notes :

Temporaries that are created in the final expression of a function body are dropped after any named variables bound in the function body, as there is no smaller enclosing temporary scope.

AFAICT this is to mirror the behavior on blocks. In other words, a

fn foo(x: Bar) -> Baz {
    /* some code */;
    /* some code */;
    /* some expression */
}

behaves (AFAIK) the same as

fn foo(x: Bar) -> Baz {
    {
        /* some code */;
        /* some code */;
        /* some expression */
    }
}

And blocks might behave this way since the final expression of a block is not really a “statement”. So the “containing statement” is bigger.

Anyways, for your examples this means:

pub fn demo() -> i32 {
    let x = RefCell::new(Foo{ val: 10 });
    x.borrow().val // this expression contains the temporary that’s holding the
    // return value of `x.borrow()`.
    // By the rule in the “note” above, this temporary has to
    // be dropped AFTER x is dropped. Which doesn’t work, since 
    // the destructor of the `std::cell::Ref` in this temporary has
    // access to and modifies the `RefCell` itself, which would
    // result in a use-after-free.
}

On the other hand

pub fn demo2() -> usize {
    
    let x = "gui".to_owned();
    let y = &x;
    y.len() // there’s (potentially / I’m not sure) a temporary here
    // that’s holding a copy of `y`, which *is* a reference to `x`.
    // But since references such as `y` don’t have any destructor
    // it wouldn’t be problematic to “drop” y after x anyways.
}

also note that in both cases the actual function calls for evaluating the return value, i.e. .borrow() or .len() happen before any local variable or temporary (from the return value expression) is dropped.

For both these examples, how “returning” in the control-flow sense works has not really got anything to do with this, and the “returning from the function” happens after any local (named or temporary) variable is dropped, anyways.

6 Likes

Much appreciated for your thorough reply! Addressed a ton of my confusion! My reply is split into 2. This is for clarification about my questions.

where the return keyword is used, which involves dropping locals, put values to be returned to caller (the calling convention you mentioned), and ret, which is a jump.

The final expression can be viewed as calling return implicitly.

I only mean any drop that is implicitly invoked by the compiler, not by the user.

Note that an explit return does not come with the same kind of special-case rule that temporaries are dropped after named variables. E.g. this version of your code compiles. I guess the most accurate translation into "explicit return" style would be by following up on my previous code example

which suggests that the translation

fn foo(x: Bar) -> Baz {
    return {
        /* some code */;
        /* some code */;
        /* some expression */
    };
}

is probably 100% accurate (if I'm not missing anything).

2 Likes

Thanks for answering the relationship of ret instruction and drop by compiler.

That's a good point. After reading your answer, I intend to define return as:

  1. set up return registers that are required by calling convention
  2. call ret instruction to rewind the function frame.

drop local is a rust language-specific stuff (like cpp destructor, called by compiler by default). The drop may happen before 1 or after 1 or interleave in 1 (multiple return values).

yes!

if so, x.borrow().val should be evaluated before anything is dropped. Then it should compile?

new reply:

Just realize in your playground code, the final expression is turned into a statement, which means the temporaries are drops at the end of this containing statement (at this line).

return x.borrow().val;

If I follow your translation, the temporary rule for the final expression still holds (playground):

pub fn demo() -> i32 {
    return {
        let x = RefCell::new(Foo{ val: 10 });
        x.borrow().val
    };
} 

The above explicit return still can't pass compiler check.

Well, the final expression applies not only to the function body but also to any block expression.

Temporaries that are created in the final expression of a function body are dropped after any named variables bound in the function body, as there is no smaller enclosing temporary scope.


previous reply:
thanks! Can't believe it can compile! Never thought implicit return and explicit return has such a big difference! Thanks a lot

2 Likes

No, since the problem is not from when evaluating this expression happens but when its temporaries are dropped. To give a different example e. g. complicated expressions such as f(foo(), &bar()) are evaluated by first evaluating each part, e. g. foo() and &bar() and finally the call to f. After evaluation, there's still a temporary containing the return value of bar() which has to be dropped at some point (I guess for foo() the temporary does also still exist but has been moved out of, so we can ignore it.) That dropping of temporaries is a seperate thing from evaluation (at least in my mind) and happens at the end of scope of that temporary, potentially larger than the expression itself.

Of course, more complicated expressions like blocks can also contain temporaries or even named variables that have a scope that's shorter than the expression, so these local variables would be created and also dropped during the "evaluation" of the whole expression.

by the way, IMO an ideomatic way to make the code compile would be not to use explicit return but to assign the return value to a local variable first. Then the temporary is dropped at the end of that assignment. Admittedly neither solution feels particularly nice/clean/elegant.

Rust Playground

2 Likes

Thanks! Now I know why it says (here)

= note: the temporary is part of an expression at the end of a block;
           consider forcing this temporary to be dropped sooner, before the block's local variables are dropped
help: for example, you could save the expression's value in a new local variable `x` and then make `x` be the expression at the end of the block
   |
11 |     let x = x.borrow().val; x
   |     ^^^^^^^               ^^^

This error message should include information about the drop rules for the temporaries and final expression.

as there is no smaller enclosing temporary scope.

  1. no smaller enclosing temp scope? Does it just refer to the whole function body?

It took me some time to digest it. If I understand correctly,

pub fn demo() -> i32 {
    let x = RefCell::new(Foo{ val: 10 });
    x.borrow().val
}

is equivalent to

pub fn demo() -> i32 {
   // declare tmp
    let tmp: blah_type;
    {
         let x = RefCell::new(Foo{ val: 10 });
         tmp = x.borrow();
    }
    // drop x
    // error: dereference the dangling pointer
    return tmp.val;
}
  1. why rust delicately has this rule that temp var is dropped at last in the final expression?

It seems fine that tmp is dropped before x. What rust needs to do is to move that value into some register and then drop everything in the normal order.

On cannot look at "return" or implicit return or just the end of a function and think about RET in assembler. For example your function could be compiled to "inline" code, in which case there will be no CALL or RET instructions generated for it.

This holds true for pretty much any other construct in a high level language. Just because you have a "loop"construct in your source does not mean there is necessarily any branch instructions generated, the compile could unroll a loop into a linear sequence of repeated instructions. For division and multiplication there may not be any MUL. or DIV instructions generated, it could be compiled to shifts and additions, say. And so on.

Whist imagining machine instructions is a useful idea when reasoning about how computers work, what you imagine may not be anything like the code the compiler generates.

While I have not studied the details of the response, I thought what you were seeing could be explained by the following.

Scope is often bound by the {} however, the compiler will try to avoid lifetime related errors by considering the sequence of expressions within a scope. If a ref is not used midway through a block of code, it may drop it, and will drop it if doing so avoids a compile-time error.

This modification to the lifetime “expectancy” of memory was introduced to avoid rejecting code that was effectively “following the rules” if it weren’t for an overly simplified view of scope.

It's important to note, however, that this only changes the compile-time analysis of borrow lifetimes. It doesn't change where destructors run. Types with destructors cannot be dropped “early” by the compiler.

There are four basic places where destructors run:

  • On leaving a block expression, destructors run for any variables local to that block, if they are initialized. (“Leaving a block” includes returning from a function body, breaking/continuing from a loop body, unwinding due to panic, or simply reaching the closing brace at the end of the block. ”Initialized” means a value has been assigned and has not been moved out.)
  • On finishing/leaving a statement, destructors run for any temporary values created during the statement.
  • On assigning a new value to a previously-initialized location, the previous value is dropped and its destructors run.
  • On calling the special drop_in_place intrinsic function.

(This is simplified slightly from the detailed rules which also include temporary scopes for match arms and operator expressions, and special cases where temporary scopes are extended to outlive the enclosing statement.)

2 Likes

Thank you for clarifying. That’s clearly germane. Accordingly, my hunch was not a viable explanation for what was going on here.

Good point. Inline function is not considered at here.

Maybe I should open a new thread.

Any specific reason to have this exceptional rule for implicit return (final expression)? Why the temporaries for the final expression cannot be dropped first? @steffahn @mbrubeck

No, it actually shouldn't. That doesn't make sense. Once a function returns, its locals are gone, whatever it left behind in its own stack frame is considered garbage (because it can be reused for subsequent function calls), and anyway the program continues with the body of its caller. There's no way a function can actively do something (e.g. call drop) after it has already returned. And it would be wrong conceptually as well, even if it were somehow technically feasible.

The right conceptual model for a function is that it does any required cleanup before it transfers control flow back to its caller. The function completes the work it was asked to perform, then it cleans up after itself. Once it's sure it no longer needs to do anything, only then does it transfer control flow back to the caller. Not doing so would be an unnecessary complication and a source of confusion, unsafety, and bugs.

further discussion about the design of this temp rule of final expression moves to

yes. the return is the rust keyword not ret instruction.

yes! agree with your conceptual model.