How do I customize the rust stack memory allocator?

How do I customize the rust stack memory allocator?

There is no stack memory allocator by default.
Are you referring to some other crate?

Using the stack is not done via any separate allocator. Stack allocations are laid out by the compiler at compile time. I don't think there's any straightforward way to change it; one simply doesn't need to change it, and it's not intended to try doing so.

Maybe you could explain what your issue/goal is and we could help you find the appropriate solution or way forward?

There is no “thing” that’s a stack allocator, at least not in the form of any run-time executed library code. Stack allocation (the “getting the memory at run-time” part) happens by modifying the stack pointer, which is usually just a one assembly instruction in the beginning and the end of function execution.

One could also say that stack allocation (the “figuring out what values go where in available memory” part) happens statically, i.e. at compile-time. In this sense there is an “allocator” algorithm, AFAIK this would be part of LLVM, and I’m not aware of ways to customize this algorithm that wouldn’t involve modifying maybe rustc and/or LLVM. But this might not at all be what you’re after anyways.

Based on the nature of your question, either you should learn more about stack allocation to begin with, in which case people can probably point out more resources beyond what’s said in this thread; or you’ve made a typo or were confusing stack and heap and were asking about customizing the heap allocator, which is possible in Rust programs.

1 Like

I wanted to implement a stack coroutine similar to golang, but couldn't find a way to intercept and customize the stack memory allocator. I expect to implement a custom stack allocator to dynamically grow and shrink the stack memory manager

I want my program to be handled by a custom allocator when it growth and shrink ‘stack’ resources(not heap resource)。

I didn't confuse the difference between heap and stack, and I know that rust can customize the heap allocator

Based on your description, that doesn't seem to be possible

So, this is no way to implement an auto-scaling 'stack' memory allocator?

Are you looking for something like stacker — Rust implementation // Lib.rs?

yes,i know this crates.

I am pessimistic about my requirements unless the built-in compiler of rust can be modified, because the compiler should be able to tell the 'stack' size that needs to be allocated and the stack size that is being applied (this happens at compile time).

I don't know what you mean by "auto-scaling".

If you know golang, the initial stack size is 2kb. And if that's not enough, it expands automatically。

It might even shrink back。

Go uses a segmented "stack" and effectively heap-allocates the "stack" too.

Rust uses the traditional, OS-native/linker-supplied stack, and it can't grow.

If you need growable allocations, use the heap.

Your idea is great, but because of the rust compiler, we don't know what stack size the function needs to apply to

In the rust code, you can only customize the heap allocator。

but。 Anyway, we can't tell from the compiler how much space we need right now, right

It's not such a crazy idea. Back in the day we used a cooperative task scheduler. A MASCOT kernel[1]. Everything was written in the C/Pascal like language PL/M. Each task had in own state stored somewhere, basically register content, and had its own stack space. The scheduler of course swapped tasks saving and restoring those registers, including the stack pointer, thus taking the tasks stack into use. One cannot do that in languages like C, Pascal, PL/M...Rust so we had to drop into assembler to do it.

I presume that this can be done in Rust as well, being a systems programming language and of the ilk of C, Pascal, PL/M etc.

As noted above, the issue is that we don't know how much stack our Rust tasks will use. We can guess and then test, like we did in the old days, or we can get the hardware to fault on stack overflow and reallocate the stack somehow, but t hat gets us into the land of recreating the threads that the OS provides already.

  1. See The Official Handbook of MASCOT. Royal Signals and RadarEstablishment. 1987: http://async.org.uk/Hugo.Simpson/MASCOT-3.1-Manual-June-1987.pdf. About the last book on software engineering I understood.

By the way it contains the great quote:

Lastly, I stand ready with a pencil in one hand, and a sponge in the other, to add, alter, insert, expunge, enlarge and delete, according to better informatlon. And if these my pains shall be found worthy to pass a second impression, my faults I will confess with shame, and amend with thankfulness to such as will contribute clearer intelligence unto me.

Preface to 'The History of the Worthies of England' Thomas Fuller(1662).

Which seems fitting for the Rust development team and process.

I'm not exactly sure what you are accusing the compiler of, and it's still not clear what you need this for, or why you can't just heap-allocate dynamically-sized data.

You're right, there's no way to know how much stack space you'll need at compile time, but I think you can set up a hook to request stack space, or call a callback function. We can find out if we need to expand the stack space in the hook.

just like this code

#[stack_alloc_callback]
fn alloc(ptr: *mut u8,req_size:usize){
   //TODO impl  grow stack size
}

However, this requires the compiler team to support this change.

if compiler team add this change.i think just like golang's GMP scheduler concurrent design will be Introduced in rust, this will make concurrent programming very, very easy

Here's a counterexample- may this crate ,
It has a fixed stack space and will panic if the program needs more stack space than the preset value。 I think it uses allocated virtual memory. The problem is that we don't know if the function is requesting more stack space without a callback to increase the stack space

While the ‘std::future::Future’ works well(But it introduces other complications, such as launching the Future in Drop)

I don't think people should be prevented from using a simpler way to implement another coroutine that has stack coroutine。

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.