Can I know the exact memory my program is going to use when it is run (when developed in rust)?


#1

Guys,before you answer this question,I would like to tell you why this question came to my mind. Im a programming enthusiast and I keep on learning new languages just for fun and my pet projects. I started my programming journey from python which is pretty good to learn programming concepts) and later moved to Java . Java was working great for me one day when my program refused to run because the JVM couldn’t start up due to lack of memory. I have 2GB laptop running on xubuntu. I had to close Firefox just to start the JVM to run my program. That’s when the importance of memory hit me .

As I look to learn rust and handle memory myself ,the above question comes to my mind.

Thank you guys…


#2

One difference between Rust and Java in this respect is that Rust gives you deterministic control over when memory is deallocated. Rust values get destroyed (and release any memory they own) immediately when they are no longer owned by any variables that are in-scope. In contrast, Java values will remain in memory for an indeterminate amount of time before the garbage collector eventually releases them. Another difference is that many things in Rust are designed to have low memory overhead by default.

This is useful for optimizing memory use, but it is not sufficient. Rust still lets you allocate memory dynamically, and depending on your program, the amount allocated could depend on user input or other unpredictable factors. It’s also up to you to find out how much memory is allocated by any library functions you call. Writing code with totally predictable memory use is possible, but it requires techniques beyond what the language can provide automatically.


#3

In answer to the question directly: yes, absolutely.

If you need to manage it that closely, yes, you can. Allocators (in themselves) aren’t yet stable, but, like in C++ and other languages (before they became normal), you can make it work. (Note: In C++ and other languages people, myself included, still use workarounds because of interface concerns).


In answer to the context driven question, which I interpret as “can it help me solve the problem”: maybe.

Without knowing the specifics of the issue, there may be other solutions. The two that come to mind quickly are:

  1. swapping/paging the data away (get a cheap USB drive if you’re concerned about your drive, or you don’t want to write to it constantly for space/capacity concerns);
  2. Using memory pools.

If you’re interested in learning Rust anyway, both of those are completely viable (and serde makes reading and writing to disk easy if you don’t want to roll your own for the sake of learning, or for the sake of raw speed).


Contextually, it’s worth noting the JVM has some overhead (by necessity/design), and that’s fine. mbrubeck already commented on the garbage collection issue, but there’s a bit more to it (and it’s not really worth going into here), and it can add up (depending on what you’re doing). But in that overhead comes the flexibility to write code in a way you’re used to writing it without certain concerns that arise in C, C++, and Rust. Rust will seem like it has an undue number of sharp edges when you start, so if this is something extremely complex internally, make sure you break it down in a modular way so you can approach it without getting frustrated (I know, basic programming advice, but it bears repeating because Rust can be infuriating at times).

The other thing is that if you decide to go the “my own allocator” route, don’t start there. If you’re worried you’ll need it later, that’s fine, but consider a reset() method, and pooling it instead. Since the faux-allocator method you’ll see used a lot is Vec::<u8>::with_capacity (it’s what I use), save yourself the potential headache and just use Vec::<Object>::with_capacity, add something like is_valid() and use reset() when you’re done with it (so it can be re-used). The end result is, for your purposes, the same, and there’s no real reason to go insane trying to resolve a potential memory leak caused by layers of unsafe code. (If you’re wondering why I’m suggesting it that way it’s because it may be as small, and as simple, as the JVM overhead being the issue; it may be far more complex, in which case your initial allocator design may be problematic.)