Vec::from_raw_parts

Documentation for Vec::from_raw_parts mentions:

Violating these may cause problems like corrupting the allocator's internal data structures. For example it is not safe to build a Vec from a pointer to a C char array and a size_t.

What is specific mechanism that makes it unsafe? Is it the way jemalloc works?

Well, first of all, it doesn't matter. It says it's not safe, don't do it, period. I say this because I've seen too many people trying to play chicken with unsafe code.

Secondly, it depends. It might happen to work in certain environments with a specific configuration, but it's absolutely not portable. So again, don't do it.

Third, the actual problem alluded to there is that you can't mix allocators. There's no way for one allocator to manage memory allocated by a different allocator. If you transfer ownership of memory allocated by malloc to Vec, and Vec isn't using malloc (which it generally won't be), then what happens when the Vec goes to deallocate that memory is... well, anyone's guess. Maybe your program crashes. If you're lucky, you just leak that memory.

Your tone is definitely not friendly, that's not what I've heard about rust community. You are implying that I want to use it, whereas I don't have that intent. I just want to know internal mechanism that makes it unsafe, from people who know that, for the sake of knowledge.

I don't know you, and have no idea where you're coming from, or what your attitude or intentions are, so I'm trying to cover my bases, especially given that the third point could be interpreted as "this is safe to do if Rust is configured to use the system malloc", which is absolutely not the point I want anyone to take from it. People also have a habit of skimming text once it has answered their question, so the safest place I've found to put qualifications and warnings is at the start.

You are implying that I'm being judgemental or condescending, whereas I don't have that intent.

1 Like

I don't care what was your intent, but your tone wasn't what I'd expect from a welcoming community. What you've said could be phrased differently with less bias.

It says it’s not safe, don’t do it, period.

What is this? You are saying that it should be followed religiously without understanding what exactly makes it unsafe. Actually I wanted to hear more about specifics of je_free implementation and what makes it incompatible with malloc, this is purely my theoretical interest. If you don't have anything to add to the topic, please don't answer then. Community would only win from that...

2 Likes

I don't think there's anything interesting to say. Even malloc and new in C++ aren't compatible. It's like asking why two games in the same genre use different save game formats or networking formats -- the default is that they're incompatible, because being the same would require very particular effort to be compatible. And compatibility actively prevents them from innovating or optimizing for different uses, and thus providing value in being different.

1 Like

Hi @NickM

It's unsafe, because C doesn't have access to Rust's jemalloc. Even if you compile your own jemalloc for C, that will be another copy that doesn't share data structures.

But 99% of time when you get a pointer from C it's from malloc(), which will crash when Rust tries to free it assuming it came from its own copy of jemalloc.

Even if you configure Rust to use system allocator, you have no guarantee that Rust uses exact pointer given by malloc/free, and not e.g. offset by a few bytes to insert extra metadata before the pointer.

There is a CVec crate for making Rust-y vectors from C pointers. You also can safely make a slice. Otherwise, copy.

1 Like

Hi Nick,

Welcome to the Rust community. I've been looking at Rust for a little more than a year and I've found the language to be very interesting and the community to be very welcoming.

I think you've "unlucky" in the sense that you asked about a potentially dangerous mechanism and @DanielKeep just wanted to keep you safe.

Lots of people come to Rust and ask questions about how to circumvent its safety features, so Daniel probably just tried to save you and himself the trouble of explaining you why your program crashes later :slight_smile:

So don't be discouraged by people trying to protect you against (potentially) shooting yourself in the foot.

3 Likes

I'd say that if you get a pointer from C there's a decent chance that it's not even dynamically allocated.

Freeing and reallocating pointers which you got from another library is not safe in general. Even in a pure C environment.

1 Like

Regardless of intent, @DanielKeep, you did sound a little condescending. Trying to understand why something is safe / unsafe is a key point to learning Rust, imo, and his question makes it apparent that this is a learning exercise, not seeing if he can skirt by and actually do this.

It's undefined behaviour by definition. Vectors can be dropped, and when dropping, Rust assumes you used its default allocator. If that assumption wasn't true, the allocator won't know what to do, likely corrupting unrelated memory.

Similar story applies to C++ if you have used it. malloc, new and new[] memory allocation mechanisms are all incompatible with themselves. See C++ FAQ for more details, the same applies to Rust (but replace new with allocation using Vec::new).

If you want a slice of C allocated type, you can use slice::from_raw_parts instead, as slices don't have destructors.