What to do when nothing is implemented to resolve E0507

help
1

In my short time with rust, I seem to encounter variations of the following problem frequently. A function from a library takes an object instead of a reference and this object does not implement copy (or, in the case which prompted this post, even clone). As far as I can tell when this happens you are not left with many options. I think that the "right answer" here is to figure out some way to get back ownership of the object, however every time I've encountered this I've really struggled to do that since whichever API I am dealing with does not make it obvious how to do that.

Rather than coming up with a contrived example, let me show what I'm trying to do which is pretty simple

pub unsafe extern "C" fn from_chunks(arrsptr: *const ffi::ArrowArray, l: usize) -> *mut ChunkedArray<Int64Type> {
    let arrs = std::slice::from_raw_parts(arrsptr, l);
    let mut cs: Vec<Box<dyn Array>> = Vec::with_capacity(l);
    for arr in arrs {
        cs.push(ffi::import_array_from_c(*arr, DataType::Int64).unwrap());
    }
    let ca = ChunkedArray::from_chunks("name", cs);
    to_jl_ptr(ca)
}

This fails on ffi::import_array_from_c with a E0507 because it can't get back ownership of arr. Since I can't do this from a reference I expect I'm going to be told that std::slice::from_raw_parts is wrong, but in this case I don't know what the alternative is.

3

I'm confused about the fact that its dereferencing arrsptr in the loop... isn't this trying to take from the top of the array on each iteration?

4

Whoops, I got very confused there, my bad! So you're passing an array of arrow2 arrays from C?

5

Yup, this is to be called from C.

So another issue I would have is that I don't expect a *mut ffi::ArrowArray to be compatible with the Option wrapper, so to do this I would have to at least have another function which can be called with *mut ffi::ArrowArray and I'm not sure that's going to work (wouldn't it end up in another E0507?).

6

Take 2:

use arrow2::{array::Array, datatypes::DataType, ffi};

pub unsafe extern "C" fn from_chunks(arrsptr: *const ffi::ArrowArray, l: usize) {
    let mut cs: Vec<Box<dyn Array>> = Vec::with_capacity(l);

    for index in 0..l {
        cs.push(ffi::import_array_from_c(arrsptr.add(index).read(), DataType::Int64).unwrap())
    }
}

That doesn't prevent you from accidentally re-using the array though, so you'll need to be careful about how you call this function

7

Ok that makes sense, thanks! There are a few things about this which don't seem ideal and I wonder if I can resolve them.

  • Is there a way I can put the array into a slice or Vec instead? I suppose I can do exactly the same thing and just push into a Vec but I think that would cost an unnecessary copy. The reason I ask is because I'm probably going to have to do this a lot and it would make more sense to have a few simple methods for passing arrays back and forth.
  • Is there any way to tell rust not to try to free the argument when it hits the end of import_array_from_c? It's not crucial but I have a bad feeling I'm going to double-free these if I do it this way, because normally I would free the array after this is called.
8

Can you explain what you mean by "can't get back ownership of arr"? If you share a copy of the full error message we'll be able to see exactly which parts the borrow checker doesn't agree with.

Yes. Using std::slice::from_raw_parts() was the correct function to use.

The only thing you've got to be careful about is 1) making sure arrsptr really does point to l number of initialised arrow arrays, and 2) the slice (or any reference into it) doesn't accidentally escape your from_chunks() function and cause some sort of use-after-free.

It looks like import_array_from_c() constructs an InternalArrowArray internally, and it mentions that users will need to use InternalArrowArray::into_raw() to get "ownership" of the C pointers back so they can be manually dropped.

That probably means you can leave the array as it is and Rust will hopefully not try to free it.

This isn't very obvious from the docs/types though, and I'm just guessing based on my reading of the arrow2 source code, so it's probably best to create a ticket on their issue tracker to find out whether there's a way to do this without accidentally messing up ownership or causing a double-free. That'd also be a good opportunity to make a PR updating their docs so the next person to come along doesn't have the same issues.

9

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.