Technicalities: Closure implementation and potential serialization?

1

Hi everyone!
I have a very technical question about closures in Rust. Or rather, how they are implemented.
I have no experience with writing compilers, so reading the source code myself is out of the question. But what I do understand from what I've read about Rust so far is that memory in Rust is handled in some way that is pretty close to slicing. Also, when looking at the Dining Philosophers example, and reading up on it a bit more, it becomes clear, that closures perform some pretty fancy magic to retain their stack/heap across function- and calling-context borders. This leads me to assume, that closures in Rust have a very well defined stack frame (so to speak).

Now, my question is: If my assumption is correct, then (given some very carefully laid out constraints) would it technically be possible to, sooner or later, make closures byte-code-serializeable so they could (in principle) be transferred across a machine/network border?

Please note: I am very aware of potential technical issues (like architecture and endianess) as well as posing a HUGE security risk when doing so (I mean, 'executing' a serialized closure from a foreign source is essentially equivalent to running a normal binary from an unknown source). This question is strictly about the technicalities and how close to possible it would be, given the current implementation. So, you could compress it to the question:

Are closures well structured enough in the current incarnation of Rust, so that they could be, in principle, exported as their own executable code? Like a shared object, for example?

The reason for my question is rooted in an idea for doing remote procedure execution. If you could 'extrude' executable code from a closure "at runtime", this would be a very, very handy feature for distributed systems. It would be close to what CORBA always wanted to be, but failed due to implementation complexity and take-away the platform independence.

So, is anyone involved enough into the current implementation of the Rust compiler to answer my question?

Thanks!
-Timo

2

Closures in Rust are basically a sugar for implementing a trait on a struct. Captures are either by reference or by value, these are then added as fields of an anonymous struct type which then has one or more of FnOnce FnMut or Fn implemented on it to allow it to be called. The code of the closure is not really different from any other function or method aside from having references to the captured variables rewritten to access them from self.

Closures are not byte-code they are compiled to native code just as any other function, so they are no easier to serialize than any other function in rust. You also have all the problems of trying to send pointers to remote systems, especially if the closures do not use 'move' by value capture. You would also face all the issues of having to serialize anything the closure may capture if you want to send them over a network channel.

1 Like
3

Thank you for your reply!

The first part of your answer makes some things clearer and does support my assumption somewhat. If everything is arranged as a struct internally, then it might not be thaaaaat difficult to stream out the structs content in one consistent block of data.

And I am aware of the problem of pointers and deep-copies when sending structures over a network. That's why I mentioned the "carefully laid out constraints". Only doing call-by-value would be one sensible constraint. Or in other words, move becomes pretty much mandatory.

Also, I understand that closures are compiled into native code. I mentioned byte-code since I think it would be wise to supplement the serialized closures with some additional information like symbols and function signatures, and, for example, to make absolute pointer values relative instead, provided they only point to data that resides inside self. That would make a serialized closure more close to byte-code than actual native code, but I agree that this might be a bit of a wonky comparison. It would probably resemble a .so library more closely than anything else.

Thanks again for taking the time to reply!