Hello there!
I just wanted to verify that I understand some rules right about the size of a struct and its align.
As far as I understand these are the rules:
For primitives (Like numeric types and char and bool) their align is equal to their size except for u128/i128 where their align is 8 instead of 16.
For structures, the align is the largest align present in the structure and their size is rounded up to the nearest multiple of their align.
Padding is inserted between types to keep them on their own align within a struct
While the RISC-V architecture supports access to misaligned data, potentially through time-consuming exceptions, those can't be used for atomic operations. Thus for the RV128I base ISA you should assume size: 16, align: 16. That will undoubtedly be true of any other architecture with hardware support for u128.
Ah, okay, that's good to know, it did seem rather weird that the u128 was an exception, but I only did playground testing. How might one cfg over that?
Playground testing on a 32-bit client would undoubtedly show that u64 has align: 4 because that's the granularity that the hardware works at. Likewise for u128 on a 64-bit client. If you consider atomic instructions, it's obvious that they have to be aligned on hardware boundaries so that they don't tear.
Why would you want to cfg over it? Do you cfg over u64 when used on a target that has align: 4 rather than align:8, such as some Intel processors? or do you code so that it will work on most potential targets, whether they have relaxed alignment or not?
That made me think... I can (and should) just depend on whatever std::mem::align_of returns instead of worrying about the value at compile time. Please disregard that post, my code will become platform-agnostic. To be more specific I'm trying to build a library that can reconstruct types at runtime using reflection information from another language, such as C#, which can tell me the order of the types, their size and align, and padding, etc. I read up on the microsoft docs for packing and alignment yesterday, (I'm mostly aiming to support C#, but other languages would be nice too!) and it seems that it follows the same rules, therefore I should be fine with the std::mem intrinsics.
Ah yes, I made an error, as though I read the size of y in Foo3 instead of the align, causing me to try to pad it to an align of 8 instead of 4. I'll update my answer.
So I believe that my question now becomes, should I blindly trust what std::mem::{size_of, align_of} report and work off of that?
I'm interested in what exactly you're wanting to try to support with C# - I've done a lot of ruminating and very little implementing for interweaving C# and rust code.
Right now, I think the 'easy' thing is, as with most other languages to just 'speak C'. But I think we can do better, and I'm interested in any thoughts you have on that. I might be up for contributing, if you have a project in mind.
Well, I was going to work on it as a hobby, so no specific reason. I might at some point, when it's actually alive showcase it at the Amethyst forum but I've got a long ways from that.
And yes, usually the only way to do it is to just "speak c" but that can become ugly and a nightmare, but most non-native languages (as in, with a GC) support some form of reflection which would provide lots of support when doing this kind of thing.
I will upload a repository either today or tomorrow if you're interested in contributing.
C# is the language I work in the most, so I'm familiar with a fair amount of it's reflection infrastructure. I'm definitely interested in what you're working on. I won't promise any significant output, but I'd love to try to dig into it if I can find the time.
This is one part of it, but for a lot of these reflection-heavy, GC'd languages like C# (and other CLI languages) and Java (and other JVM languages), reflection allows you to not only inspect types, but in some cases, emit new types. It may be possible to do some truly exotic things if we work carefully and can create tighter bindings.
For C#, at least, we could also probably allocate objects and then fix them and pass pointers to Rust, and have effectively-shared data, on the C# heap. This, rather than what we usually have now, where you can perhaps pass objects from the Rust heap, carefully, but usually, you're passing data on the stack.
Well, in certain cases with GC languages it gets annoying and ugly, like for example, I tried to pass a mutable array from C# to rust (as in it wasn't declared as mutable in C#, instead I just wanted to mutate its contents) and I found out that I couldn't use the standard way, (as documented in the rust ffi omnibus) and instead needed to use a fixed statement like so (Which also required passing a new flag into the project config for /unsafe usage):
So I wanted to solve that, make it easier to work with, and also add some more ease of life improvements, like a wrapper around a Vec/List to be able to work with heap-allocated items in both languages (Such as strings).
When I searched solution to cooperate between Java and Rust after some research I choose JNI,
and solving annoying code repeating by some kind of DSL (for example to rule how convert rust date type to Java date type).
It would be interesting to look at solution from other direction.
I'll admit I'm not very familiar with Java reflection, or with JNA and JNI. For C#, yes, there's some significant overhead with P/Invoke (which is how you call native code from C#), and there may be some for invoking in the other direction. But, this is for static bindings. Interestingly enough, it may be possible (again, if we're diligent and careful) to shed some of that overhead. Much of that overhead is to do with moving managed GC objects down to COM and C.
If we're creating objects as structs for interop, we may get out of a lot of that. In C#, structs are bare objects, more like rust's, while classes are generally heap-based, boxed objects. There's been a lot of work to allow more granular control over these type of operations in C#, mostly to support the Unity game engine. It just so happens that these operations, while needed for optimizing math in Unity, may also help us in FFI.
Aside from all that, I'm not sure performance is paramount for FFI. Making easy (and hopefully idiomatic) FFI possible can allow you to better focus on writing performant C# and Rust code, and make the interop code's performance matter less.
My idea was to produce a kind of static "registry" of struct layouts, and every time a value is passed via pointer, it also needs to get an entry into the registry, which from their its values can be accessed and casted to the appropriate types.
This would be done using a set of exposed methods which would need to be used in the managed, so for example, we would have the following in rust:
static mut TypeDefinitions: MaybeUninit<[MaybeUninit<TypeDef>; 128]> = MaybeUninit::new();
static OccupiedTypes: AtomicUsize = AtomicUsize::new();
extern "C" fn register_type(name: *const u8) -> *mut StructDef {
//Form a StructDef with associated name
let boxed = Box::new(struct_def);
Box::leak(boxed)
}
extern "C" fn add_member(this: Option<&mut StructDef>, typename: Option<&'static mut TypeDef>) {
let this = this.expect("Could not find self pointer!");
let other = typename.expect("could not find other pointer!");
this.members.push(other); //Simplified
//We need to worry about sizing and padding at runtime
}
extern "C" fn finalize_struct(this: Option<&'static StructDef>) -> *const TypeDef {
let this = this.expect("Could not find self pointer!");
let boxed = unsafe { Box::from_raw(this) };
let this = *boxed;
//Push onto TypeDefinitions up top.
//Increment the counter
unsafe {
&TypeDefinitions.data[OccupiedTypes.fetch_add(1, Ordering::Relaxed)] as *const _ //Not too sure about ordering...
}
}
I'm planning to expose something like this to define types, unless a better idea arises.
The thing is, it's still all done over a C API, it just happens to be that it's no longer statically typed. It's not dissimilar to the dynamic type in C# where rules are checked at runtime:
public static void Foo(dynamic x) {
x += 34;
x.abc();
x.z = 2;
x = "a";
}
Although this is a bogus example, the rules for this would be checked at runtime so we wouldn't get an error at compile time, but we wouldn't need to know the type of x either. The only thing that wouldn't be supported would be reassigning to a different type.
The point of this is essentially to introduce the following:
A kind of ffi-generics
An easy way to ffi structures from other languages into rust
A way to be generic over the actual type while accessing a value, similar to the following in js:
function foo(point) {
point.x += 2;
}
where point can really be anything, as long at is has an x field.
Cool. I'm working on a C#/rust app in my hobby time at home, I'll try to take a look. I have some ideas for the C# side, so I'll try to take a look some time this weekend and see if they mesh with what you're doing. If I get anywhere with it, I'll write up a PR.
Otherwise, I'll just file an issue with some commentary and ideas to get the thoughts flowing, at least. Let me know if you have any area you'd like me to dig into more specifically.
I've already taken a crack or two at getting C# and rust to talk nicely in my app, but I'm to the point where I need to put together a lot more UI before I'm going to actually be fleshing out the interface between the languages.
