I need to create a Vector of u8 whose memory address is sector size aligned.
I saw another post which talks about same problem(Link) but when I tried to implement it(Playground link), It was not working. Can anyone help me with this.
vector size is know at runtime time only.
Thanks in Advance.
println!("align_of_val {:?}", mem::align_of_val(&kp)); // should be 64
Surely not? That's the alignment of the vector (ptr-size-capacity triple) itself, not the alignment of the buffer.
You are also misunderstanding what align_of_val() returns. From the docs:
Returns the ABI-required minimum alignment of the type of the value that
valpoints to in bytes.
(Emphasis mine.) It would be enough to check that ptr as usize % 64 == 0.
Also, if you resize() the vector while its type is Vec<u8>, then of course you can't expect the alignment to still be a multiple of 64. For this reason, you shouldn't expose a Vec but only a Box<[u8]>.
The implementation is also unnecessarily unsafe and actually exhibits Undefined Behavior, because you are deallocating with alignment 1 whereas the allocation was created with alignment 64 (you can see this if you run your code under Miri in your Playground).
Here's a safer and simpler (and hopefully correct) implementation:
pub struct AlignedByteBuf(Vec<AlignTo64>, usize);
impl AlignedByteBuf {
pub fn with_capacity(n_bytes: usize) -> Self {
let n_units = (n_bytes + size_of::<AlignTo64>() - 1) / size_of::<AlignTo64>();
let aligned = vec![AlignTo64([0; 64]); n_units];
AlignedByteBuf(aligned, n_bytes)
}
}
impl Deref for AlignedByteBuf {
type Target = [u8];
fn deref(&self) -> &Self::Target {
unsafe {
core::slice::from_raw_parts(self.0.as_ptr().cast(), self.1)
}
}
}
impl DerefMut for AlignedByteBuf {
fn deref_mut(&mut self) -> &mut Self::Target {
unsafe {
core::slice::from_raw_parts_mut(self.0.as_mut_ptr().cast(), self.1)
}
}
}
Alignment is a static (compile time) property of each type. It can't be set via variable. Having an alignment on the outer type doesn't change the alignment of a contained u8. Starting with a value of a type with one alignment and translating it into a different type doesn't change the alignment of the different type.
If you want to know what a value happens to be aligned to, you could use something like
fn happens_to_be_aligned_to<T: ?Sized>(ptr: *const T) -> usize {
let addr = ptr as *const u8 as usize;
1 << addr.trailing_zeros()
}
To make what you want, you'll probably need a newtype in order to enforce alignment for any operation that could reallocate. Your newtype could feign variable alignment by overallocating and ignoring the head of the allocation when needed. You'll also probably have to juggle a lot of MaybeUninit, unless you initialize the head and take a lot of care to only go from Vec<u8> to Vec<AlignTo> when the initialized elements end at a boundary of correct size.
Incidentally, this is undefined behavior as it creates a reference to uninitialized memory:
let mut aligned: Vec<AlignToSixtyFour> = Vec::with_capacity(n_units);
aligned.set_len(1);
println!("align_of_val {:?}", mem::align_of_val(&aligned[0]));
Also, there's no precondition for callers of the function to meet, so this should probably be a safe function containing unsafe blocks, not an unsafe function.
What that snippet does is UB: a Vec<T> expects the backing heap-allocated [T; capacity] buffer to be T-aligned. Constructing a Vec with a manually heap-allocated buffer with lower or higher alignment is forbidden.
Usually the trick with higher alignement is to start like that post did, with a Vec<AlignedTo<64, [u8; 64]>>, for instance, and never leave it.
use {
::core::mem::align_of,
::elain::{Align, Alignment},
};
#[repr(C)]
struct AlignedTo<const MIN_ALIGNMENT: usize, T = [u8; MIN_ALIGNMENT]> (
Align<MIN_ALIGNMENT>,
T,
)
where
Align<MIN_ALIGNMENT> : Alignment,
;
impl Deref{,Mut} for … {
type Target = T;
}
One question which may make things way easier, first and foremost, is thus:
do you need the a posteriori size-changing capabilities of a
Vec?
If not, then things can become way easier, because any impl 'static + DerefMut<Target = [u8]> will behave like a Box<[u8]>, API-wise, and thus hide that inner AlignedTo<64, [u8; 64]> type.
Else things become messier, because you'll have to keep your own Vec newtype around, somehow exposing those details.
Going for the former, you could then have:
// Desired alignement
desired_alignment!(0x40);
macro_rules! desired_alignment {( $A:tt ) => (
#[derive(Clone, Copy)]
#[repr(C, align($A))]
struct AlignedBytes([u8; $A]);
pub
struct BoxAlignedBytes /* = */ (
Box<[AlignedBytes]>,
);
impl ::core::ops::Deref for BoxAlignedBytes {
type Target = [u8];
fn deref (self: &'_ BoxAlignedBytes)
-> &'_ [u8]
{
unsafe {
::core::slice::from_raw_parts(
::core::ptr::addr_of!(*self.0).cast(),
self.0.len() * $A,
)
}
}
}
impl ::core::ops::DerefMut for BoxAlignedBytes {
fn deref_mut (self: &'_ mut BoxAlignedBytes)
-> &'_ mut [u8]
{
unsafe {
::core::slice::from_raw_parts_mut(
::core::ptr::addr_of_mut!(*self.0).cast(),
self.0.len() * $A,
)
}
}
}
impl<'r> From<&'r [u8]> for BoxAlignedBytes {
fn from (buf: &'r [u8])
-> BoxAlignedBytes
{
let mut len = buf.len();
if len % $A > 0 {
// pad with `0`s.
len += ($A - len % $A);
}
let count = len / $A;
let mut boxed = Self(
vec![AlignedBytes([0; $A]); count]
.into_boxed_slice()
);
boxed[.. buf.len()].copy_from_slice(buf);
boxed
}
}
)} use desired_alignment;
If you wanted Vec growability capabilities, you'd have to stick to a Vec<AlignedBytes> and handle the .len() and .capacity() indexing over those big AlignedBytes contiguous chunks, basically reinventing Vec's API, but amended to fit this situation.
This is doable, but it requires way more unsafe: you'll have to play with the Allocator API yourself, by constructing a runtime Layout value. Afterwards, exposing a wrapper around this with BoxAlignedBytes API should be easy; but if you want both runtime alignment and growability it's gonna be very painful, and error-prone.
const-generic not suffice? Alignment needs are supposed to be known at compile-time by the end user, so a const generic (+ elain - Rust) ought to do the job@Yandros,consumers of the library needs data buffer with different alignment although there are only two types 64 and 512. So, Maybe I can handle that somehow. Also, the size of vector const change.
But I need Vec only, no other custom datatype will work.
I don't understand what you are asking here. The vector in my solution is aligned to 64 bytes, that's the whole point.
@H2CO3 In your Implementation datatype of kp is AlignedByteBuf. I need it to be Vec. Is it not possible. I need it as vector to ensure backward compatability. Help me understand If I am Wrong.
On nightly, you could use Vec::new_in with an allocator that enforces the minimum alignment you need, and on stable you can replace the global allocator so that all allocations are 512-byte aligned (which is also 64-byte aligned, satisfying both requirements).
Otherwise there's no reliable way to get a custom-aligned Vec<u8>. Even if you check the alignment that you actually have, there's no guarantee that it will remain when the vector reallocates.
No. Unless you
there is no way to guarantee that the Vec<u8> will handle correct alignement.
You will have to change the type involved, one way or another:
Are you sure about that? The documentation implies that any crate can specify the global allocator, but there will be a compile error if any program contains two such specifications:
The
#[global_allocator]can only be used once in a crate or its recursive dependencies.
Oh interesting. Well if anything it doesn't look like a third-party lib should ever be touching this. But yeah, if it's a library crate within you ends-in-a-binary workspace, I guess you could do that
To be clear, that's because when it re- or deallocates the memory it needs to pass the current layout to the allocator, and not because there is a general problem with having a higher alignment than expected by the type? I wouldn't normally ask but UB is very scary, and I need all the night lights you have...
Yes, you're right I should have clarified 
. Using a &T or a &mut T whiich happens to have higher alignment is fine, but indeed the allocator API is special insofar it "is allowed to remember" the Layout (basically a (size, align) pair) with which you queried the allocation, and trigger UB if you request a free (dealloc or realloc) of said pointer with a different Layout.
It's pretty easy to imagine an allocator that uses the alignment to pick a bucket of memory, if nothing else.
I skimmed the thread, but it looks like you could use aligned_vec, which allows you specify the alignment of the first item in an AVec as a generic parameter.
For example:
let mut aligned: AVec<u32, ConstAlign<128>> = AVec::new();
aligned.push(5);
should align the first element at a 128 byte boundary. It seems that it is also possible to use a runtime alignment value via the RuntimeAlign type.
Note that this crate is quite young, not widely used and contains some unsafe code, as it needs to reimplement large parts of std::Vec.
This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.