Hi, I'm having a little trouble reading the Vec
documentation.
Vec::with_capacity
The documentation of Vec::with_capacity
says:
Panics if the new capacity exceeds
isize::MAX
bytes.
I don't know what exactly "capacity" refers to here:
let _ = Vec::<i32>::with_capacity(usize::MAX);
// thread 'main' panicked at 'capacity overflow'
let _ = Vec::<()>::with_capacity(usize::MAX);
// ok
Obviously, this capacity does not refer to the parameter n
of Vec::with_capacity(n)
, because the latter example does not painc. So I'm guessing that capacity refers to the memory size (aka. size_of::<T>() * len
), but the following example is still not panic:
let result = std::panic::catch_unwind(|| {
let _ = Vec::<u8>::with_capacity(isize::MAX as usize + 42);
// memory allocation of 9223372036854775849 bytes failed
// I'm not very confident, but this doesn't seem to be a panic.
});
assert!(result.is_err());
I read the source code of Vec::with_capacity
, but I didn't find any check related to this.
slice::binary_search_by
Here is the source code of slice::binary_search_by
.
pub fn binary_search_by<'a, F>(&'a self, mut f: F) -> Result<usize, usize>
where
F: FnMut(&'a T) -> Ordering,
{
let mut size = self.len();
let mut left = 0;
let mut right = size;
while left < right {
let mid = left + size / 2;
// ^^^^^^^^^^^^^^^ Here it was `(left + right) / 2`
// But because of ZSTs, it was changed to avoid overflow.
/* ... */
}
/* ... */
}
pointer::offset
Some documentation from std::pointer::offset
:
The compiler and standard library generally tries to ensure allocations never reach a size where an offset is a concern. For instance,
Vec
andBox
ensure they never allocate more thanisize::MAX
bytes, sovec.as_ptr().add(vec.len())
is always safe.Most platforms fundamentally can’t even construct such an allocation. For instance, no known 64-bit platform can ever serve a request for 263 bytes due to page-table limitations or splitting the address space. However, some 32-bit and 16-bit platforms may successfully serve a request for more than
isize::MAX
bytes with things like Physical Address Extension. As such, memory acquired directly from allocators or memory mapped files may be too large to handle with this function.
Question
Vec::with_capacity
saysVec::len()
should no larger thanisize::MAX
, but doesn't check it.slice::binary_search_by
(and this PR) indicates thatVec::len()
should no larger thanisize::MAX
, except for ZSTs.pointer::offset
saysVec::len() * size_of::<T>()
should no larger thanisize::MAX
, butVec::with_capacity
doesn't check it.
So here is my question:
- Does the standard library specify the maximum value of collections length (e.g.
Vec::len()
), and if so, what is it? - As mentioned above, in the documentation for
Vec::with_capacity
, the use of the word "capacity" is somewhat ambiguous. Do we need to fix it? - Do we need to add
assert!(capacity <= isize::MAX as usize)
forVec::with_capacity
and other collection types?