Borrowing iterator for file-parsing without lifetimes using unsafe Rust

[programmerjake]

@H2CO3

You would use e.g. Rc::from(vec![0; size]) for that.

actually it's more efficient (1 allocation instead of 2) to use (0..size).map(|_| 0).collect::<Rc<[u8]>>().

if you want the currently most efficient way to create a cleared Rc<[u8]> of dynamic size and you don't mind both nightly and unsafe, you can use unsafe { Rc::<[u8]>::new_zeroed_slice(size).assume_init() }, since that specifically asks the allocator for zeroed memory, which is more efficient since it can give you memory it already cleared or the OS already cleared, rather than having to always clear it as a separate step after allocation.

[parasyte]

@isibboi

Parasyte, I don't see how your code works when the input is not given as a &[u8], but any type that implements Read instead. Remember that the scenario is "streaming", i.e. we don't want to load the whole file into memory first, but process its entries one-by-one.

Yes, but since you are streaming you can just throw away the iterator when you need more data. Pick up where you left off later. Supporting streaming adds some complexity, but it's still easy enough to do all of this in efficient safe Rust without jumping through too many hoops: Rust Playground