Hi guys, I'm trying to write a program where a large vector of u64's, with about 53000 elements of predefined values, needs to be stored and used. However, the program takes a long time to compile, about 24 seconds on my Ryzen 2700X machine. It was taking 47 seconds previously if I tried assigning t…

You may also want to post this on the internals forum . A lot more people working on rustc itself tend to hang out there. I'm guessing this is because rustc needs to parse the long array literal and keep track of a non-trivial amount of extra information (e.g. span). This isn't great for memory usag…

I have no idea but I a suspicion that the macro building the array is taking a long time to generate all the initialization code. vec![ 0x8000000000000, 0x4000000000000, ... ] Is that matrix immutable in your finished code? I have a program that uses a table of 40,000 prime numbers. I gen…

Thanks for the reply! Yes, that matrix is immutable in my code. In my case, those are precomputed values used in general while generating Sobol low-discrepancy sequences, so I don't even need to generate them; they're just there. So I wonder what's wrong...

Thanks! I see. Maybe I'll go with storing the values some file and reading from there if all else fails.

If you can read them from a file you can have the build.rs script do that at compile time and create a static array of values which then gets compiled into your code. I reckon that will be pretty fast and then your executable does not need to carry that data file around.

You can also use include_str or include_bytes. You then either parse them or if you store them in binary form simply use a transmute from &[u8] to &[u64] (or whatever).

[image] ZiCog: If you can read them from a file you can have the build.rs script do that at compile time and create a static array of values which then gets compiled into your code. That's the equivalent of what he's doing at the moment. Although instead of creating the static array of values…

The macro is not at fault. First of all, it doesn't happen in debug mode. Finished dev [unoptimized + debuginfo] target(s) in 1.15s Using cargo rustc --release -- -Z time-passes shows the following: Compiling large_vec_allocation v0.1.0 (/home/skade/Code/rust/large_vec_allocation_repro) …

[image] Michael-F-Bryan: That's the equivalent of what he's doing at the moment. Although instead of creating the static array of values using a build script, they're part of a file that already exists. The problem is that rustc still needs to parse and compile that static array (which would be …

Indeed, it seems that Debug does not have this problem. In that case, does it make sense to raise an issue in Github?

Long compile times for a vector with 50K u64 values

help

CAD97 December 12, 2019, 2:51pm 14

Note that in this case, you are required to ensure that the array is u64-aligned and also handle the little-endian/big-endian cases.

1 Like

Topic		Replies	Views
Large vectors take a long time to compile help	5	735	July 23, 2020
Std library inclusion policy, and a data point of compilation times	52	7170	October 24, 2022
Possible to merge static arrays at compile time? help	8	2619	December 3, 2021
Possible to calculate array values during compilation? community	6	777	October 5, 2020
Is a 6000 line enum, included ~125 times, likely to trigger a 5s compile time increase?	7	456	January 17, 2023

Long compile times for a vector with 50K u64 values

Related topics