Hello,
I am learning scientific computing , high performance computing. I am very at the beginning of it.
Does Rust language has limits for calculating data ?
I mean analyzing a TB or more storage file blocks ?
Thanks and happy hacking !
Rust has no notable limits in working with large data, beyond those imposed by the hardware.
Tangentially, some general advice: for such large data sets, you will often want to process them in a streaming fashion as much as possible — that is, reading, processing, and writing are done concurrently or in reasonably-sized chunks, rather than reading the entire data set into memory at once. This is a standard technique going back to the earliest days of computing, when computers barely had enough memory to hold one line/record at a time, but it’s somewhat out of fashion for general-purpose computing today due to the fact that most files aren’t that big. But given that you have working on big data as your goal, it might be good to practice this sort of thing. The classic sort of simple program that you might do this with is a program that reads a text file one line at a time, and does its work as it reads each line.
Thanks, I am gonna think implementation then !