How to save a vector of float 64

Hi,
Is there an example of how I can save a vector of float (Vec) to a file and then read it and store it in a variable?
Thank you

Any requirements on what kind of format you want the information in the file to have?

2 Likes

No requirement on the type of format. I usually prefer to reduce dependencies on external libraries, so anything that use std will be better in my case. Also if it can be in a simple txt file is better as it can be read by a human. :grin:

Here’s some demonstration implementation using Serde and the JSON format. Note that the same kind of code also works for lots of other types that support de-/serialization via Serde, not only Vec<f64>.

use std::error::Error;
use std::fs::File;
use std::io::BufReader;
use std::io::BufWriter;
use std::io::Write;
use std::path::Path;

fn read_vec_from_file(path: impl AsRef<Path>) -> Result<Vec<f64>, Box<dyn Error>> {
    // Open the file in read-only mode with buffer.
    let file = File::open(path.as_ref())?;
    let reader = BufReader::new(file);

    // Read the JSON contents of the file as a `Vec<f64>`.
    let v = serde_json::from_reader(reader)?;

    // Return the `Vec<f64>`.
    Ok(v)
}

fn write_vec_to_new_file(path: impl AsRef<Path>, value: &[f64]) -> Result<(), Box<dyn Error>> {
    let file = File::create(path.as_ref())?;
    let mut writer = BufWriter::new(file);
    serde_json::to_writer(&mut writer, value)?;
    writer.flush()?;
    Ok(())
}

fn main() {
    let v1 = vec![1.0, 1.5, 2.0, 2.5];
    println!("v1: {v1:?}");
    write_vec_to_new_file("test.json", &v1).unwrap();

    // for demonstration purposes let's look at the file, too
    println!("Here's what the file looks like:");
    println!("========================================");
    println!("{}", std::fs::read_to_string("test.json").unwrap());
    println!("========================================");

    // get back the vector
    let v2 = read_vec_from_file("test.json").unwrap();
    assert!(v1 == v2);
    println!("v2: {v2:?}");
}

(run in the playground)

I’ve written this before reading about the question to reduce dependencies; I’ll take a look at how easy this might be with std along; probably not all that hard…

2 Likes

Tks @steffahn, super useful! Looks like serde does not support csv format. Which library do you recommend for csv file?

This is an example without dependencies:

use std::error::Error;
use std::fs::File;
use std::io::BufReader;
use std::io::BufWriter;
use std::io::Write;
use std::io::BufRead;
use std::path::Path;

fn read_vec_from_file(path: impl AsRef<Path>) -> Result<Vec<f64>, Box<dyn Error>> {
    // Open the file in read-only mode with buffer.
    let file = File::open(path.as_ref())?;
    let mut reader = BufReader::new(file);

    let mut v = vec![];

    let mut buf = String::new();

    while reader.read_line(&mut buf)? != 0 {
        v.push(buf.trim_end().parse()?);
        buf.clear();
    }

    Ok(v)
}

fn write_vec_to_new_file(path: impl AsRef<Path>, value: &[f64]) -> Result<(), Box<dyn Error>> {
    let file = File::create(path.as_ref())?;
    let mut writer = BufWriter::new(file);

    for x in value {
        write!(writer, "{x}\n")?;
    }

    writer.flush()?;
    Ok(())
}

fn main() {
    let v1 = vec![1.0, 1.5, 2.0, 2.5];
    println!("v1: {v1:?}");
    write_vec_to_new_file("test.json", &v1).unwrap();

    // for demonstration purposes let's look at the file, too
    println!("Here's what the file looks like:");
    println!("========================================");
    println!("{}", std::fs::read_to_string("test.json").unwrap());
    println!("========================================");

    // get back the vector
    let v2 = read_vec_from_file("test.json").unwrap();
    assert!(v1 == v2);
    println!("v2: {v2:?}");
}

(run in the playground)

2 Likes

It's pretty easy with std, but serde is pervasive and mature enough that it's likely not worth inventing your own format just for the sake of avoiding a dependence on serde.

That said, using std only, it's something like:

pub fn write_to_file<P>(destination: P, values: &[f64]) -> io::Result<()>
where
    P: AsRef<Path>
{
    let mut file = BufWriter::new(File::create(destination)?);
    values.iter().try_for_each(|&x| writeln!(file, "{}", x))
}

pub fn read_from_file<P>(source: P) -> io::Result<Vec<f64>>
where
    P: AsRef<Path>
{
	BufReader::new(File::open(source)?)
		.lines()
		.map(|line| line?.trim().parse().map_err(|e| io::Error::new(io::ErrorKind::Other, e)))
		.collect()
}
2 Likes

Wow I don't understand all your code but at least it is giving me stuff to learn. Thank you!

It does. You are probably misunderstanding what Serde is. Serde is not tied to a particular format, it's a generic library for hooking up arbitrary serializable types (including user-defined types) with arbitrary serialization formats (e.g., JSON, CSV, cbor, BSON, you name it) through a unified data model.

The unsurprisingly-named csv crate supports writing out Serde-serializable types in CSV format.


Actually, if you are trying to read and write CSV, then you have tabular data. Are you sure you wouldn't be better served by an embeddable database (e.g. SQLite) instead of writing and reading whole files? The story rarely ends at "I need to write out these floats"; you will presumably want to do something else with them anyway, especially if you are asking how to read them back too.

3 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.