How to pass struct keys to a function to load data from JSON

[rajasankar]

Hi All,

I want to write a function that can load data from JSON strings. However, if there are different structs and Keys, I dont know how to pass to function.

#[derive(Serialize, Deserialize, Debug)]
pub struct TestOne
{
	pub a : u64,
	pub b : u64	
}

#[derive(Serialize, Deserialize, Debug)]
pub struct TestTwo
{
	pub c : String,
	pub d : u64	
}
let mut map : HashMap<u64,u64> = HashMap::new();
let file_read = File::open("file_one,json".to_string()).unwrap();
for line in BufReader::new(file_read).lines() 
{
        let mut line_struct: TestOne = serde_json::from_str(&line.unwrap().to_string()).unwrap();
	    map.insert(line_struct.a, line_struct.b);	
    }

Now, I want to make that as function to pass struct values so that I can reuse the same code. Something like this,

pub fn load_file(file_name : String, Struct_definition : ???, key : ??? , value : ??? ) ->HashMap<u64,u64>
{
let mut map : HashMap<u64,u64> = HashMap::new();
    let file_read = File::open(file_name).unwrap();
    for line in BufReader::new(file_read).lines() 
    {
            let mut line_struct: Struct_definition = serde_json::from_str(&line.unwrap().to_string()).unwrap();
    	    map.insert(line_struct.key, line_struct.value);	//want to use the struct values like this
        }
map
}

In other words, I want to write a common function for all these kind of loading files into memory.

Is this possible?

Thanks for your help.

[dtolnay]

Something like this could work:

let map = load_file(file_path, |line: TestOne| (line.a, line.b))?;

Full example:

use serde::{de::DeserializeOwned, Deserialize};
use std::collections::HashMap;
use std::fs::{self, File};
use std::hash::Hash;
use std::io::{self, BufRead, BufReader};
use std::path::Path;

#[derive(Deserialize)]
struct TestOne {
    a: u64,
    b: u64,
}

#[derive(Debug)]
pub enum Error {
    Io(io::Error),
    Json(serde_json::Error),
}

pub type Result<T> = std::result::Result<T, Error>;

impl From<io::Error> for Error {
    fn from(e: io::Error) -> Self {
        Error::Io(e)
    }
}

impl From<serde_json::Error> for Error {
    fn from(e: serde_json::Error) -> Self {
        Error::Json(e)
    }
}

pub fn load_file<P, T, F, K, V>(file_path: P, mut entries: F) -> Result<HashMap<K, V>>
where
    P: AsRef<Path>,
    T: DeserializeOwned,
    F: FnMut(T) -> (K, V),
    K: Eq + Hash,
{
    let mut map = HashMap::new();
    let file = File::open(file_path)?;

    for line in BufReader::new(file).lines() {
        let object = serde_json::from_str(&line?)?;
        let (key, value) = entries(object);
        map.insert(key, value);
    }

    Ok(map)
}

fn main() -> Result<()> {
    let file_path = "/tmp/demo-lines.json";
    fs::write(file_path, "{\"a\":1,\"b\":100}\n{\"a\":2,\"b\":200}\n")?;

    let map = load_file(file_path, |line: TestOne| (line.a, line.b))?;
    println!("{:#?}", map);

    Ok(())
}

[dtolnay]

Alternatively, if you have the names as runtime strings:

let map: HashMap<u64, u64> = load_file(file_path, "a", "b")?;

Complete example:

use serde::de::DeserializeOwned;
use std::collections::HashMap;
use std::fs::{self, File};
use std::hash::Hash;
use std::io::{self, BufRead, BufReader};
use std::path::Path;

#[derive(Debug)]
pub enum Error {
    Io(io::Error),
    Json(serde_json::Error),
}

pub type Result<T> = std::result::Result<T, Error>;

impl From<io::Error> for Error {
    fn from(e: io::Error) -> Self {
        Error::Io(e)
    }
}

impl From<serde_json::Error> for Error {
    fn from(e: serde_json::Error) -> Self {
        Error::Json(e)
    }
}

pub fn load_file<P, K, V>(file_path: P, key: &str, value: &str) -> Result<HashMap<K, V>>
where
    P: AsRef<Path>,
    K: Eq + Hash + DeserializeOwned,
    V: DeserializeOwned,
{
    let mut map = HashMap::new();
    let file = File::open(file_path)?;

    for line in BufReader::new(file).lines() {
        let object: serde_json::Value = serde_json::from_str(&line?)?;
        let key = K::deserialize(&object[key])?;
        let value = V::deserialize(&object[value])?;
        map.insert(key, value);
    }

    Ok(map)
}

fn main() -> Result<()> {
    let file_path = "/tmp/demo-lines.json";
    fs::write(file_path, "{\"a\":1,\"b\":100}\n{\"a\":2,\"b\":200}\n")?;

    let map: HashMap<u64, u64> = load_file(file_path, "a", "b")?;
    println!("{:#?}", map);

    Ok(())
}

[rajasankar]

Thanks for the detailed code @dtolnay Appreciated. I will test and post the final code here.

I dont know I could do these things in Rust. I have moved lot of code from Python but this one is kind of difficult for me.