Help understanding some idiomatic Rust for error handling, Result<T,Error> and usage of Structure defaults

[sarvi]

I want to write a function that does the following

1. construct a path from some components using PathBuf
2. Read & Parse the yaml file from the path, into strongly typed Configuration Structure that is a mix of HashMaps, Vectors of String, and possibly other fields.
3. Start with a default instance of this Configuration structure with default values for all fields.
4. Read & Parse a list YAML config files, that might each define subset of the fields possible in the struct.
5. Arriving at a final Config struct instance where the values defined in last file on the list takes precedence over the default and previous yaml files.

Writing this code ends up being quite verbose and non-elegant. I suspect it is my limited knowledge on Error handling, Result<T,E> return types an

Couple of problems I ran into.

1. Defining a function that takes some config file path components and returns the Structure ran into multiple issues relating Generic Types and what Error to use in Result<T,Error>. The function handles PathBuf operaitons, file read operations, and serde_yaml::from_str(content.as_str()) which all can return differet types of Errors. But using std::error::Error causes compile time errors saying size is not known.

pub fn configread<T>(basepath: &str, conf: &str) -> Result<T,Error> {}

2. For some cases, I want the program to error out, with a message. The closses I got to achieving that is something like the following that still generates more than just an error and exit.

let config : T = serde_yaml::from_str(content.as_str()).unwrap_or_else(|e| {
        debug(format_args!("Error parsing {:?} file {}\n", e, cfgpath.to_string_lossy()));
        panic!();
    });

I couldnt get serde_yaml::from_reader(open(filepath)) to work so I am now reading the content into string and calling from_str() as above.

So my question, what is the most idiomatic and cleanest way to achieve the following pseudo code in Rust

pub fn configread<T>(basepath: &str, conf: &str) -> Result<T,Error>
where
    T: de::DeserializeOwned,
{
    let cfgpath = Path::new(basepath)
                         .parent().unwrap_or_else(|| {
        debug(format_args!("Error accessing path parent of {}\n", basepath));
        panic!();
    });
    debug(format_args!("cfgpath: {}\n", cfgpath.to_string_lossy()));
    let cfgpath = cfgpath.parent().unwrap_or_else(|| {
        debug(format_args!("Error accessing path parent of {}\n", cfgpath.to_string_lossy()));
        panic!();
    });
    debug(format_args!("cfgpath: {}\n", cfgpath.to_string_lossy()));
    if let Err(cfgpath) = Path::new(cfgpath).join("config").join(conf)
                         .canonicalize() {
        return to so we can skip this file if the file does not exist.
    };
    debug(format_args!("cfgpath: {}\n", cfgpath.to_string_lossy()));
    let content = std::fs::read_to_string(cfgpath.to_owned()).unwrap_or_else(|e| {
// If the file exist, reading and parsing should be successfull OR panic/exit
        debug(format_args!("Error reading {:?} file {}\n", e, cfgpath.to_string_lossy()));
        panic!();
    });
    let config : T = serde_yaml::from_str(content.as_str()).unwrap_or_else(|e| {
        debug(format_args!("Error parsing {:?} file {}\n", e, cfgpath.to_string_lossy()));
        panic!();
    });
    config
}

#[derive(Default, Debug, serde::Deserialize, PartialEq)]
pub struct Config {
    only64bit: String,
    myhash: HashMap<String, String>,
    myvecstrings: Vec<String>,
    myvecofints: Vec<i32>,
}

const CONFIG_DEFAULTS: &str = "
wsonly : true
var2: 1
var3: something
pi: 3.14159
 xmas: true
 french-hens: 3
 calling-birds: 
   - huey
   - dewey
   - louie
   - fred
 xmas-fifth-day: 
   calling-birds: four
   french-hens: 3
   golden-rings: 5
   partridges: 
     count: 1
     location: "a pear tree"
   turtle-doves: two
";

pub fn readmultipleconfigs() -> Config {
// If any of the config file exists but has a parse error, the program should exit with parse error informaiton
    let defconf:Config = serde_yaml::from_str(CONFIG_DEFAULTS.as_str()).unwrap(); 
    let conf:Config = { ..configread("/a/b/c/d", "file1.ini"), ..DEFAULT};
    let conf:Config = { ..configread("/a/b/c/d", "file2.ini"), ..conf};
    let conf:Config = { ..configread("/a/b/c/d", "file_nonexistent.ini"), ..conf};   // ignored
    let conf:Config = { ..configread("/a/b/c/d", "file3.ini"), ..conf};
    config
}

What is the best way to achieve this in Rust.
I am not expecting anyone to write this full piece. Even pointers on simple idiomatic ways of achieving pieces of this puzzle that I can put to gether would be helpful

[H2CO3]

@sarvi

But using std::error::Error causes compile time errors saying size is not known.

1. First of all, std::error::Error is not an error type, it's a trait that specific error types implement. You have to say what specific error type you want to put in your Result.

2. your panic!()s are unwarranted inside a function where an error is expected or recoverable. You should propagate the errors instead using the ? operator, then handle them at the top-level, e.g. main().

3. If you intend your Config struct to have default values, then it's not exactly pretty to derive Default just to later override it by weakly-typed values loaded from a string. Why not implement Default manually with the expected, strongly-typed default values in the first place?

4. debug(format_args!(…)) is unnecessarily complicated. Use println!() or eprintln!() for printing to standard out and standard error, respectively.

5. In the code std::fs::read_to_string(cfgpath.to_owned()), the to_owned() is unnecessary, as the read_to_string() function doesn't need to own the filename, it only uses it to open the file. It's also unnecessary to separately read the file into a string and then use from_str() on the string contents – just use from_reader() directly.

6. You should respect naming conventions. E.g. function names are supposed to have underscore separators between words. So readmultipleconfigs() should be read_multiple_configs(). Furthermore, type annotation of variables should have a space between the colon and the type, so foo: Type and not foo:Type.

7. What you want regarding multiple config files is not possible by just using the struct update syntax (Config { values, ..default }). It looks like you want to parse fragments of the config from different files and merge them. For that, you will need to support missing fields, something that could be handled e.g. by #[serde(default)] and making the potentially missing fields into Options. In addition, you'll need to manually merge the configs. In that regard, I don't get what exactly you want, but:

All in all, you could do something like this playground:

use std::path::Path;
use std::fs::File;
use std::collections::HashMap;
use serde::de::DeserializeOwned;
use serde_derive::Deserialize;

#[derive(Debug, Clone, PartialEq, Eq, Deserialize)]
pub struct Config {
    #[serde(default)]
    foo: Option<String>,
    bar: HashMap<String, String>,
    #[serde(default)]
    numbers: Vec<i32>,
}

impl Default for Config {
    fn default() -> Self {
        Config {
            foo: "something".to_owned().into(),
            bar: HashMap::new(),
            numbers: vec![1, 2, 3],
        }
    }
}

pub fn read_config<T>(basepath: &str, conf: &str) -> Result<T, String>
where
    T: DeserializeOwned,
{
    let cfgpath = Path::new(basepath).parent().ok_or_else(|| {
        format!("Error accessing path parent of {}\n", basepath)
    })?;
    let cfgpath = cfgpath.parent().ok_or_else(|| {
        format!("Error accessing path parent of {}\n", cfgpath.display())
    })?;
    let cfgpath = Path::new(cfgpath).join("config").join(conf).canonicalize().map_err(
        |e| e.to_string()
    )?;
    let file = File::open(cfgpath).map_err(|e| e.to_string())?;
    serde_yaml::from_reader(file).map_err(|e| e.to_string())
}

pub fn read_multiple_configs(basepath: &str, paths: &[&str]) -> Result<Config, String> {
    let mut config = Config::default();
    
    for &path in paths {
        let update: Config = read_config(basepath, path)?;
        
        // apply update with existing/non-empty fields
        if let Some(foo) = update.foo {
            config.foo = Some(foo);
        }
        if update.bar.len() > 0 {
            config.bar = update.bar;
        }
        if update.numbers.len() > 0 {
            config.numbers = update.numbers;
        }
    }
    
    Ok(config)
}

fn main() -> Result<(), String> {
    let paths = ["file1.ini", "file2.ini", "file_nonexistent.ini", "file3.ini"];
    let config = read_multiple_configs("/a/b/c/d", &paths)?;

    println!("{:#?}", config);

    Ok(())
}