Config loading and transformation patterns

[augis122]

Problem

I'm looking for design pattern suggestions, possibly including crate recommendations.

In most of my projects there's some configuration the app loads from a file at the start of the runtime. The problem happens when I need to transform the configuration data: the code becomes dirty, and the problem gets bigger as the number of configuration variables and nesting increases.

Right now I'm using serde deserialization with RON files in the pattern outlined bellow. I'm looking to change this approach.

Example

This contrived example is of a car trip app. In real projects the config file is longer and more complex.

Suppose I want to refactor the code bellow so the RON file accepts color as a hex String rather than [f32; 3], but Car object construction will still need color as [f32; 3].

src/main.rs

mod config;

fn main() {
  let config = config::init("config.ron");

  // use configs in various ways, like to construct structs
  let car = Car::from_config(&config.car_config);
  let fuel_cost_efficiency = car.fuel_cost_efficiency();
  let trip_cost = fuel_cost_efficiency * config.road_length;
  let fun_factor = (car.loudness() + car.red_amount()) / trip_cost;
}

src/config.rs

#[derive(serde::Deserialize)]
pub struct CarConfig {
  pub fuel_type: String,
  pub color: [f32; 3],
}

#[derive(serde::Deserialize)]
pub struct Config {
  pub car_config: CarConfig,
  pub road_length: f32,
}

pub fn init(filename: &str) -> Config {
  let config_str = std::fs::read_to_string(filename)
    .expect(&format!("No configuration file `{filename}` found"));
  ron::from_str(&config_str).expect("Error while reading the configuration file")
}

config.ron

(
  car_config: (
    fuel_type: "gasoline",
    color: [1.0, 0.0, 0.0],
  ),
  road_length: 100.0,
)

PS: Solutions I've considered

I can think of 3 ways to refactor the code but none of them are ideal:

• Splitting the CarConfig struct into: CarConfigRaw that gets deserialized and stores color as String, and CarConfig that gets passed to Car::from_config and stores color as [f32; 3]. The problem appears when there are many nested config structs and each one has many fields. Converting between *ConfigRaw and *Config requires affecting all the parent structs and copying the unaffected fields one-by-one. This creates ugly code and requires repetition.
• Converting from a hex String to [f32; 3] inside Car::from_config. This is not ideal because now everything is coupled: config validation and transformation happens deep inside unrelated code.
• Constructing Car using Car::new(car_config.fuel_type, transform(car_config.color)) instead of Car::from_config(&car_config). This requires repetition when there are many fields.

[ogeon]

In the case of parsing and serializing to hex values, you can use some of the attributes that allow you to change the (de)serialization format per field: Field attributes · Serde

[jwodder]

If you just want to change how one field gets deserialized, I'd go with using #[serde(deserialize_with = ...). The serde_with crate might help you out here.

[derspiny]

Taking the color example in isolation, there's a decomposition here I think is quite nice:

1. Introduce a type representing a string containing a hex triple, which handles deserialization and validation (and can therefore promise to callers that the value no longer needs validation).
2. Expose a colour() helper on CarConfig that takes the field (containing a now-validated hex triple string) and produces a colour as [f32; 3] from it.

The serialized form would then look as you wish, while code can call color() to get the colour in the form most useful inside your program. The transformation from stored format to the useful format is cheap enough that there's little need to store it.

use std::convert::TryFrom;

#[derive(Clone, Copy, serde::Deserialize)]
#[serde(try_from = "String")]
pub struct HexTriple(pub u8, pub u8, pub u8);

impl TryFrom<String> for HexTriple {
    type Error = std::convert::Infallible; // replace with an error type

    fn try_from(value: String) -> Result<Self, Self::Error> {
        unimplemented!()
    }
}

#[derive(serde::Deserialize)]
pub struct CarConfig {
  pub fuel_type: String,
  pub color: HexTriple,
}

impl CarConfig {
    pub fn f32_color(&self) -> [f32; 3] {
        let HexTriple(r, g, b) = self.color;
        [
            r as f32 / 256.0,
            g as f32 / 256.0,
            b as f32 / 256.0
        ]
    }
}

You could, equally, implement HexTriple as struct HexTriple(f32, f32, f32); and do the conversion from [0..255] to [0..1] there, instead.

More generally, your problem statement feels to me like a symptom of trying to make the CarConfig type handle both the configuration of the car and all subsidiary concerns; Serde, in particular, doesn't force you to do this, so if adding more types makes the problem easier to break down, do so.

[mroth]

In my opinion, the main issue lies in this signature:

impl Car {
    pub fn from_config(config: &CarConfig) -> Self {
        todo!()
    }
}

This tightly couples unrelated concerns (often called "domains"):

• Configuration domain (struct Config & struct CarConfig)
• Application domain (struct Car)

A better approach is to decouple them using free functions and composition.

@augis122

Constructing Car using Car::new(car_config.fuel_type, transform(car_config.color)) instead of Car::from_config(&car_config).

This is almost the right approach.

@augis122

This requires repetition when there are many fields.

You need some way to construct your application objects. If a simple new() becomes unwieldy, an alternative is needed. In the Rust community, the builder pattern is quite popular (though there are other approaches I personally prefer).

For better composition, define functions that create objects from their input parameters:

fn car_from_car_config(cfg: &CarConfig) -> Car {
    let fuel_type = cfg.fuel_type;
    let color = cfg.color;
    ...
    Car::new(fuel_type, color, ...)
}

Then, in main:

fn main() {
    let config = config::init("config.ron");

    let car = car_from_car_config(&config.car_config);
    ...
}

Now, if you decide to change color from [f32; 3] to String, you:

• Create a conversion function:

  fn color_from_str(color: &str) -> [f32; 3] { ... }
  

• Modify car_from_car_config():

  let color = color_from_str(&cfg.color);
  

This keeps your logic flexible and decouples the configuration from the application.

If you have complex dependencies like this:

let fuel_cost_efficiency = car.fuel_cost_efficiency();
let trip_cost = fuel_cost_efficiency * config.road_length;
let fun_factor = (car.loudness() + car.red_amount()) / trip_cost;

...then they are what they are.

Find meaningful groupings and extract functions to improve composition:

fn trip_cost_from_car_and_road_length(car: &Car, road_length: f32) -> f32 {
    car.fuel_cost_efficiency() * road_length
}

fn fun_factor_from_car_and_trip_cost(car: &Car, trip_cost: f32) -> f32 {
    (car.loudness() + car.red_amount()) / trip_cost
}

Yes, the function names are long, but does it really matter? They’re mostly used once in main, and clarity is far more valuable than brevity.