Generics vs type erasure

Hi,

I have started working on a package supporting time series with the long-term aim to transform it into tool for handling energy price indexes. From the very beginning my idea was to use generics for parameters like time resolution, commodity, currency. Such an approach allows straightforward implementation of many useful features: easy conversion of time series into different time resolutions, avoiding aggregation of values denominated in different currencies etc.

So basic definition of time series could look like this:

The problem appears when I start to work with actual data. Main issue that I am struggling with is Rust expectation of precise output type when constructing a generic struct. If I get the data from SQL table I cannot know ex ante the value of ResolutionModel parameter (having separate SQL table for each ResolutionModel is not an option). I know that I can box the trait or use enum but then I would no longer be able to use it as type parameter.

I would appreciate any advice on the subject. Is there any way to have anonymity in db and avoid type erasure on Rust side? Is there any standard approach that professionals use in this kind of situations?

Thank you in advance,
Piotr

The same code after minor correction:

Are you trying to implement something similiar to this? (psuedocode)

fn read_db(d) -> Resolution<impl ResolutionModel> {
   let resolution = type_of_resolution();

   if resolution == "Hours" {
      return Resolution::<Hours>::read_db(d);
   }
   if resolution == "Minutes" {
      return Resolution::<Minutes>::read_db(d);
   }
}

If so, I think returning Resolution<ResolutionModelEnum> and perform a downcast would be fine. (or use trait object and downcast_rs.

// By perform a downcast, I mean something like this

enum ResolutionModelEnum {
  Hours,
  Minutes,
}


impl Resolution<ResolutionModelEnum> {
    fn minutes(self) -> Resolution<Minutes> {
       match self.model {
          ResolutionModelEnum::Minutes => Resolution { id: self.id, model: Minutes },
          _ => panic!()
       }
    }
}

// or with downcast_rs

impl Resolution<Box<dyn ResolutionModel>> {
    fn downcast<T: ResolutionModel>(self) -> Resolution<T> {
       Resolution { id: self.id, model: self.model.downcast().unwrap() },
    }
}
1 Like

Thank you for suggestion! At a glance downcast_rs looks promising. I will play with it and get back.

Thank you for the hint about downcast_rs. It is good to know this technique but I do not think this is what I need. If I understand correctly downcasting is about converting from trait object into concrete type. If I new this concrete type I would not need downcasting at all, I would create the struct directly from SQL data. The thing is that I do not know the type (its generic parameters to be precise) when pulling the data from SQL. I only know that it implements certain trait but it seems this is not enough.

I would appreciate your feedback if I got it right. Thanks again!

Yes, this is the case. If you want a concrete type, you have to know the type at some point. My assumption is that,

  1. You read SQL data into an unified RawSqlData type.
  2. And you can infer the type from RawSqlData.
  3. Then you perform a downcast-like action to get a concrete type.

If there's no way to perform step 2, then where is the datatype info stored? Maybe you could write some pseudocode to demonstrate what you want to achieve?

I think what I mean can be simplified to following problem:

const QUERY: &[(usize, &str)] = &[(1, "hours"), (2, "days")];

trait ResolutionModel {}

struct Hours;
impl ResolutionModel for Hours {}

struct Days;
impl ResolutionModel for Days {}

struct Resolution<R: ResolutionModel> {
    id: usize,
    model: R
}

fn main() {
    for row in QUERY {
        // let resolution = 
    }
}

Is it possible to come up with constructor that takes row and produces Resolution with type parameter specified in row[1] i.e.:

  • Resolution<Hours> for first row,

  • Resolution<Days> for second row.

I assume the answer is no. If I am correct what are the reasonable alternatives? Is boxing the trait (and losing type parameters) the only way to go?

Thanks,
Piotr

Would static dispatch with enum helps? playground
It's a little boilerplate, and luckily enum_dispatch do the boilerplate work. example[1]


  1. Click the third green round button to see the macro expansion. ↩ī¸Ž

This obviously makes sense but Resolution is not generic anymore. I really hoped to keep Resolution generic so I could use the ResolutionModel parameter in other structs build upon Resolution e.g. I could allow easy conversion (between different Resolutions) of TimeSeries using From trait instead of normal function which would be far less elegant.
I guess it is either type erasure (trait objects or enums) on Rust side or 'one type - one sql query' approach.

I'm still confused by what you're looking for. You want a heterogenous collection as the result, but you want to provide the type of the collection?

I assume you mean the general type of the query, eg one table has Pizza and Burger items, and another has Cat and Dog items, and you want to write something like query::<Food> returning Vec<Food> collection and query::<Pet> returning a Vec<Pet>?

This doesn't sound quite right from what you've been saying, but it's as close as I can guess as to what you might be wanting.

If so, its pretty straightforward to define a enum Food { Pizza, Burger } etc, then have a trait that transforms raw RowData into Self (this can be the standard TryFrom<RowData>) on each of these.

I am not interested in the collection. I want a generic struct constructor with concrete type parameter (Hours or Days) known at run-time and provided using SQL query. So it would yield Resolution<Hours> for row[0] and Resolution<Days> for row[1].

So I guess I need a constructor like:

fn resolution<R: ResolutionModel>(row: (usize, &str)) -> Resolution<R> { ... }

The problem with this definition is that R cannot be inferred from &str. I could rewrite the constructor like this:

fn resolution<R: ResolutionModel>(id: usize, model :R) -> Resolution<R> { ... }

but model: R still would have to be derived from &str in the main function. And this is the impossible part I guess... Is it clear now? Does my reasoning make sense?

That's what FromStr for. playground

Whether you implement the conversion using FromStr or match you still have to declare a concrete type parameter R for Resolution<R>. And you do not know this type in advance, you get to know it only at run-time when executing the SQL query. In the example you posted (line 19) you have:

let _: Resolution<Hours> = resolution(row);

so it works fine for row[0] but panics for row[1].

And this?

Rust is static-typed. What exactly you want is impossible.

In this example Resolution is not generic.

Thanks. This is what I supposed but wanted to be 100% sure before changing approach. So basically there is following alternative: either erase the type (via enum or trait objects) or change the db/query so I get one type that I know in advance?

Generic parameters require you to know the type exactly, which become a hurdle when parsing since it's happens in runtime. You might be interested in Any trait with downcasting.

It's easy to parse and convert between different resolutions of TimeSeries via enum.

Thank you. Will look into Any.

playground

for &(id, s) in QUERY {
    let _ = parse_hours(id, s); // parse to hours if hours, if not, then do the convertion behind the scene
}

fn parse(s: &str) -> Result<Box<dyn Any>> {
    match s {
        "hours" => Ok(Box::new(Hours)),
        "days" => Ok(Box::new(Days)),
        _ => Err(()),
    }
}
fn parse_hours(id: usize, s: &str) -> Result<Resolution<Hours>> {
    let a = parse(s)?;
    Ok(if let Ok(model) = a.downcast() {
        Resolution { id, model: *model }
    } else {
        // convertion from other kind to hour
        Resolution { id, model: Hours }
    })
}