Hi, I want to write a wrapper for a library with a more convenient api. The library I want to wrap works like this:
The user registers the signals he wants to collect of the device with a Vec<u32>
Collect the data with get_data(n: usize) -> Vec<f64>
Convert to the data to the correct semantic types, like f32, f64, i32, timestamps...
The structure of the vector is row wise: The length is v.len() == n * num_signals with v[ind_sample*num_signals + ind_signal] being the data of sample ind_sample of signal ind_signal.
What I want is an api that handles the indexing and conversion of the data. Is it possible that the user defines a collection (like tuple or struct of signals) and with that he gets something like: fn get_iterator(data: &[f64], collection_of_signals) -> Iterator<Item = CollectionOfConvertedF64>>? I guess I need a macro for that, right?
I started with a trait:
type ID = u32;
trait Signal {
const ID: ID;
type Output;
fn convert(&self, value: f64) -> Self::Output;
}
struct S1 {}
impl Signal for S1 {
const ID: ID = 1;
type Output = u32;
fn convert(&self, value: f64) -> Self::Output {
value as u32
}
}
And for a single signal I can create the iterator with conversion, but how can I do that for a generic collection?
you can use slice::chunks() to turn the data array into an iterator of chunks, then you can implement conversions from slice of raw floating point values to the type, here's an sketch of it, it's incomplete but you should see the method:
trait Signal {
const COLUMN_COUNT: usize;
fn from_raw(row: &[f64]) -> Self;
}
fn as_signals<S: Signal>(raw: &[f64]) -> impl Iterator<Item=S> {
raw.chunks(S::COLUMN_COUNT).map(S::from_raw)
}
// example for simple signal
impl Signal for S1 {
const COLUMN_COUNT: usize = 1;
fn from_raw(row: &[f64]) -> Self {
Self::new(row[0])
}
}
// example for compound signals
impl Signal for (S1, S2, S3) {
const COLUMN_COUNT: usize = 3;
fn from_raw(row: &[f64]) -> (S1, S2, S3) {
// slice::chunks() should ensure `row` has length of 3,
// so `try_into()` should succeed.
let [x, y, z]: [f64; 3] = row.try_into().unwrap();
(S1::new(x), S2::new(y), S3::new(z))
}
}
you can use generic implementation for different sized tuples, you can also use blanket implementation for simple signals, e.g.:
impl<S> Signal for S where f64: Into<S> {
const COLUMN_COUNT: usize = 1;
fn from_raw(row: &[f64]) -> S {
row[0].into()
}
}
Hi, thanks for your answer. Sorry if I was not clear enough, but I think I want the generic implementation, but on library side. So this is how it could look like from the user side:
// main.rs
let signals = vec![S3::ID, S1::ID, S2::ID]; // signals defined in library
let connection = connect(&signals);
let data = connection.get_data(n);
for (t, i, u) in data.iter() {
// t: DateTime, i: i32, u: u32
...
}
The signals and their corresponding converter functions should be in the library. The user/application will simply choose which (number of) signals to use and in what order, without needing to worry about the specific types or indexing of those signals.
If they want typed output -- converted values -- the library consumer has to care about types at some point.
A key question is: where do you expect the list of signals to come from? If it's not available at compile time, you would need some sort of type erasure or enums that can hold every possible signal, and you'd need runtime conditionals (downcasts, enum matching) to try to convert those into concrete signal types. It will generally look much less clean.
The reason is that Rust is statically typed -- that is, the type of every compiled expression is known at compile time. If the list of signals can be arbitrary in your snippet, then the output of data.iter() must have the same type no matter what.
Everything that follows assumes the list of signals is available at compile time.
Rust is statically typed, so the only way t, i, u can be different types is if there is some sort of generic type inputs to data::itersomewhere, preferably somewhere that plays nice with inference. The types can't be inferred from a Vec<u32>.
In the playground I got rid of L1 and put the annotation on connect (L2). This has the benefit of not allowing a mismatch between the fetch size on L3 and the iteration on L4.[1]Here's another way to invoke it that moves the annotation closer to the consumption site.
You could keep the integers and move the generic inputs all the way down to the iter method, but then there would be a possibility of a mismatch; if you wanted to check for that it would be a runtime check; and the annotating may be more awkward too. The library consumer still needs to know the types they want out of the iteration (because, again, Rust is statically typed).
If the connection need not care, the generics could be moved to get_data instead. (But putting it on the connection is what your snippet does.) ↩︎
The type-erasing code will use dyn UsefulStuff. If COLUMN_COUNT was a method instead, Signal could be made dyn-compatible instead.
Next, I started on the dynamic case which was my main focus: the iterator item is Vec<Box<dyn UsefulStuff>>, as both the actual Signal types and the length of the Vec are assumed to be unknown. My first stab kept your "u32 identifier" pattern, but I quickly decided it wasn't worth it, as you give up type safety and end up with logic error landmine fields like this all over:
So the next addition was a type that can represent any valid Signal implementor.
#[derive(Copy, Clone, Debug)]
pub enum Identifier {
S1,
S2,
S3,
}
// You have to keep your variant => concrete type logic intact here,
// but it's all consolidated in one place and there are no unhandled
// catch-alls to keep in sync, so it's a big improvement over integers.
//
// (You could have a `TryFrom<u32> for Identifier` if you really wanted.)
impl Identifier {
fn column_count(self) -> usize { /* ... */ }
fn box_dyn(self, data: &[f64]) -> Box<dyn UsefulStuff> { /* ... */ }
}
And then modified Connection and Data to use that:
This was going fine, but I'm afraid there's no playground, as when writing that part out I realized that this could be generalized to also support the strongly typed functionality from my previous reply.
And here's how. We can support both tuples with statically known types and Vec<Box<dyn UsefulStuff>> by being generic over how the identifiers are stored, and by providing a mapping from the identifiers to the iteration item.
Sticking with the dynamic use case for now, that looks like...
pub trait Storable {
type Item;
fn width(&self) -> usize;
fn item(&self, data: &[f64]) -> Self::Item;
}
impl<S: Storable> Storable for Vec<S> {
type Item = Vec<S::Item>;
// ...
}
impl Storable for Identifier {
type Item = Box<dyn UsefulStuff>;
// ...
}
And then you wire it up to the types that store identifiers:
pub struct Connection<Storage = Vec<Identifier>> {
ids: Storage,
width: usize,
}
// `Data<Storage>` and `UsefulIter<'_, Storage>` are analogously generified
And using it looks somewhat like so.
use library::{Connection, Identifier as Id};
let connection = Connection::connect(vec![Id::S1, Id::S2, Id::S3]);
let data = connection.get_data(10);
for signal in data.iter() {
// `Vec<Box<dyn UsefulStuff>>`
println!("{}", type_name_of_val(&signal));
}
(See the final playground for the code.)
This is how things would look if you only handled the dynamic case, too.
Finally to bring back the statically known Signal types. Above we had Identifier which represented any Signal type; this time we need a representative type for eachSignal type. That's the role of our next addition:
// Todo: arrays, other tuple sizes, ...
impl<S0, S1, S2> Storable for (S0, S1, S2)
where
S0: Storable,
S1: Storable,
S2: Storable,
{
type Item = (S0::Item, S1::Item, S2::Item);
// ...
}
And using it looks like so.
use library::{S1, S2, S3, Connection, representative as rep};
let connection = Connection::connect((rep::<S1>(), rep::<S2>(), rep::<S3>()));
let data = connection.get_data(10);
for signal in data.iter() {
// `(S1, S2, S3)`
println!("{}", type_name_of_val(&signal));
}
Here's the playground.
There's a good argument here that I should have gone with an enum instead of dyn UsefulStuff... ↩︎
.