Help to get correct data structure for a project

Hi,

I need help is creating data structures for my application. I am working to implement communication protocol which is a series of tag=value delimited by some character as below -

8=FIX.4.4|9=126|35=A|49=theBroker.12345|56=CSERVER|34=1|52=20170117- 08:03:04|57=TRADE|50=any_string|98=0|108=30|141=Y|553=12345|554=passw0rd!|10=131|

All the tags are u32, value can be anything - strings, float or bool etc. Each tag & value means something like 35=A means logon message.

I started with something like this -

struct Price (f64); // one field type
struct Amount (f64);
struct Length (u32);

pub trait FixType {
	type Output;
    	fn get_val(&self) -> Self::Output;
    	fn set_val<T>(&mut self, val: T);
}

and then implemented the trait for types like Amount, Price etc.

I also have -

// I want this to be generic field which has tag of u32
// and value can be anything

pub struct Field<T: FixType> { 
            tag: u32,
            value: T,
        }

pub struct Message<T: FixType> {
    	fields: HashMap<u32, Field<T>>
    }

What I was (un-successfully) trying to achieve was to a Message struct which contains tag and value as Field (field again has the same tag and value) -> This does not work clearly because HashMap needs one homogeneous concrete type.

I changed the field struct to -

pub struct Field {
    tag: u32,
    value: Box<FixType>,
}

And message struct to -

pub struct Message {
	fields: HashMap<u32, Field>
}

But this also does not work because struct Field needs the associated type ‘Output’ if I am using Box. I am not sure how do I model this into correct data structures. I tend to think like Java because that is the base implementation I want to write in rust. Could anyone please help correct my data structures and my thinking in the process?

Below is the error I get -
value: Box,
| ^^^^^^^ missing associated type Output value

Usually when I’m designing a communication protocol you’ll try to separate the actual serialized format from the data being sent.

As you mentioned, each piece of data comes as a key-value pair, where the value can be essentially anything. This quite naturally lends itself to a Pair struct where the value is an enum.

struct Pair {
  key: u32,
  value: Value,
}

enum Value {
  Float(f64),
  Integer(u32),
  String(String),
}

It’s then easy enough to parse the data into these key-value pairs, taking advantage of the std::str::FromStr trait via the str::parse() method.

fn parse(src: &str) -> Result<Vec<Pair>, Error> {
  let mut pairs = Vec::new();

  for word in pairs.split("|") {
    let pair = Pair::from_str(word)?;
    pairs.push(pair);
  }

  Ok(pairs)
}

impl FromStr for Pair {
  type Err = ...;

  fn from_str(s: &str) -> Result<Self, Self::Err> {
    let equals = match s.find('=') {
      Some(ix) => ix,
      None => /* return an error saying we couldn't find the "=" */
    };

    let key: u32 = s[..equals].parse()?;    
    let raw_value = &s[equals+1..];
    let value = raw_value.parse()?;

    Ok(Pair { key, value })
  }
}

impl FromStr for Value {
  type Err = ...;

  fn from_str(s: &str) -> Result<Self, Self::Err> {
    if let Ok(i) = s.parse() {
      Ok(Value::Integer(i))
    } else if let Ok(f) = s.parse() {
      Ok(Value::Float(f))
    } else {
      Ok(Value::String(s.to_string()))
    }
  }
}

Then once I’ve parsed the input into Pairs, you’d normally have an enum which contains each message type and a function that uses a match statement to transform a Pair to the correct message variant (or emit an error if the datatypes aren’t correct or the key is unknown). Possibly adding a catch-all variant which can be used to hold messages you don’t currently understand, when you don’t want to throw away the information.

pub enum Message {
  Login(String),
  PriceChange(f64),
}

fn parse_message(pair: Pair) -> Result<Message, Error> {
  match pair.key {
    35 => Message::Login(pair.value),
    _ => Err(...),
  }
}

// or you can use more strongly typed variants if they'll be passed around

pub enum Message {
  Login(Login),
  ...
}

fn parse_message(pair: Pair) -> Result<Message, Error> {
  match pair.key {
    35 => Ok(Message::Login(Login::from_pair(pair)?)),
    _ => Err(...),
  }
}

struct Login {
  user: String,
}

impl Login {
  fn from_pair(pair: Pair) -> Result<Login, Error> {
    if pair.key != 35 {
      Err(InvalidKey)
    }  else if let Value::String(user) = pair.value {
      Ok(Login { user }) 
    }
  }
}

Separating between the underlying communication protocol (our Pair and Value) and the high-level data being conveyed (Message) tends to make things easier. Using an enum lets you think in terms of the data being conveyed instead of its behaviour, especially when your FixType trait was just a thing with setters and getters. It also gives you a lot more control and the ability to inspect everything. YMMV.

Sorry for the wall of code! It’s kinda hard to explain without examples.

1 Like

Hi Michael,

Thanks a lot for suggestion and code. I will try to use enum as a field and see how it goes. The message is basically collection of the fields (pair in your example). And there are different types of message depending on what different fields (pairs) are being used. I am not able to wrap my head around message being an enum. But this gives me enough to chew on and try things. Thanks a lot ! I will update this thread about my findings.

Regards,

Gaurav

1 Like