Hello Rustlers 
,
I am currently building up a dynamic modifyable parser tree based on a memory arena. This means we have parser nodes within a vector (all have the same lifetime) whereas these nodes are mainly Boxed trait objects. This enables building up a tree based on indexes, which reduces the complexity. The parsing itself is done via the nom framework.
The input of the main library method is an &[u8] whereas the output is a vector of parsed "results".
My main problem is now: If these results refer to contents of the input, it works only if the input lives longer than the parser tree.
This means this works:
fn ethernet_success() {
// Create some input
let input = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 8, 0];
// Create the parser tree
let mut peel = Peel::new();
peel.new_parser(EthernetParser);
// Parse the result
let result = peel.traverse(&input, vec![]).unwrap();
assert_eq!(result.len(), 1);
}
But this wont:
fn ethernet_failure() {
// Create the parser tree
let mut peel = Peel::new();
peel.new_parser(EthernetParser);
// Create some input
let input = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 8, 0];
// Parse the result
let result = peel.traverse(&input, vec![]).unwrap();
assert_eq!(result.len(), 1);
}
error: `input` does not live long enough
--> tests/lib.rs:31:1
|
29 | let result = peel.traverse(&input, vec![]).unwrap();
| ----- borrow occurs here
30 | assert_eq!(result.len(), 1);
31 | }
| ^ `input` dropped here while still borrowed
|
= note: values in a scope are dropped in the opposite order they are created
error: aborting due to previous error
error: Could not compile `peel`.
To learn more, run the command again with --verbose.
Currently (on my master branch) all parsers copy the parsed data, which works in any cases. To make references possible I added a lifetime to the Parser trait. The parser trait looks like:
//! General parser descriptions and traits
use nom::IResult;
use arenatree::{Node, Arena};
/// The type which will be stored within the tree structure
pub type ParserBox<'a, R, V> = Box<Parser<'a, Result = R, Variant = V> + Send + Sync>;
/// Arena tree for parsers
pub type ParserArena<'a, R, V> = Arena<ParserBox<'a, R, V>>;
/// A node within a `ParserArena`
pub type ParserNode<'a, R, V> = Node<ParserBox<'a, R, V>>;
/// The parsing trait
pub trait Parser<'a> {
/// The type for result reporting, usually an enum
type Result;
/// The type of the parser itself, usually an enum
type Variant;
/// Parse using nom and return the result
fn parse(&self,
input: &'a [u8],
node: Option<&ParserNode<Self::Result, Self::Variant>>,
arena: Option<&ParserArena<Self::Result, Self::Variant>>,
result: Option<&Vec<Self::Result>>)
-> IResult<&'a [u8], Self::Result>;
/// Return the actual enum variant of the parser
fn variant(&self) -> Self::Variant;
}
The Result should refer to the input, for example with a simple EthernetParser:
impl<'a> Parser<'a> for EthernetParser {
type Result = Layer<'a>;
type Variant = ParserVariant;
fn parse(&self,
input: &'a [u8],
_: Option<&ParserNode<Layer, ParserVariant>>,
_: Option<&ParserArena<Layer, ParserVariant>>,
_: Option<&Vec<Layer>>)
-> IResult<&'a [u8], Layer<'a>> {
do_parse!(input,
d: take!(6) >>
s: take!(6) >>
e: map_opt!(be_u16, EtherType::from_u16) >>
(Layer::Ethernet(EthernetPacket {
dst: MacAddress(&d[0], &d[1], &d[2], &d[3], &d[4], &d[5]),
src: MacAddress(&s[0], &s[1], &s[2], &s[3], &s[4], &s[5]),
ethertype: e,
}))
)
}
fn variant(&self) -> ParserVariant {
ParserVariant::Ethernet(self.clone())
}
}
I will stop adding more code here since I think it is really hard to understand without the full context. The full source is available within a separate branch where I removed irrelevant code. The tests/lib.rs shows my current problem.
Do you have any idea how to solve this? Thank you very much for your help! Best greetings from germany!
Sorry for not being more insightful, I haven't had much of a chance to play with your particular example. Sometimes you just have to fiddle with lifetimes to get the exact semantics right.
I will try out the raw pointer solution in the next days, this will just mean that the parser returns a raw pointer which whill be casted to the actual 