Integrating an allocator inside a parser that uses recursive data structures

1

Hello, Im writing a parser for a programming language Im working on. The parser struct only contains the token stream, but recently ive been trying to integrate an allocator inside it to optimize some parts of the parsing progress (my Expression is a recursive enum, so references/smart pointers are required). Ive changed the type declarations to make them use Bump from the bumpalo crate for anything that needs allocations, I put the Bump inside the Parser struct so that any time a parsing functions would run I could easily pass the allocator to them, however Ive encountered an error: my parsing function signatures all take a &mut self, but since Expressions use lifetimes for boxes and stuff like that the compiler thinks that every constructed expression actually takes a reference to Parser, which is not really what's happenning. Is there a solution for my issue or should I just use the global rust allocator?

2

Generally, any time one type references another (as bumpalo's containers refer to Bump), you cannot then put both of those things in one struct (in plain Rust). Trying will almost always result in unsolvable borrow errors.

Change your Parser to reference the Bump instead.

struct Parser<'b> {
    bump: &'b Bump,
    ...
}
4 Likes
3

This is where https://lib.rs/crates/ouroboros and similar crates come into play. But it is generally less ergonomic so consider other approaches first.

2 Likes