I have a set of programs that perform profit & loss, and other types of calculations of a broker's F&O trades for a settlement period. These process between 100 TB and 150 TB of data for each type of analysis.
The production versions are written in Go. I have ported the P&L calculator to Rust for learning purposes.
The pre-processing steps of the pipeline sort the input data chronologically (down to jiffies), and group it by the broker's clients. The current program then employs the standard structure of:
- one queue for buys,
- one queue for sells,
- a running totals structure,
- control break logic when the date changes (intraday vs. carryover), and
- control break logic when the broker's client changes.
As each trade is read, it is parsed into an instance of
Trade. Depending upon whether it is a buy or a sell, we try to offset it against available sells or buys, respectively, performing several calculations and accumulating them. In case of a left over quantity, the trade is stored in the appropriate queue. In that case, a
last_trade points to the corresponding entry in the appropriate queue; else, it is owned by
So, a given trade may or may not get into a queue. But, in case it does, it outlives
last_trade, which always points to the most recent input entry.
Since I am new to Rust, I could not figure out how to model this in a way that satisfies the borrow checker. After a few trials, I ended up cloning each trade as needed. I quickly achieved accuracy parity with the Go version; but, the Rust version took 11x the running time of the Go one.
After some thinking, I removed cloning, and changed the queues and
last_trade to have types
Rc<RefCell<Trade>>, respectively. I changed the rest of the program accordingly. This brought the running time of the Rust version down to 3x that of the Go one.
I understand that at least one of the reasons for the lower performance of the Rust version is the millions/billions of field accesses that require either
borrow_mut() at runtime (rather than compile-time).
What is a good way to model multiple ownership that doesn't so adversely impact performance?