Daedalus - data rendering and manipulation to HTML

I have released a tool I have been working on for the last 3 years: daedalus.

The tool is somewhat hard to summarise, which is never good for a product! It takes files and renders them into HTML, providing some context and formatting along the way.

daedalus was born out of boredom with Excel and Powerpoint, the goto tools for the industry I work in.
I wanted something similar to Jupyter Notebooks, but will a consolidated reporting feel to it that did not take much effort. Since I commonly work with periodic reports, having a system that took minimal effort to update was favourable. My initial efforts produced papyrus, which coincidentally begun around the same time as evcxr.

I have always liked the feedback of a REPL and thought that would be the way to build reporting tools. It can handle the data processing and manipulations and then some markup could handle the visualising. Certainly this is how Notebooks can handle it. It struck me however that this is still a very programming orientated way of thinking, and while I adore programming in Rust, it must be recognised that the learning curve would cause too much friction for users just wanting to use something in anger.

This prompted a pivot (typical) and daedalus tries to focus on the presentation of data. At the same time I was not going to let the REPL go, which led to my own expression language: ogma.

All of this has been built in Rust (with some Typescript/Javascript to glue the HTML together). Some highlights have been:

Creating a DOM generation framework

I used this to generate the HTML of a file. It was fun to leverage Rust's type system to control what elements can be added to what, along with the builder system to create immutable DOM trees. A sample of the DOM framework looks like this:

let dropdown = ::dom::dropdown(
	Dom::div() + Dom::em("images ") + Dom::text("🔽"),
	self.imgs.iter().enumerate().map(|(i, img)| {
		let mut name = format!("`#{}:`", i + 1);
		if let Some(md) = img.title.as_ref().or_else(|| img.caption.as_ref()) {
			name.push(' ');
			name += md.as_str();

let selector = Dom::div().with_metadata(script) + dropdown;

Creating a language

When I moved away from using papyrus as the REPL, I looked for suitable replacements. Python, OCaml were options, and some more Rust orientated projects such as gluon and nushell were very interesting. I decided (mostly for pedagogical reasons) to build my own (yet another language!). ogma was born. It was heavily influenced by nushell's data pipeline and how working with tables is a first-class functionality. I also needed performance, so rolling my own allowed me to control for this. Developing ogma has been a fantastic learning experience, and has given me a peephole into the complexities any lang and compiler team would face.

Deploying servers

At the end of the day, the binary tool was only part of the product. SaaS was the end goal, and this dropped me in the deep-end of async. async/await has come a long way in Rust, and using the great libraries of tokio, hyper, and warp allows one to very quickly produce working servers that are extremely robust. warp in particular has been extremely powerful for testing a web service!

While this post has been a shameless announcement for my product, I do believe that daedalus has been enabled by Rust. Whether it is the code I have written or the libraries I depend on, Rust's tagline, A language empowering everyone to build reliable and efficient software, rings true.

1 Like