Xazz — An open-source data & ML DSL built from scratch in Rust

announcements
1

Hi everyone

I'm a high school student and developer from South Korea, and I've been building a programming language and platform for AI/data pipelines in Rust.

It's called Xazz.

I started the project after running into a few problems with the usual Python-based workflow (pandas + PyTorch):

  1. Type and missing-value errors often surface only at runtime.
  2. Moving data between pandas, NumPy, and PyTorch can introduce additional conversions and copies.
  3. Data pipelines usually don't have a first-class policy/security layer.

So instead of building another Python wrapper, I decided to explore what it would look like to build the language and runtime layer itself.

Xazz currently includes:

  • A compiler pipeline written in Rust: Lexer → Parser → AST → Type Checking → IR
  • A Polars LazyFrame-based data-processing runtime
  • Schema-aware and statically checked data pipelines
  • Option<T> for nullable values
  • A node-based Visual IDE
  • A Burn integration for defining and running ML models
  • A Synthetic Data Engine (SDE)
  • A Neural Query Planner (NQP) research prototype
  • Policy-as-Code and audit-related components for data pipelines

A small example looks like this:



type AirData = {
    temp:     float,
    humidity: float,
    pm10:     Option<float>,
}

v dataset = load("air_data.csv") :: AirData
    |> fillNull("pm10", strategy: "mean")
    |> select(["temp", "humidity", "pm10"])

model AirPredictor {
    Dense(64) -> ReLU() -> Dense(1)
}

v model = dataset
    |> train(AirPredictor, target: "pm10", epochs: 10)

v prediction = dataset
    |> predict(model, as: "pm10_pred")
    |> chart {
        type: line,
        x: station,
        y: pm10_pred,
        title: "PM10 Prediction",
    }

I also benchmarked the same four-stage pipeline on real Seoul air-quality data.

For datasets ranging up to 4.09 million rows, Xazz was measured at up to 2.62× faster than pandas in my benchmark.

The project is still experimental, and there are definitely rough edges. I've had plenty of crashes, cross-compilation problems, edge cases, and other bugs along the way. Building the parser, type checker, runtime, execution layer, and tooling myself has been a much bigger challenge than I initially expected.

That's also why I'm opening the source.

I'd particularly appreciate feedback on:

  • language and type-system design
  • compiler architecture
  • runtime/data representation
  • Rust implementation choices
  • safety and policy enforcement
  • areas where the current design is unnecessarily complicated

If you're interested in taking a look, the repository is here:

github.com/x1zzdev/Xazz

I'm still learning a lot from this project, so critical feedback is very welcome.