Tenferro-rs: a scientific tensor runtime for Rust

We are developing tenferro-rs, a Rust-native tensor runtime for scientific computing:

It started as the computational core for our own tensor network calculations in physics, but we are now shaping it into a general-purpose library. It currently provides CPU and CUDA backends, eager and traced automatic differentiation, einsum, FFT, and linear algebra.

A key design choice is explicit backend and session management. Execution resources such as CPU thread pools, NUMA domains, GPU devices, streams, memory pools, and backend-specific handles are all explicit, while tensor values themselves stay independent of the execution context.

It is not meant to replace existing Rust array libraries, and we see it as complementary to ML-oriented stacks such as Candle and CubeCL rather than competing with them. For example, ndarray or mdarray storage can be borrowed through strided tensor views, so tenferro can serve purely as the computation layer. We also try to contribute upstream where possible: we are extending CubeCL with complex number support (tracel-ai/cubecl#1300), and some of our changes have already been merged (#1301).

We also rely heavily on AI coding agents in development, and we are experimenting with stricter engineering practices for AI-assisted scientific software: strict repository rules, executable documentation, backend-parity tests, automated correctness checks, and benchmark gates against performance regressions.

The project has grown quite a bit recently, and we would appreciate feedback from people doing scientific computing in Rust, especially on the API, the execution model, interoperability, and performance. Since our own use cases are biased toward tensor networks, we would particularly like to hear about use cases in other fields that the current design does not serve well.

The explicit session/backend split is the part I like here. Most Rust array crates bury that in globals and it gets painful the moment you have two GPUs in a box.

Question on the dtype side: is Tensor generic over the element type, or is there a runtime DType enum? Asking because once you go below f32 the interesting cases are the ones where storage type and compute type differ (store bf16, accumulate f32), and that split is awkward to express if dtype is only a runtime tag.

Related: does traced autodiff survive an einsum with mixed-precision inputs, or do you promote everything to a common type first?

I have been working on a small spec for exactly this problem (typed grouped low-precision tensors, GitHub - singhpratech/grit-datatype: GRIT — Grouped Reduced-precision Interchange Type: a checkable 64-byte descriptor + contract for block-scaled, sparse, sub-byte tensors. Spec, conformance vectors, and 5 zero-dependency implementations (C, C++, Rust, Python, TypeScript). grit scan audits GGUF/safetensors files you already have. · GitHub) so I am curious how you are handling it, whether or not that turns out to be useful to you.

Sorry for the slow reply, I have been traveling. Thanks, glad the session/backend split resonates. Multi GPU was exactly the case that pushed us there.

On element types, the answer is "both". The non AD tensor layer offers two types:

  • TypedTensor<T, R> is generic over the scalar T (and optionally the rank R). T is bounded by a sealed TensorScalar trait, currently implemented for f32, f64, i32, i64, bool, Complex32 and Complex64. This is the Eigen style: dtype and rank live in the type system, and mismatches are compile errors.
  • Tensor is a runtime enum over those TypedTensor<T> variants with a DType tag. This is the NumPy style: dtype is a runtime property, and it is what you want when the dtype comes from data or from user input.

The autodiff and traced execution layers are built on the runtime Tensor, so a graph is not monomorphized per dtype and backends dispatch at runtime. The enum is closed today. That is a consequence of the backend parity guarantee (every dtype must work on every backend and every primitive op), not a philosophical stance. We would like downstream crates to be able to bring their own dtype eventually, but we have not found a clean way to do that without breaking that guarantee. Ideas welcome. Some idea is here:

On precision: our first target is scientific computing, where f32 or better is the norm, so bf16/f16 have been deferred rather than ruled out. We do want to support them. Today there is no sub f32 storage and no storage type vs compute type distinction. Each primitive op carries a dtype policy, binary ops promote to a common dtype (f32 + f64 gives f64), and einsum, eager or traced, promotes its inputs to one dtype and contracts in that dtype. A "store bf16, accumulate f32" path would be new design work for us.

I would be genuinely interested in your low precision tensor spec if it is public. Do you have a link?