I've just open-sourced batch_forge, a specialized inference runner for the JAX/Equinox \ ecosystem. The project is written in pure Rust and leverages metal-rs for custom compute kernels. Current features include a zero-copy Safetensors loader, an async tokio request manager, and stateful attention state management. Feedback on the memory-mapped loader and Metal buffer synchronization would be greatly appreciated.
1 Like
This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.