New week (and soon new year), new Rust! What are you folks up to?
1 Like
My wife and I will observe New Year's Day by participating in a First Day Hike conducted by the New York State Department of Parks, Recreation and Historic Preservation, with my festivities continuing afterwards in the form of some celebratory reading about Rust. 
Our walk will take place in Ridge, on Long Island, and there are dozens of others throughout the state. Hopefully, hikers in other regions of the state will also continue their celebrations afterwards by reading about Rust. 
For the curious and the adventurous: First Day Hikes
1 Like
Before getting back to work next week I thought I'd have some fun with AI and Rust. Yeah I know, there is a downer on AI here but bear with me. This is for fun and I won't show any code.
I thought it would be a good wheeze to get the free AI built into my Warp terminal (Written in Rust you know, and very good) to write a neural network in Rust. I mean, what is the point of an AI that cannot reproduce an AI?
So I got it to read this article on back propagation A Step by Step Backpropagation Example | Matt Mazur, which includes numeric examples of inputs and outputs that can be used for testing. Then prompted the AI to write the neural net and tests in Rust. The idea was not to write any code myself. None.
Well it did. And the tests passed. I had to make some very minor tweaks using my Zed editor (Written in Rust you know, and also very good).
Now I'm trying to figure out what it did, because the algorithm it implemented is not exactly like the one given in the article. It has one bias per neuron instead of one bias per layer.
Well, you did ask...
2 Likes
Him, so I took a peek at the Python version of the NN that is given in the article I linked to above. The Rust code generate by the Warp AI is totally different.
The AI Rust version is 150 lines of easily readable Rust that follows from the text of the article. The Python version is 230 lines of.... well, I don't understand Python.
Made a little app to make it easier to fold stack traces:
app: https://azriel.im/stacktrace/ (click on the Java sample on the top right)
repo: https://github.com/azriel91/stacktrace
doesn't work so well on Rust backtraces yet though -- since rust includes line numbers. Need to parse tokens instead of just doing character matching
If you strip out the comments it is a lot less though - just to be fair. But rust can be quite concise.
I guess you didn't need to use any libraries for this example - I found in the past that chatgpt easily gets confused about the many versions ndarray has existed in - frustrating waste of time - chatgpt, not ndarray - though I tend the prefer nalgebra.
There are a ton of blog posts and books on neural networks. I read Haykin's book recently - which is good - and implemented a couple of different nn architectures on a toy problem he uses in the book (half moons).
I must confess was a bit lax in ignoring the comments in the Python code. But, hey, the AI put useful comments in the code in generated as well. Very nice.
The AI did not use any dependencies except assert_approx_eq for use in the tests it generated. Thing is the text of the article I gave it hardly uses any arrays pretty much laying all the calculations line by line, so the AI dutifully implemented that. With the downside of course that the code is not easily extensible to more than a two layer network with two neutrons per layer. Which is exactly what I wanted, I did not want any details, data layouts etc hidden.
Ah heck, here is the code the AI produced. I only tweaked a few characters in the whole file. Do forgive me everybody, for this bit of festive fun:
/*
* Neural Network Implementation with Backpropagation
*
* This module implements a simple neural network with two layers and backpropagation
* learning algorithm. The implementation includes forward propagation, backward propagation,
* and training functionality. The network architecture consists of:
* - Input layer: 2 neurons
* - Hidden layer: 2 neurons
* - Output layer: 2 neurons
*
* Author: AI Assistant
* Date: December 30, 2024
*
* Based on "A Step by Step Backpropagation Example" by Matt Mazur
* https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example
*/
use std::f64;
pub struct NeuralNetwork {
w1: [[f64; 2]; 2], // weights for layer 1
w2: [[f64; 2]; 2], // weights for layer 2
b1: [f64; 2], // biases for layer 1
b2: [f64; 2], // biases for layer 2
learning_rate: f64, // learning rate for gradient descent
}
impl NeuralNetwork {
pub fn new() -> Self {
NeuralNetwork {
// Initialize weights as given in the article
w1: [[0.15, 0.20], [0.25, 0.30]],
w2: [[0.40, 0.45], [0.50, 0.55]],
b1: [0.35, 0.35],
b2: [0.60, 0.60],
learning_rate: 0.5,
}
}
pub fn calculate_total_error(&self, input: &[f64; 2], target: &[f64; 2]) -> (f64, f64, f64) {
let (_, outputs) = self.forward(input);
let e1 = 0.5 * (target[0] - outputs[0]).powi(2);
let e2 = 0.5 * (target[1] - outputs[1]).powi(2);
let etotal = e1 + e2;
(e1, e2, etotal)
}
fn sigmoid(x: f64) -> f64 {
1.0 / (1.0 + f64::exp(-x))
}
fn sigmoid_derivative(x: f64) -> f64 {
let sx = Self::sigmoid(x);
sx * (1.0 - sx)
}
pub fn forward(&self, input: &[f64; 2]) -> (Vec<f64>, Vec<f64>) {
// First layer calculations
let h1_in = input[0] * self.w1[0][0] + input[1] * self.w1[0][1] + self.b1[0];
let h2_in = input[0] * self.w1[1][0] + input[1] * self.w1[1][1] + self.b1[1];
let h1_out = Self::sigmoid(h1_in);
let h2_out = Self::sigmoid(h2_in);
// Second layer calculations
let o1_in = h1_out * self.w2[0][0] + h2_out * self.w2[0][1] + self.b2[0];
let o2_in = h1_out * self.w2[1][0] + h2_out * self.w2[1][1] + self.b2[1];
let o1_out = Self::sigmoid(o1_in);
let o2_out = Self::sigmoid(o2_in);
(vec![h1_out, h2_out], vec![o1_out, o2_out])
}
pub fn train(&mut self, input: &[f64; 2], target: &[f64; 2]) -> f64 {
// Forward pass
let (hidden_outputs, outputs) = self.forward(input);
// Calculate hidden layer inputs (needed for backprop)
let h1_in = input[0] * self.w1[0][0] + input[1] * self.w1[0][1] + self.b1[0];
let h2_in = input[0] * self.w1[1][0] + input[1] * self.w1[1][1] + self.b1[1];
let o1_in =
hidden_outputs[0] * self.w2[0][0] + hidden_outputs[1] * self.w2[0][1] + self.b2[0];
let o2_in =
hidden_outputs[0] * self.w2[1][0] + hidden_outputs[1] * self.w2[1][1] + self.b2[1];
// Calculate output layer errors
let o1_error = target[0] - outputs[0];
let o2_error = target[1] - outputs[1];
let o1_delta = o1_error * Self::sigmoid_derivative(o1_in);
let o2_delta = o2_error * Self::sigmoid_derivative(o2_in);
// Calculate hidden layer errors
let h1_error = o1_delta * self.w2[0][0] + o2_delta * self.w2[1][0];
let h2_error = o1_delta * self.w2[0][1] + o2_delta * self.w2[1][1];
let h1_delta = h1_error * Self::sigmoid_derivative(h1_in);
let h2_delta = h2_error * Self::sigmoid_derivative(h2_in);
// Update weights and biases
// Output layer
self.w2[0][0] += self.learning_rate * o1_delta * hidden_outputs[0];
self.w2[0][1] += self.learning_rate * o1_delta * hidden_outputs[1];
self.w2[1][0] += self.learning_rate * o2_delta * hidden_outputs[0];
self.w2[1][1] += self.learning_rate * o2_delta * hidden_outputs[1];
self.b2[0] += self.learning_rate * o1_delta;
self.b2[1] += self.learning_rate * o2_delta;
// Hidden layer
self.w1[0][0] += self.learning_rate * h1_delta * input[0];
self.w1[0][1] += self.learning_rate * h1_delta * input[1];
self.w1[1][0] += self.learning_rate * h2_delta * input[0];
self.w1[1][1] += self.learning_rate * h2_delta * input[1];
self.b1[0] += self.learning_rate * h1_delta;
self.b1[1] += self.learning_rate * h2_delta;
// Return total error
0.5 * (o1_error * o1_error + o2_error * o2_error)
}
}
#[cfg(test)]
mod tests {
use super::*;
use assert_approx_eq::assert_approx_eq;
// All expected results and their accuracy taken from the article sited above.
#[test]
fn test_forward_pass() {
let nn = NeuralNetwork::new();
let input = [0.05, 0.10];
let (hidden, output) = nn.forward(&input);
// Test intermediate values from the article
assert_approx_eq!(hidden[0], 0.593269992, 1e-9); // h1_out
assert_approx_eq!(hidden[1], 0.59688437, 1e-8); // h2_out
assert_approx_eq!(output[0], 0.75136507, 1e-8); // o1_out
assert_approx_eq!(output[1], 0.77292846, 1e-8); // o2_out
}
#[test]
fn test_training() {
let mut nn = NeuralNetwork::new();
let input = [0.05, 0.10];
let target = [0.01, 0.99];
// Train for a few iterations
let initial_error = nn.train(&input, &target);
let final_error = (0..1000).fold(initial_error, |_, _| nn.train(&input, &target));
assert!(final_error < initial_error);
// Check if outputs are closer to targets after training
let (_, outputs) = nn.forward(&input);
assert!((outputs[0] - target[0]).abs() < 0.1);
assert!((outputs[1] - target[1]).abs() < 0.1);
}
#[test]
fn test_error_calculation() {
let nn = NeuralNetwork::new();
let input = [0.05, 0.10];
let target = [0.01, 0.99];
let (e1, e2, etotal) = nn.calculate_total_error(&input, &target);
// Verify the specific error values from the article
assert_approx_eq!(e1, 0.274811083, 1e-9);
assert_approx_eq!(e2, 0.023560026, 1e-9);
assert_approx_eq!(etotal, 0.298371109, 1e-9);
}
}
I'm mplementing an algorithm to find links between my markdown files and store them in some sort of basic KV pairs, so that I can use them in my blog. Although the format does not matter, JSON is an option, because I can then use it in the JS frontend.
This is pretty easy and entirely for learning purposes. All I need actually is some Regex parsing and File I/O, which I could do in Javascript fairly quick and easy, but want to stay consistent with learning Rust this year so...there it is 
1 Like
Welcome to the Rust Programming Language Users Forum, @kenan!
We hope you are enjoying learning Rust and the work on your blog.
You can find some useful introductory material about this forum in the discussions Welcome to the Rust programming language users forum and Forum Code Formatting and Syntax Highlighting.
There have also been some discussions here about resources for learning Rust, for example, What is the best way to learn rust language? and Seeking Recommendations for an Introductory Book on Rust. Members of this community have been very helpful in that regard.
If you would like to tell us anything more about the project with links between your markdown files, the most recent installment of this weekly discussion is What’s everyone working on this week (2/2025)?.