I am looking for something similar to YOLO but for Arduino. I know that there is PixyCam that can attach to the arduino however you are "jailed" to using their camera and I don't want to be locked into their camera, I want to be able to use any camera and I'd rather have AI processed on the Arduino then in a separate hardware and I want to do on an Arduino that is not an ARM processor running Linux, just one of those that runs without an operating system like Arduino Uno.
No way are you going to run anything like YOLO on the 8 bit processor of the Arduino Uno. Far too slow, far too little memory, no IO bandwidth.
The smallest thing I have run YOLO on is a Raspberry Pi 3.
For small, micro-controlled like machines that can do some of that you might want to check out:
Seeed Studio Vision AI Module V2
Analog Devices MAX78000
STMicroelectronics STM32N6
Texas Instruments MSPM0G5187
Sipeed Maix-BiT (Kendryte K210)
I know very little about these but I was tempted to get hold of the Sipeed. It has a nice RISC-V processor and neural net hardware. But I suspect getting object detection working is not so simple.
I didn't say that, in terms of similarity I want something that is object detection library that can run on Arduino uno.
As mentioned before I prefer if the processing was done on the Arduino, not on a separate hardware.
It seems like the pixycam you mentioned is doing a lot of the image processing and target detection on it's board. Likely the reason that specific "camera" can work with an Arduino at all is because it processes down to the "X-Y coordinates, size, and unique ID" and sends only that small amount of info over the wire to the Arduino.
Looking around Arduino's board specs, even the Atmega 2560 only has 256KB of flash memory for the program. Without specialized image processing components, I'm not sure what kind of image analysis you could do on general purpose hardware like that.
Wait a minute. You said "I am looking for something similar to YOLO but for Arduino". What am I supposed to think?
I reckon if you attached a bunch of photodiodes up to Uno, with that and a few lines of code you could detect objects with it. To some crude extent.
Or forget light, use ultrasonic transmitters/recievers to "ping" objects.
According to https://docs.arduino.cc/hardware/mega-2560/ there is only 4KB of EEPROM. The datasheet lists another 16KB of flash for the uart chip, but you would only be able to use part of it even if you manage to reprogram this chip. And the main cpu has just 8KB of SRAM. If you manage to make the camera produce a black and white image (no grayscale!) then you can only fit a 256x256 pixel image in there. You will almost certainly need to have it be at least 8bit color to have even a reasonably chance of detecting objects somewhat accurately. But then you can only have a 90x90 pixel image, which probably too small. And that is not even counting all memory you need for things like the stack or whatever processing the object detection algorithm itself does or the code that is actually going to use the output of the object detection algorithm.
Is this close to what you are after?
Looks interesting but is this an arduino though still I am a little confused?
Can I do rust programming with tinyml?
Just to clarify, tinyML is just a term used to describe the field of "machine learning inference" on "low-power embedded devices". That being said, you certainly can do tinyML on Rust. Now, more specifically to your question about object detection, there are two things you need:
- an object detection model that fits in your MCU's memory
- an inference engine capable of running on your MCU
So, for number 1. your best bet is to search for a pre-trained model for "tensorflow lite micro" (TFLM). You are lucky if you want to detect persons or gestures, as most tutorials tinyML/TFLM focus on that scenario.
Now, for the inference engine, there are a few ways I can think of:
-
microflow: Perhaps the most efficient way to do what you want, as it seems to be a direct replacement of TFLM C++ library, and directly uses the
.tflitemodels. Note that it is "research-grade" software, meaning the idea behind it is sound (I have toyed around that idea myself), but you may get limited support if you encounter problems. I have not used this myself. -
burn: a more establish framework (but still in heavy development) in Rust. To run a model on an MCU, you do inference together with ndarray. Burn can import ONXX models, so you might need to transform your model in 1. from
.tfliteto.onxxformat. I have used this on an MCU (but not for object detection), and is quite cool, but the learning curve is quite steep.
There are obviously other inference engines in Rust, but most of them do not support running on MCUs. Hope this helps. Have fun!
How accurate would you say is tensoreflow lite for detecting objects?
They are typically less accurate than the big models, which is expected. Nevertheless, according to this table from the model zoo, you can expect around 80% accuracy (not sure how that is measured/determined). You can run the most accurate (85.9% accuracy) model (mcunet-320kb) potentially on an ESP32-C3 (4 MB Flash, 400 kB RAM).
Regardless of those numbers, important is to see if they are accurate enough for your particular application. You can only determine that by testing (which might require training a model with your own dataset).
So just so I am clear, I would train my model using tensorflow lite micro, then export it to onxx and then I would run the model using burn.
Would I be able to run burn with arduino embeded-hal crate with the model? Is this how it works?
The tensorflows names are confusing, here is a short overview:
tensorflow: the main deep ML framework from google, will all the bells and whistles. Here is where the main training happens.
tensorflow lite (now called LiteRT): among other things, offers a tool to optimize (reduce size model, inference speed, etc.) simple tensorflow models (i.e., less operators supported) that result in .tflite models. It also has an inference engine for embedded devices that run .tflite models.
tensorflow lite micro: an inference engine for .tflite models, similar to LiteRT but with less bloat (i.e., for MCUs).
So just so I am clear, I would train my model using tensorflow lite micro, then export it to onxx and then I would run the model using burn.
In very general terms, you train your models using tensorflow, pytorch, or burn (or any other). You then optimize those models into a .tflite model using LiteRT. An inference engine runs your model on the embedded target (e.g., with burn and ndarray as backend, you need to transform the model from .tflite to .onxx format). This is of course not the only way; for instance, wakevision.ai can also perform the training (I have never used it, though).
Another way, perhaps the easiest, is to find a pre-trained model that does what you want and run it on the MCU with an appropriate inference engine. That would be the closest equivalent to a YOLO-like library for Arduino using Rust mentioned in your original post.
Would I be able to run burn with arduino embeded-hal crate with the model? Is this how it works?
yes and yes.
Please note that what you are asking is "most probably" doable (I cannot be 100% sure, I've never done it myself), however it is not trivial. It requires expertise in different areas (not sure which ones you have) and a lot of experimentation. Best of luck!
Just for reference may I know which MCU in particular did you use, is it the one that you linked me to? Would it also be suitable on Arduino Uno R4?
Lately I have been using the ESP Rust Devboard. It runs on a ESP32-C3. But in the past, I mostly used STM32 (Cortex-M from ST). It seems like the Uno R4 might be able to run the smallest models only.