Rten what is the best model to perform ocr on handwritting?

1

Using this library: GitHub - robertknight/rten: ONNX neural network inference engine · GitHub what is the best model to use to perform ocr on handwriting?

    let detection_model_data = read_file(&format!("{}/text-detection.rten", MODELS_DIR));
    let rec_model_data = read_file(&format!("{}/text-recognition.rten", MODELS_DIR));

    let detection_model = rten::Model::load(&detection_model_data).unwrap();
    let recognition_model = rten::Model::load(&rec_model_data).unwrap();

    let engine = OcrEngine::new(OcrEngineParams
    {
        detection_model: Some(detection_model),
        recognition_model: Some(recognition_model),
        ..Default::default()
    }).unwrap();

I am currently using this however it half works with handwritting was wondering for something better?

2

There is a TrOCR example in the RTen repository which can be used with the trocr-base-handwritten model. Another way to find compatible models is to search Hugging Face for ONNX models that mention "ocr".

PP-OCR is probably the best known "small" model for OCR. The README on Hugging Face shows it being used on an image of handwritten text.

3

Here is a quick demo for PP-OCR created with Claude: GitHub - robertknight/rten-pp-ocr: A demo showing how to use PP-OCR models in RTen · GitHub

4

Just tested it out didn't work tell well with these kinds of images:

image

5

That’s a lot to ask tbh. As a human I initially read that as 18192026. I’m sure some models could get the slashes from context, but a “small” model can probably be expected to fail :confused:

6

Rten works great! Even when the model achieves a poor fit to my data, library performance is very reliable.

As a strategy, OCR excels when the handwriting is legible. In more realistic situations, the handwriting is illegible and poorly conformant to its bounding boxes, like this example from my dataset:

image
image697×156 75.7 KB

I would argue that OCR is not the best tool to reach for here, a gem of insight I was foolish enough to discover personally.

7

I wonder what happened in the second row of your data set? :thinking:

8

I found PP-OCR to be better. Rten from the dates dataset was quite bad, this other library is better but still not workable.

I will train my own model which is very specific to dates and see how that goes. Anyways thank you!

9

Sorry I could not resist it, I gave DeepSeek-V41-Flash that date sample to look at and it actually came back with 18 September 2026 (18/9/2026).

Mind you, it took it 41 seconds and burned 150,000 tokens. So about $0.0006.

10

That is crazy