Transformrs - An interface to AI API providers

announcements
1

Last week, I was thinking a bit about where AI is heading. I wrote a blog post about it and concluded that (1) cloud based AI gets about 90% cheaper each year and (2) things that worked in the past won't necessarily work now.

For (2), my reasoning is that a lot of things probably will be rebuilt from scratch instead of just continuing on top of existing tooling. For example, in 2022 I would have expected that Visual Studio Code would be the favorite editor for most people. But then Cursor AI came along. Same for DeepSeek.

So that got me thinking, how can I build tools from the ground up with AI? And preferably on the terminal. I would like to have tools that run in my terminal and that I can mix and match unix-style.

Looking at the costs, it will have to be cloud-based. I would love to run everything locally, but at the same time it's hard to justify buying a $2000 EPYC server to run DeepSeek R1 at 2-3 tokens per second while there are multiple cloud providers that offer these models at higher speeds for about $1 per million tokens. And again, as said in (1) the price will probably decrease by 80% again next year. I say this because learning curves can be surprisingly persistent as we all know from computer chips. But also for solar panels and batteries. So maybe this is just part of how things are now? We will probably depend more on cloud computing than we would in the 1980s.

With these things in mind, I wanted to start building command line tools so Rust was my first choice. However, to avoid locking people into only one provider I wanted to support multiple.

That's why I created the transformrs crate (GitHub - rikhuijzer/transformrs: Interface for AI APIs). This for example is how to describe an image:

use transformrs::openai;
use transformrs::Message;
use transformrs::Provider;

#[tokio::main]
async fn main() {
    let messages = vec![
        Message::from_str("system", "You are a helpful assistant."),
        Message::from_str("user", "Describe this image in one sentence."),
        // To pass a local image, use `Message::from_image_bytes`, for example:
        // Message::from_image_bytes("user", "jpeg", include_bytes!("sunset.jpg")),
        Message::from_image_url("user", "https://transformrs.org/sunset.jpg"),
    ];
    let keys = transformrs::load_keys(".env");
    let key = keys.for_provider(&Provider::DeepInfra).unwrap();
    let model = "meta-llama/Llama-3.2-11B-Vision-Instruct";
    // Using the OpenAI-compatible API for chat completions.
    let resp = openai::chat_completion(&key, model, &messages)
        .await
        .unwrap()
        .structured()
        .unwrap();
    println!("{:?}", resp.choices[0].message.content);
}

A sunset over a body of water.

More examples including text to speech and text to image can be found on https://transformrs.org.

For now, transformrs supports DeepInfra, Google, Hyperbolic, and OpenAI. I'm working on adding support for Azure, CentML, FriendliAI, SambaNova, Together.ai, Fireworks, Groq, Cerebras, and Novita.

For all these providers, my plan is to add API keys to the CI tests to ensure that everything works. Currently, CI does already test against Deepinfra, Google, Hyperbolic, and OpenAI. This works surprisingly well so far. These providers are rock solid so CI is not flaky at all. My hope is that by testing all these providers extensively, that developers who use transformrs don't have to. The aim is that developers can create tools with transformrs and let users decide themselves which provider they like to use.