Cai - The fastest CLI tool for prompting LLMs

announcements
1

I set myself the goal to build the fastest AI for the command line!

Of course by first using the fastest language and also the fastest LLM right now:
Mixtral powered by Groq at up to 500 tokens per second!

And the first results are already quite impressive! :sunglasses:

You can find the installation instructions here:

Fast, however, means to get the final result fast and sometimes you need more than one model. So today I added support for explicitly specifying a provider:

Which additional features would you like to see? :grinning:

2 Likes
2

And I just released a new version of cai! :tada:

Including a feature I wanted for a long time:
Prompting several LLMs at once! :sunglasses:

2024-04-13t1627_all
2024-04-13t1627_all1500×944 127 KB

3

v0.6.0 is out! :tada:

Now with support for all Groq and Ollama models!

4

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.