APIs as First-Class Citizen

community
1

A few years ago, I encountered neovim and a new concept: APIs as first-class citizens.

As I used it more and more in my work, I became increasingly aware of its importance.

Exposing the software's state and operations through APIs greatly improves the debugging experience, provides extensibility, and makes it very convenient for other programs to call them.

On the other hand, centering on APIs can also improve program structure, because the entire program is written around a single context.

Currently, I usually use HTTP as the external interface because it allows for operations using curl and exposes these operations to the front end.

Previously, I used Axum as the HTTP server and utoipa to export the API to OpenAPI. Then, I used tools to convert the OpenAPI as TypeScript code for front-end.

However, I quickly encountered some problems. First, utoipa's documentation isn't generated during the build process; I need to run cargo test separately to generate the OpenAPI documentation. Second, utoipa's procedural macros are sometimes complex than the functions themselves. Finally, Axum's type system is overly complex, making every interface writing or modification a painful experience.

Furthermore, while OpenAPI to TypeScript tools are available, their quality is often inconsistent, and the generated code is rarely usable.

I'm hoping for a tool that generates Rust/OpenAPI/TypeScript code by writing something like a proto file. For Rust, ideally, it would generate a trait, so implementing the trait would automatically implement the HTTP interface. In other words, I'd like a tool like this:

interface API {
    @get
    void hello_world();

    @post
    string hello_world2();
};

I've carefully studied tools like gRPC/TypeSpec, but I'm not entirely satisfied.

Since I have prior experience writing parsers and code generation tools, I finally decided to create my own—XIDL. I made the following improvements:

First, I generated the trait and OpenAPI documentation using xidl-build, thus resolving the inconsistency in OpenAPI generation time.

Second, I exposed the HTTP interface as an asynchronous trait, hiding its internal complexity within the generated codes. This eliminates the need for developers to struggle with Axum's complex types.

Furthermore, I strictly defined the interface behavior using RFCs, ensuring consistent presentation across different languages. Simultaneously, I used behave and hurl to ensure interoperability among all generators.

Furthermore, I wrote a simple LSP that allows real-time preview of the OpenAPI interface via scalars. This enables direct HTTP API calls within the browser.

2

A new concept? Really?

I started out programming in the late 1970's. We were introduced to early ideas about program design. "Structured Programming", The "MASCOT design methodology" and such like.

A simple thing was the idea of a "context diagram". Basically a circle representing your entire program, crucially showing all its inputs and outputs. It's API as we would call it today. That could be definitions of function signatures, or I/O message content and formats.

With that in hand your dev team can get on with the internal design. Meanwhile other teams (both software and hardware) could get on with whatever was going to use your thing or be used by it.

In short the interface was always a first class citizen. When did people forget that?

3

To your questions, OOP is the short answer: famously many bad things have been said about it's ideas but the one that specifically bugs me is the idea that you should use the methods on whatever state objects as the public supported API was the worst. Smalltalk and C++ kind of just fell backwards into promoting this, but Java really pushed for it explicitly.

But that's all besides the point, as the OP is seemingly referring to APIs in the remote service sense exclusively, not the static compilation sense: we need better conventions to separate those, as they're very different in design and purpose.

Service APIs for local applications have been occasionally popular but they've never really taken off as a "here's a great idea you should do" for whatever reason, they are fairly handy. Personally I like to serve a custom debugger web app on the API root too so you can poke around with just navigating to localhost:[some port].

Personally I'm not a big fan of REST/OpenAPI for RPC style APIs like this often turns into, it pushes you into clunky decisions when your internal model doesn't map cleanly to a set of independent resources with basic CRUD operations. The explicitly RPC libraries that just dump (selected!) methods on the wire feel a lot more natural to me. But, do whatever feels natural and works for your projects, as always!

4

I implied "remote service" sense when I said "or I/O message content and formats.". I was not just talking about compiled function signatures. Those "context" diagrams I mentioned would often be showing a processes inputs and outputs as a messages received over the wire from other processes likely on different machines. Even then the things we were building were multi-process and multi-processor.

The high level point was to specify your inputs and outputs, at whatever level. To my mind that is API design. API was always a first class citizen.

I'm inclined to think you are right. Shortly after the era I describe came OOP and with it Unified Modelling Language (UML). When I was first confronted with a company that insisted on using UML (Nokia) I thought I was losing my mind. Requiring phenomenally complicated diagrams that it was impossible to extract any useful information from. Luckily I found that every one else felt the same, over time UML got sidelined in those projects, except for sequence diagrams... Which kind of got us back to where we were with the old context diagrams.