Does anyone use a pattern with traits generated from a specification?

1

I'll admit my grasp on this idea is vague, so my terminology is probably off in at least some way. Here's the workflow:

  1. Write a specification in some domain-specific parsable format
  2. Write/run a tool which outputs a Rust file containing translations of any pure data types into structs/enums, and any behaviors as traits that work off of the data types
  3. In another module, implement all of those traits on your own types
  4. Pass the custom types through the generated types to an underlying layer of the application

If implemented properly, I believe this makes it easy to write software that matches a specification, while not compromising on maintainability. I see this mostly being useful for bridging the "design" of an application (in the form of the specification) with a third-party framework.

The pattern itself is abstract, so I'll try to give some more concrete examples of it in hopes of getting the point across more clearly:

Trustfall adapter stubs

Trustfall is a crate for turning "anything" into a database. To use it, one writes a schema (GraphQL-like) and feeds it to the library at runtime, and the library calls back into an Adapter with strings indicating what data to provide. This is a powerful model, but in my opinion the dependence on string-typed data is a hindrance to design and iteration. There's a tool that generates the ideal skeleton automatically, but it's only usable one time (can't update the output without discarding changes).

Instead, imagine that the tool output enums for each of the edge types (where it currently generates string matches inside of various files). Then it output a domain-specific "adapter implementation" trait for users to implement, which has methods for each case that is currently generated as a todo!(). Finally, it would generate a bridge struct implementing the trustfall::provider::Adapter trait in terms of the generated trait to close the gap. Shown in pseudo-types:

// not shown: the actual enums generated by this process
trait AdapterImplementation {
    // methods implemented in terms of the enums, in place of the todo! items
}
struct AdapterBridge<A>(A);
impl<A> trustfall::provider::Adapter for AdapterBridge where A: AdapterImplementation {
    // implements the normal trait in terms of the generated enum
}

I suggested this concept (explaining it even more poorly; I should really stop only doing GitHub while heavily sleep-deprived) last August: feature request/concept: statically-typed+reexecutable alternative to `trustfall_stubgen` · obi1kenobi/trustfall · Discussion #801 · GitHub. Unfortunately, I haven't actually built this yet, but I plan to get there "soon" (for some value of soon).

OpenAPI Spec → Axum

One complaint I have with how OpenAPI is used in practice is that the majority of the time, the specification is generated from the server code (as documentation) rather than being written as a specification and then implemented on the server. Once again, there's tooling that can bootstrap a server implementation from an OpenAPI specification, but nothing that can do so more than once, so if the specification evolves, nothing is done to update it.

Instead, the specification document could be used to produce a trait which has methods for handling each route, with the appropriate parameters encoded into the trait methods. It could also generate a function that creates a Router from an implementation of the trait. The main issue right now is that a design which allows passing other extractors, such as state, to the handlers is currently left as an exercise for the reader.

Just like for Trustfall, this would allow for statically ensuring that the server endpoints match the specification without losing the ability to freely update the specification itself. It encourages designing a "good" API first without preventing gradual feature addition and modifications. The primary downside for this design is that it strips away Axum's flexibility.

Those two are the main use cases I've considered for this pattern. What do you all think of this as a concept? Does anyone here use or know of a tool that behaves like this?

1 Like
2

I also think this would be useful.

Regarding the conversion OpenAPI to Axum and the need to manually adjust extractors : if the logic for the specification is sufficiently complete (i.e., can cover a wide range of use cases), it should be possible to generate both the OpenAPI specification and the Axum handlers from your higher-level specification.

In some way, this approach looks like a compiler from the DSL to rust & OpenAPI.

1 Like
3

I hadn't considered the option of making a new DSL and generating both the code and the OpenAPI from it. Definitely worth looking into that.

My original plan (that I left out of the original post because it's not a "clean" solution in my eyes and I'd like it if someone came up with a better one) is a lot more basic; put an extension in a normal OpenAPI doc. Using the semi-default Pet Store example, something like this:

paths:
  /pet:
    put:
      x-axum-server-extra-extractor:
        - crate::State

Boom, there it is. But putting parameters like that (implementation detail) into the specification (design document, in this model) feels like a bad idea, even if it's one of the simpler ways to do it (at least until request bodies become something that matters to the extractors and proper extractor ordering becomes a non-trivial concern). Using a different document and extracting both the design and implementation artifact from it at least keeps those details out of the API spec.

At the risk of thinking in writing, maybe the DSL could deal with the harder parts of that problem by allowing the user to overwrite the signature arbitrarily. Definitely worth prototyping it out.

4

Would it not make more sense to generate these specifications from the code? I believe this is what GitHub - oxidecomputer/dropshot: expose REST APIs from a Rust program does for example (generate an openapi spec from the rust server code) as well as GitHub - async-graphql/async-graphql: A GraphQL server library implemented in Rust (generate the graphql schema from the code).

Why would you prefer the other way around? Keeping things in sync is much harder, since you need to merge changes into the code base when you regenerate the code after changing the schema (as you pointed out is an issue with trustfall already).

Having a separate layer of traits isn't ideal either, now you have to look in two places to track what is going on, and you have more code for the compiler to deal with (slowing down compile times).

I don't think the spec->code will ever work as well as code->spec. Or at least, it will be a significantly more complex way of doing it.

5

I disagree pretty strongly with doing this, in this case. In both examples, interoperability between separate components/codebases that rely on the specification is important (databases have queries, web servers have clients). Therefore, changing the spec will usually be a high-cost action that will require changing both of the sides, and that gives me reason to believe that there's a good case for making the specification the shared source of truth and building the implementations against it. That's different from other cases, such as documentation, where making the code the source of truth is a good idea.

Put another way, I think that "keeping things in sync" is a hard engineering problem by default, and the specification-first approach has the effect of making a specification update an explicit decision rather than a result of an implementation change. In exchange for a bit more effort, it's more obvious when things are changed, and it creates a harder API boundary (though I'll admit that whether a hard boundary is good or bad is situational and subjective).

Also, keep in mind that while this is technically spec → code, it only generates an interface layer; very little about the actual behavior is generated (essentially just boilerplate). I think generating code from specifications is generally frowned upon for good reasons, but in these cases, generating similar interface skeletons is already a fairly normal thing to do.

In the OpenAPI/Axum case, in particular, it's worth pointing out that the OpenAPI Initiative has a stated preference:

There have been a number of heated debates over the relative merits of these two approaches but, in the opinion of the OpenAPI Initiative (OAI), the importance of using Design-first cannot be stressed strongly enough.

Going the other way is very doable in Axum, using GitHub - juhaku/utoipa: Simple, Fast, Code first and Compile time generated OpenAPI documentation for Rust. It's just not an approach I care for when given a choice.

On the other hand, I can see more little value in a code-first approach for Trustfall (I'm pretty sure most users, including the use cases I've actually worked with, are single repositories where the result of being out of sync is mostly having integration tests fail until the client is updated). However, the library does not support doing this right now, and writing that code → spec generator would almost certainly be a lot more labor-intensive than the interface code generation approach. Also, I think that working in terms of the schema first will probably make the schema much easier to understand and use than extracting the schema from the code.