Tracing and logging for web services (OpenTracing)

Coming from a company that primarily develops services in python: we use python logging and datadog tracing for service monitoring. Here's what I expect to see from a combination of tracing and logging:

  • idiomatic logs with severities and arbitrary metadata... preferably output in JSON format compatible with our existing python JSONLogger infrastructure, but other formats are acceptable if necessary.
  • logs can be grouped by log type (e.g. "Error adding part %s to order %s" rather than "Error adding part 123 to order ABC")
  • traces (nested traces) of operations compatible with DataDog/OpenTracing
  • log messages are associated with their corresponding trace
  • BEST: libraries should come with builtin tracing e.g. database access libraries, AWS/S3, etc. If not, at least I can easily wrap libraries, whether they are synchronous or asynchronous.

I see a couple different crates related to tracing but I'm not sure if there is a good best-practices guide? Here's what I've found so far:

Thoughts? I'd love for this to be more of a solved problem and I'm just missing the right docs/guide!

Crates that supply log and tracing information are probably using the log crate. This crate doesn't provide the code that does something with the log, it's just a facade with a few macros for sending messages to the log, and a separate crate is used to send them somewhere.

The crates.io page for log lists a few backends for doing various different things with the messages. Take a look and see if any satisfy your needs.

It appears that tracing is intended to work in the same manner as log: a generic tracing facility that you can then hook up to any backend (and it itself integrates with logging so that log events are associated with traces). I guess I'm surprised that I don't see an OpenTracing backend that works with tracing.