Searching / monitoring https://docs.rs/tracing/latest/tracing/ events in real time

#1

Problem: I've finally reached the point where printf debugging no longer works.

I have 4-5 Rust servers all logging on /app/logs.

This is currently logging events under "PrettyPrint", but I can just as easily change it to Json / OpenTelemetry formats. I don't actually care about the logging format and am happy to switch to whatever has the best tooling.

I am currently doing cat /app/logs/* | grep ... | less. I suspect there is better tooling for this.

I am interested to hear what others are using. I do need it to be something I can self host. (Not a fan of depending on third party service).

#2

A small trick: Instead of cat you could use tail -f. This will print every line as they are added to the log files. I think it only accepts a single file as argument though so you may need to pipe the result of several tail -f commands into a single file and then tail -f this file. You could also use named pipes as log "files". You can make one using mkfifo. Of course something with rust integration would be nicer than this hack.

#3

It supports multiple. And if your logs rotate, use -F instead of -f.

(The GNU version anyway, I didn't check any others.)