Using Rust to read log files with a Linux Tail like logic

Hi,

I want to read log files i.e. SSH logs if there is incoming traffic on Port 22 or 2222 that looks like this in Linux tail -f /var/log/some/ssh.log

Which crate could I use for that or implementing it by myself. I already observe the incoming traffic and identify it as possible SSH network traffic.

The uutils coreutils Rust rewrite has an implementation of tail. It is published as a library, but I'm not sure it is meant to be used as such by uutils-external projects. Could offer some insights into how to implement tail yourself though. I personally would probably run tail -f as a subprocess.

2 Likes

Thanks :slight_smile:

I also use tail -f most of time. However recently I needed to see log files with timestamps in seconds from epoch converted to a human readable format, so I asked AI, and it brought me the code. It works great however the option -f is missed. So certainly I will look in your reference.

tail -f apparently just polls stat and reads the difference in file length, so it's not too tricky to do yourself. It's likely there's a bunch of edge cases you may or may not care about.

The uutils implementation seems to use inotify (or kqueue etc depending on platform) rather than polling stat. This lets it get notification on the file in being written to. They use the notify-rs crate for this. There is a polling fallback though.

Are you saying that GNU coreutils does something more basic?

I'm going from a description, but it might be either out of date or as a fallback - fs watches are less reliable in some contexts (network shares infamously so). It's also unavailable for some types like pipes, but I forget if tail -f just refuses those or degenerates to cat.

I looked at the code, in particular coreutils/src/uu/tail/src/follow/watch.rs at 5202ac137d06c8f187f55961ad4994f7fe48ce0b · uutils/coreutils · GitHub

Yeah I was taking about GNU coreutils - I've heard that described as polling on stat for forever, but I've never checked. Using a watch makes sense but it does have issues. I think even notify-rs falls back to polling...