Console logging crate with a clearable buffer?

[nk9]

I have a CLI utility which logs status as it processes hundreds of features. It only knows once the processing is done for each feature whether the feature is error-free or not. Currently, it logs the status for everything as it's going through, which means that the features with errors are mixed in amongst hundreds of features which are just fine.

I would like to only log features with errors. I'm using env_logger right now, and there's a section in the docs on Filtering results, but that uses regex, so isn't what I need. I want the logger to place lines in a buffer, which I can flush when the current feature finishes and has errors, or clear when the feature is OK.

Does anyone know of a logger which has this ability? Or is there another approach which I should consider instead? I asked ChatGPT to write me a logger that would do this, and it obliged… but I don't really trust it, LOL. It also doesn't compile due to borrow checking issues.

I have found the buffered_logger crate, but it seems to assume that you're flushing to a file, and also doesn't have the ability to clear the buffer.

[jumpnbrownweasel]

Given that multiple threads are outputting trace messages at once, to implement the ability to cancel output for a feature you would need a buffer per feature. So that's much different than simple buffering. Although it is something you could implement yourself since you already know when a feature is complete, and I assume you have an feature object of some kind where a buffer could be stored.

I think there are some tracing libs with hierarchical messages, tracing_tree for example, but I don't see a way to cancel output of a given tree. Perhaps there is a requirement to not lose messages when there is a crash, and buffering makes that more difficult or impossible.

Do you have a requirement to log the buffered messages when there is a crash?

[nk9]

Ah, multiple threads isn't something I'd even considered. Fortunately I won't be processing an individual feature on multiple threads, so I don't think that will be an issue. Also, losing messages when a crash happens would be fine.

I had considered attaching an array of strings to the feature object, but thought it would be annoying to manage… However, it didn't turn out to be that hard. The most annoying bit is that my feature object now has to be declared mutable everywhere, but that was just a matter of adding &mut where the compiler said. I've written a macro to make it almost as concise as the standard info! call:

#[macro_export]
macro_rules! log_info {
    ($def:expr, $($args:tt)*) => {
        $def.log(Level::Info, &format!($($args)*));
    };
}

//...
pub struct BoundaryDef {
    pub log_lines: Vec<String>,
// ... etc
}

impl BoundaryDef {
    pub fn log(&mut self, level: Level, line: &str) {
        let timestamp = Local::now().format("%H:%M:%S").to_string();
        let level_str = match level {
            Level::Error => format!("{color_red}ERROR{color_reset}"),
            Level::Warn => format!("{color_yellow}WARN {color_reset}"),
            Level::Info => format!("{color_green}INFO {color_reset}"),
        };

        self.log_lines
            .push(format!("[{} {:<5}] {}", timestamp, level_str, line));
    }

    pub fn clear_log(&mut self) {
        self.log_lines.clear();
    }

    pub fn flush_log(&mut self) {
        if !self.log_lines.is_empty() {
            println!("{}", self.log_lines.join("\n"));
            self.log_lines.clear();
        }
    }

And then:

log_err!(def, "No segments found for: {}", name);

And I can ignore the log at the end if I want, or print everything out. I guess this solution works pretty well, thanks for the nudge.

[jumpnbrownweasel]

Glad to hear it worked out. In case you're still unhappy with the &mut: You could wrap your Vec in a RefCell to avoid that, which adds only a very cheap runtime check when you add a string.

[nk9]

Brilliant! Thanks for the tip, that's much nicer.