Writing into a buffer and then reading back out of it


#1

Hi there, I’m having a bit of trouble understanding best practices around creating a buffer, having another function write into it, and then reading back out of it again. Ownership gets handed off to the other function, and I’m no longer allowed to access the buffer from the original context.

I’m writing a test for a new slog formatter:

#[cfg(test)]
mod tests {
    use std::io;

    use slog;
        use slog::DrainExt;
        use slog_stream;

        #[test]
        fn test_formatting() {
            let vec = vec![0; 1000];
            let buf = io::Cursor::new(vec);
            let root = slog::Logger::root(slog_stream::stream(buf, super::default()).fuse(),
                                          o!("key1" => "val1"));
            info!(root, "formatted: {}", "foo");

            let s = String::from_utf8(vec).unwrap();
            println!("{}", s);
        }
}

Note that slog_stream::stream here is looking for something with the Write trait.

The compilation error looks like this:

$ cargo test
   Compiling slog-test v0.1.0 (file:///Users/brandur/Documents/projects/slog-test)
error[E0382]: use of moved value: `vec`
   --> lib.rs:610:35
    |
605 |         let buf = io::Cursor::new(vec);
    |                                   --- value moved here
...
610 |         let s = String::from_utf8(vec).unwrap();
    |                                   ^^^ value used here after move
    |
    = note: move occurs because `vec` has type `std::vec::Vec<u8>`, which does not implement the `Copy` trait

error: aborting due to previous error

This makes sense to me (the ownership of my cursor gets handed off to the logger), but I’m not really sure how to work around it. I tried a few other things like (a) unwrapping the original buffer from my Cursor after I’m finished, and (b) handing Cursor a mutable reference to my vector instead, but unfortunately, no dice.

This is sort of an issue specific to my usage of slog, but I would think that working with these types of buffers would be a pretty common pattern, especially in testing. Does anyone have a recommendation on how I could do this? Thanks in advance!


#2

Mutable references to Writers are also Writers, so you can do something like this (note that the Cursor is unnecessary):

#[cfg(test)]
mod tests {
    use std::io;

    use slog;
    use slog::DrainExt;
    use slog_stream;

    #[test]
    fn test_formatting() {
        let mut vec = vec![];
        {
            let root = slog::Logger::root(slog_stream::stream(&mut vec, super::default()).fuse(),
                                          o!("key1" => "val1"));
            info!(root, "formatted: {}", "foo");
        }

        let s = String::from_utf8(vec).unwrap();
        println!("{}", s);
    }
}

#3

Thanks for the help!

I think this almost works, but for some reason it seems that Rust would like my vec variable to have a static lifetime:

$ cargo test
   Compiling slog-test v0.1.0 (file:///Users/brandur/Documents/projects/slog-test)
error: `vec` does not live long enough
   --> lib.rs:606:68
    |
606 |             let root = slog::Logger::root(slog_stream::stream(&mut vec,
    |                                                                    ^^^ does not live long enough
...
615 |     }
    |     - borrowed value only lives until here
    |
    = note: borrowed value must be valid for the static lifetime...

error: aborting due to previous error

I would think that this wouldn’t be necessary since all of these instances are going out of scope. Any ideas?

Here’s the modified code, which is almost identical to the suggested answer above (modulo some automatic changes from rustfmt):

#[cfg(test)]
mod tests {
    use std::io;

    use slog;
        use slog::DrainExt;
        use slog_stream;

        #[test]
        fn test_formatting() {
            let mut vec = vec![];
            {
                let root = slog::Logger::root(slog_stream::stream(&mut vec,
                                                                  super::default())
                                                  .fuse(),
                                              o!("key1" => "val1"));
                info!(root, "formatted: {}", "foo");
            }

            let s = String::from_utf8(vec).unwrap();
            println!("{}", s);
        }
}

#4

Ah, looks like slog needs a 'static writer: https://docs.rs/slog/1.4.0/slog/struct.Logger.html#method.root.

You might need to have a thread_local! buffer and a custom writer that appends to it so you can get access to it.


#5

You might need to have a thread_local! buffer and a custom writer that appends to it so you can get access to it.

Ah, got it. I see now that there’s a little more going on here than I’d originally thought about. Thanks for the help!