Serde and BufWriter

help
1

I am trying to serialise a struct with serde

Following the documentation I have been adding

#[derive(Serialize, Deserialize)]

on the structs, and recursively the contained structs

My struct writes logs so has a reference to a struct that has a a BufWriter (it writes logs of the computations it is involved with) and I get the error:

serde::Serialize is not implemented for std::io::BufWriter<std::fs::File>

I tried putting #[serde(skip_serializing)] before the offending field to no avail. (I do not need to serialise the reference to the log writer).

W

2

You probably don't want to have the struct that writes logs annotated with Serialize or Deserialize. Instead, you should be annotating structs that represent a record/row in the log.

Can you expand a bit on why you have the current setup?

3

Here is a minimal example. If I could get this to compile....


#[macro_use] extern crate serde_derive;
extern crate serde;
extern crate serde_json;

use std::io::BufWriter;
use std::fs::OpenOptions;
use std::time::SystemTime;
use std::fs::File;


#[derive(Serialize, Deserialize, Debug)]
struct Fud<'a> {
    keys:Vec<String>,
    r:&'a Recorder,        
}

#[derive(Serialize, Deserialize, Debug)]
pub struct Recorder {
    buffer:BufWriter<File>,
    created:SystemTime,
}
impl Recorder {
    fn new(file_name:&str) -> Recorder {
        Recorder{
            buffer:BufWriter::new(OpenOptions::new()
                                  .append(true)
                                  .create(true)
                                  .open(file_name).unwrap()),
            created:SystemTime::now(),
        }
    }
}

fn main() {
    let r = Recorder::new("temp.txt");
    let fud = Fud{keys:vec!["a".to_string(), "b".to_string(), "c".to_string()], r:&r};
}
4

Right - only Fud should be serializable here, not the recorder, and it shouldn’t need a reference to the recorder as a result.

How do you intend/want to do the writing and from where?

5

The Recorder struct is used. Maybe not in this minimal example but Fud will be using it.

I should have included a impl for Fud in the example.

This is clearer


#[macro_use] extern crate serde_derive;
extern crate serde;
extern crate serde_json;

use std::io::BufWriter;
use std::io::prelude::*;
use std::fs::OpenOptions;
use std::time::SystemTime;
use std::fs::File;


#[derive(Serialize, Deserialize, Debug)]
struct Fud<'a> {
    keys:Vec<String>,
    r:&'a mut Recorder,        
}
impl<'a> Fud<'a> {
    pub fn foo(& mut self){
        self.r.write_line("A string");
    }
}

#[derive(Serialize, Deserialize, Debug)]
pub struct Recorder {
    buffer:BufWriter<File>,
    created:SystemTime,
}
impl Recorder {
    fn new(file_name:&str) -> Recorder {
        Recorder{
            buffer:BufWriter::new(OpenOptions::new()
                                  .append(true)
                                  .create(true)
                                  .open(file_name).unwrap()),
            created:SystemTime::now(),
        }
    }
    fn write_line(&mut self, line:&str) {
        let now = format!("{} ", self.created.elapsed().unwrap().as_secs());
        self.buffer.write(&now.into_bytes()[..]).unwrap();
        self.buffer.write(&line.to_string().into_bytes()[..]).unwrap();
        self.buffer.write(&"\n".to_string().into_bytes()[..]).unwrap();
    }
}

fn main() {
    let mut r = Recorder::new("temp.txt");
    let mut fud = Fud{keys:vec!["a".to_string(), "b".to_string(), "c".to_string()], r:& mut r};
    fud.foo();
}
6

So Fud will be json serialized somewhere as well as writing to the BufWriter directly? If so, you should make a view of Fud (i.e. a separate struct) that is used for json (serde) serialization. So something like:

#[derive(Debug)]
struct Fud<'a> {
    keys: Vec<String>,
    r: &'a mut Recorder,
}

impl<'a> Fud<'a> {
    pub fn foo(&mut self) {
        self.r.write_line("A string");
    }

    pub fn get_view(&self) -> FudView {
        FudView { keys: &self.keys }
    }

    // or hide the FudView entirely
    pub fn serialize<S: serde::Serializer>(&self, s: S) {
        let rec = FudView {keys: &self.keys};
        rec.serialize(s).unwrap();
    }
}

#[derive(Serialize)]
struct FudView<'a> {
    keys: &'a [String],
}

#[derive(Debug)]
pub struct Recorder {
    buffer: BufWriter<File>,
    created: SystemTime,
}

Alternatively, restructure the types involved to more clearly separate the serde vs direct writing.

7

I see. For my needs, currently, that is a nice solution.

The point being I cannot include a BufWWriter in a serialised object? That would make some sense, but I could see a case where I would want to resume logging.

What can and cannot be serialised?
W

8

That's right - there's no implementation of serialization for it, which makes sense because it's not an object you'd want to serialize (under reasonable circumstances). You could, for example, store it as a Option<BufWriter<File>>, and add #[serde(skip)] so that it'll be ignored. But that pollutes your type with an otherwise unnecessary Option.

Whatever has a serde::ser::Serialize - Rust implementation for it, which may also be (frequently) derived (like you were trying to do for your types).

If you've not yet, you may want to read through https://serde.rs/ to get a better understanding of serde.

1 Like
9

Thanks. I have enough to be going on with now.
W