How to work around a crate's structs having Deserialize but not Serialize


#1

Hello, I’m working on learning Rust, and I thought I’d try and use it for a problem that I had at work a few days ago. Background, for anyone interested:

I am a developer, and our company has a system where we don’t have credentials for our production AWS environment. We have to get our operations people to do anything that involves accessing production AWS. I needed to get the results of a few dozen AWS SWF executions to debug an issue, so I wrote a short Python script to get the info and save it to a JSON file for my analysis. It should have been simple, but the particular ops engineer had some issues with his Python install and it ended up taking a while to get the dependencies sorted out. I thought it would be nice if I could do the same task in a compiled language that would not have any dependencies on the system it was installed on to avoid this, so I decided to try to do the same thing in Rust.

The task seems simple, in that I just need to get a few dozen responses to AWS SWF calls and write the full body as JSON to a file. There seems to be a handy rusoto_swf crate that will set up AWS credentials and make all of the required requests for me. Great! Only problem is that the response object I need, WorkflowExecutionDetail, implements Deserialize, but not Serialize. It’s handy that it comes back as a nice Rust struct, but I just want to write the JSON back out to a file.

It seems I can’t make any changes to the crate’s struct, or to the type returned by the crate’s method, so the only way I could figure out to make it work was to copy the code for those particular calls into my app and make the changes there. I was able to modify the code to parse the response as a generic serde_json::Value, which allowed me to combine it with the rest of the responses and output back out as JSON text. This all seems pretty awkward though, could there be any easier way to do this?

I did also try outputting the data structures using the debug output. That does produce output, but it isn’t JSON, so it’s much harder to search through with other tools.


#2

Is the lack of Serialize just an omission? If so, maybe make a pull request to the project defining the struct.

If the struct isn’t in your project, and the trait isn’t in your project, then you can’t implement it yourself. Copy&paste of the struct is necessary. You could probably write a macro to make that copy-paste easier.

For debug there’s a pretty-print version {:#?} that’s slightly easier to read, but the debug format is not a serialization format and may change.


#3

you could create a newtype and implament Serialize on the newtype.
some thing like

struct WEDSerialize<'a>(&'a WorkflowExecutionDetail);

impl<'a> Serialize for WEDSerialize<'a> {
...
}

then you could use serde_json::to_string(&WEDSerialize(&wed))


#4

That’s probably what I would’ve done as well. It sounds like rusoto does not provide a lower level API where you can essentially reuse its logic to perform the REST calls, but just get the raw json payload back. Writing your own Serialize impl over a newtype is possible, but unnecessary work given you don’t care about the structs in the first place. Moreover, it’s possible their object model skips some data from the raw json, but which you may find useful for your own needs - that also means even if they were to derive Serialize, you may still end up missing this data in your output.


#5

On the one hand, I might consider it an omission, since adding it wouldn’t hurt anything and would be handy in cases like mine. On the other hand, basically the entire rusoto project’s source code is automatically generated from AWS API documentation in JSON, so changing that particular struct’s trait macros used would probably require changing it for all of them in the project. That’s an awful lot of politics for a handy feature for a minor utility.

I did try out the debug pretty-print, but unlike JSON, there’s no way to pull it back into programs for analysis.


#6

Interesting, but does the macro to auto-implement Serialize work for that? Seems like more work to actually write a serialize implementation than to copy and tweak the project’s code.


#7

Thanks, that’s the impression I was getting on the situation.


#8

I’ve found there is actually a solution to this:

https://serde.rs/remote-derive.html


#9

Good find! I’ll give that a try.