Question About Trait Misuse and Overgeneralization


#1

Hello everybody!

I’ve been thinking a lot about how to write more testable code in Rust lately, and have been working to adopt some patterns that make things like dependency injection and mocking easier. In particular, I’ve been trying to write code that I would have written like

impl Test for HttpTest {
    type Data = (String, String);

    fn run(&self, data: Self::Data) -> Result<Response, Error> {
        let client = hyper::client::Client::new();
        for (param, data) in  self.test_data {
            let mut url = Url::parse(self.url);
            url.set_query(Some(&format!("{}={}", param, data)));
            let _ = client.get(url.as_str()).send();
        }
    }
}

instead something like

impl<T: Uploader<Data=(String, String)>> Test<T> for HttpTest {
    fn run(&self, uploader: &T, data: Self::Data) -> Result<Response, Error> {
       for (param, data) in self.test_data {
            let _ = uploader.upload((param, data));
       }
    }
}

and I would delegate having the uploader set up by the same code that sets up HttpTest. This approach lets me inject my IO-performing dependencies, so I can write mock implementations to use for testing, and that’s pretty nice.

However I’ve run into a bit of a conundrum wherein I actually have multiple, different but similarly “shaped” traits just like the Uploader you saw above which all specify their input data type, an output data type, and one method that “does the thing that needs doing” so to speak. In my case, I have to write one new implementation of an appropriate Uploader/Downloader/whatever trait for every Test implementation I write, and the result is that I feel like I’m duplicating a lot of effort and misusing traits (since all of these Uploader-lilke traits are exactly the same except for their names and method names, along with their ability to effectively do anything).

I’ve created a pastebin that illustrates my situation in greater detail but which isn’t really meant to be executable- it’s more of a demonstration of what I’m grappling with in my (closed-source) codebase at work.

I’d like to get feedback from more experienced Rust developers on how better to deal with this situation. I’ve thought about using closures instead of trait implementations, but I fear that doing so would simply result in these Uploader / Downloader / etc. implementations being moved to the content of a closure written inside of a (in my example) TestCategory implementation, which I don’t believe should have the responsibility of specifying details about making HTTP requests, setting up FTP file transfers or whatever the case may be. Further, I’m not sure how I would be able to reduce the issue of these traits (or closures, if I went down that path) being so over-generalized. Because each one takes some input of an associated type and likewise produces an output of an associated type, I feel like these traits are almost meaningless (since they could be implemented meaningfully by literally any type) save for the fact that they can be used to bound the generic T parameter accepted by Test in my example.

Please feel free to ask for clarification if any of this is confusing- I realize my concerns are quite involved. I’d very much like to take this opportunity to do some blogging about designing Rust programs for more intermediate developers, so any general guidelines would also be greatly appreciated.


#2

Not thought this issue through but you should consider if macros are more suited rather than (/as well as) generics.


#3

There’s no end to it. You’ll always find that you can parametrize whatever you have. Maybe you’ll have streaming and non-streaming uploader, and then you’ll have generic uploader over streamablebufferrable trait, which then can be configured with strategy and observers, etc. etc.

For graphics I’ve started with struct {r,g,b}, and ended up with Pixel<Channels<Colorspace, Gamma, Range>>> where each pixel has multiple phantom types. It was fun to write, and if I type “r,b,g” instead of “r,g,b” the typesystem will catch it, but now I have function signatures that take 3 lines :smiley:

In general I suggest:

  1. Always start with a concrete implementation, and get it working and tested first.
  2. Add abstraction only when you find you need it, and don’t add more. YAGNI.
  3. Consider using macros instead.

Traits takes complexity to the next level, so fighting both algorithm correctness and abstraction at the same time makes things twice as difficult. Also traits in Rust have some limits (orphan rules, types not constrained by impl or self), so there’s a risk that if you overdo abstraction Rust will say it’s all too ambiguous and you can’t have it.

Macros are fine for tests. They’re not your public API and don’t have to work with everything. They’re also good for stdlib traits (e.g. Add/Sub/etc.).