How To Override #[cfg(test)] "Mocks"?

Suppose I have a rust function named "Foo" that makes a call to a function in another file/module that gets some json. For the actual build and integration tests I want it to really make the api call, but for unit tests I want it to return some hardcoded blob of json.

Here is what my code looks like for this file:

use serde_json::Value;
use std::error::Error;

use crate::shared_constants::URL;
use reqwest::blocking::get;

#[cfg(not(test))]
pub fn get_some_json() -> Result<Value, Box<dyn Error>> {
    let response = get(URL)?.text()?;
    let serde_result = serde_json::from_str::<Value>(&response)?;
    Ok(serde_result)
}

#[cfg(test)]
pub fn get_some_json() -> Result<Value, Box<dyn Error>> {
    let input = r#"{"index":0,"name":"AB/CDE/FG/402/test_int4","sts":"on","time":"2021-06-05 03:28:24.044284300 UTC","value":8}"#;
    Ok(serde_json::from_str(input)?)
}

This does work but it's not perfect... for one Rust analyzer in my ide makes the true implementation of the function darkened to the point where I can't even read it... is that supposed to be helpful? Lol

Also, this code is giving me errors for unused imports on lines 4 and 5. Is the correct thing to do actually mark both 4 and 5 with the #[cfg(not(test))] macro? That would seem like an awfully not of duplication of the not test macro to me...

Ok, so the REAL question I wanted to ask is how can I override the test implementation. Suppose for some of my Foo unit tests I want to check that it does the right thing with the json response, but for some Foo unit tests I was to check that it correctly handles the care where getting json actually fails and returns the error variant of Result... what is the recommended way to do this?

thanks!

Maybe better not to override it?

use serde_json::Value;
use std::error::Error;

pub fn get_some_json() -> Result<Value, Box<dyn Error>> {
    todo!()
}

#[cfg(test)]
mod mock {
    use serde_json::Value;
    use std::error::Error;
    
    fn get_some_json() -> Result<Value, Box<dyn Error>> {
        todo!()
    }

    #[test]
    fn real_test() {
        super::get_some_json().unwrap();
    }

    #[test]
    fn mock_test() {
        get_some_json().unwrap();
    }
}

The issue with the proposal above is that mock::get_some_json() cannot be called transitively by other functions you are testing.

You can either use one of the many "mocking" crates, or look at the more general (and much simpler, IMHO) approach of dependency injection. DI in Rust is best done parametrically with generics and trait bounds. This article appears to be a decent introduction to the idea: Rust traits and dependency injection - Julio Merino (jmmv.dev) Another one is the DI example in Types Over Strings: Extensible Architectures in Rust | Will Crichton.

Also, it is worth pointing out this quote by staticassert:

A lot of people conflate Dependency Injection with DI Frameworks - like Spring or Juice.

Exactly. I don't see how @ThalliMega's response is at all relevant to my question... I want to override the mocked "get_some_json" in my Foo unit tests.

No offense @parasyte, but that DI code you linked to... is the ugliest, grossest Rust code I have ever seen. :joy: :joy: :joy:

I didn't write it, so I'm not bothered in the slightest! And it is definitely not the worst Rust code I have ever seen. :wink:

On a serious note, "just pass your database-or-whatever as a generic parameter" is the only thing there really is to know about DI in Rust. Here are some other threads that go over the details, if it's helpful:

In my experience, I've found that it's almost always a bad idea to use #[cfg(test)] to mock out the implementation of something.

Using dependency injection[1] is the proper solution here. If your code depends on some sort of HTTP client then pass it into get_some_json() as an argument. Once you start passing dependencies into your code instead of hard-coding them by (for example) using reqwest::blocking::get() directly, it becomes pretty easy to abstract out.

For example, you could use a closure that will execute a GET request:

pub fn get_some_json<F>(get: F) -> Result<Value, Box<dyn Error>>
where 
    C: Fn(&str) -> Result<reqwest::blocking::Response, Box<dyn Error>>
{
    let response = get(URL)?.text()?;
    let serde_result = serde_json::from_str::<Value>(&response)?;
    Ok(serde_result)
}

In the real code you would do something like this:

fn main() {
    let data = get_some_json(|url| {
        let response = reqwest::blocking::get(url)?;
        Ok(response)
    }).unwrap();
}

And in testing you would do something like this:


let dummy_response = reqwest::blocking::Response::from(dummy_response);

let data = get_some_json(|url| {
    // your test probably wants to make sure it hit the right endpoint
    assert_eq!(url, "https://example.com/");

    // create a dummy response that we can use
    let payload = r"#{ ... }";
    let dummy_response = http::Response::new(payload.to_string());
  Ok(dummy_response.into())
}).unwrap();

assert_eq!(data, ...);

You could introduce some sort of HttpClient trait and implement that for both reqwest::blocking::Client and some dummy testing struct if you find that cleaner (closures are a poor man's object, after all), but the idea remains the same - pass the dependency into your code instead of hard-coding the dependency and trying to hackily patch it out later.


  1. The CS concept, not the Java-esque monstrocities you get when people try to abstract things out into a "framework". ↩︎

I don't really see what the difference is between your code dependency injected code and just passing it as a regular function argument. And to me it seems like you aren't really mocking or testing anything here, just moving pieces out to be not tested somewhere else.

I wasn't trying to compare dependency injected code versus function arguments, they're the same thing. I was trying to show that if your code has a dependency it should be passed in as an argument instead of hard-coding that dependency and using conditional compilation to try and hack in mocks for testing purposes.

Dude, it has to be passed in as an argument from somewhere so passing it in is not an answer. How are you testing it in the place you passed it in from?

I'm not sure if you've misunderstood the point was Michael was trying to make, but I don't think its obvious the function to call has to be passed in from anywhere, because your original setup was not doing that - it was resolving the function by looking up a static name, and then you had a mess of cfg directives to change how that static name was defined. Presumably your function Foo has some expression that looks like,

fn Foo(...) {
   // ...
   let json = get_some_json();
   // ...
}

To make Foo specifically more testable, it's easier to have

fn Foo<F>(..., json_getter: F) 
   where F: Fn() -> Result<Value, Box<dyn Error>> 
{
   // ...
   let json = json_getter();
   // ...
}

And then your tests can pass whatever dummy function they want as the json_getter. This doesn't improve the testability of the real get_some_json, but it does decouple testing that function from testing Foo. You can continue this idea and restructure get_some_json for better testability as well, but eventually you'll hit a layer that does the actual IO, and you fundamentally can't "unit" test that because it needs that external service. You can, however, make that layer as thin as possible so less can go wrong.

4 Likes

The larger question you are asking (perhaps without realizing it) is "how do you write testable code?" That is what @Michael-F-Bryan is addressing.

If you don't want to write testable code and you really just want to hard code functions depending on build profile, there is a macro for that: https://crates.io/crates/test_double

2 Likes
use reqwest::blocking::get;
use serde_json::Value;
use std::error::Error;

// use crate::shared_constants::URL;
const URL: &str = "https://httpbin.org/anything?hello=world";

pub fn get_some_json(input: &str) -> Result<Value, Box<dyn Error>> {
    let serde_result = serde_json::from_str::<Value>(input)?;
    Ok(serde_result)
}

fn main() -> Result<(), Box<dyn Error>> {
    let response = get(URL)?.text()?;
    let json = get_some_json(&response)?;

    println!("{json:#?}");
    Ok(())
}

#[cfg(test)]
mod tests {
    use super::*;
    use serde_json::json;

    #[test]
    fn test_get_some_json() {
        let input = r#"{"index":0,"name":"AB/CDE/FG/402/test_int4","sts":"on","time":"2021-06-05 03:28:24.044284300 UTC","value":8}"#;
        let json = get_some_json(input).unwrap();

        assert_eq!(
            json,
            json!({
                "index": 0,
                "name": "AB/CDE/FG/402/test_int4",
                "sts": "on",
                "time": "2021-06-05 03:28:24.044284300 UTC",
                "value": 8,
            })
        );
    }

    #[test]
    fn test_get_some_json_with_very_different_json() {
        let input = r#"[1, 2, "skip a foo", 99, 100]"#;
        let json = get_some_json(input).unwrap();

        assert_eq!(json, json!([1, 2, "skip a foo", 99, 100]));
    }
}

You'll have to forgive me for the contrived example, but honestly I'm just taking the code from the OP and making it testable. There is no magic, nothing special is needed. It's just organization.

You may claim that this kind of code organization doesn't do anything at all. And you will be right! I think everyone in the entire thread has said as much.

It should go without saying, but the reason this "pass in what you want to use" organization is better for testing than conditional compilation is that you are given the opportunity to pass any number of inputs for your tests. With the organization shown in OP, get_some_json() can only return one thing.

Once again, the example tests are contrived; there is almost nothing interesting happening here. I argue, that's the way it should be. Tests are meant to be readable. But the point stands that it is far more capable than the hardcoded function.

Also, please keep in mind that we are sincerely trying to help you. You are free to reject the help.

4 Likes

I guess this is the best we can do in Rust.

Sure, it's easy to only test the pure functions, but my point is that you are not testing all the coordination logic that you have now moved to main.rs.

You can make the function that handles the dependency injection private, and only export the function with your original signature. Then you can test that public function as well as the private mock-able version to cover all of your bases

use std::error::Error;

// use crate::shared_constants::URL;
const URL: &str = "https://httpbin.org/anything?hello=world";

// the module is here to prevent main from being able to call `get_some_json_inner`.
mod json {
    use reqwest::blocking::get;
    use serde_json::Value;
    use std::error::Error;

    /// Public function with the same interface as your original.
    pub fn get_some_json() -> Result<Value, Box<dyn Error>> {
        let response = get(crate::URL)?.text()?;
        get_some_json_inner(&response)
    }

    /// Can't be used outside of the `json` module
    fn get_some_json_inner(input: &str) -> Result<Value, Box<dyn Error>> {
        let serde_result = serde_json::from_str::<Value>(input)?;
        Ok(serde_result)
    }

    #[cfg(test)]
    mod tests {
        use super::*;
        use serde_json::json;

        // We can still test the inner function without making a network request
        #[test]
        fn test_get_some_json_inner() {
            let input = r#"{"index":0,"name":"AB/CDE/FG/402/test_int4","sts":"on","time":"2021-06-05 03:28:24.044284300 UTC","value":8}"#;
            let json = get_some_json_inner(input).unwrap();

            assert_eq!(
                json,
                json!({
                    "index": 0,
                    "name": "AB/CDE/FG/402/test_int4",
                    "sts": "on",
                    "time": "2021-06-05 03:28:24.044284300 UTC",
                    "value": 8,
                })
            );
        }

        #[test]
        fn test_get_some_json_inner_with_very_different_json() {
            let input = r#"[1, 2, "skip a foo", 99, 100]"#;
            let json = get_some_json_inner(input).unwrap();

            assert_eq!(json, json!([1, 2, "skip a foo", 99, 100]));
        }

        // But we can also make sure the public interface function is behaving as expected, without making any assumptions in the test about how it's called.
        #[test]
        fn test_get_some_json() {
            let json = get_some_json().unwrap();

            assert!(json.is_object());
        }
    }
}

fn main() -> Result<(), Box<dyn Error>> {
    // main doesn't need to know about the inner function, it can use the old function signature just like before
    let value = json::get_some_json()?;

    println!("{value:#?}");
    Ok(())
}

2 Likes

Thanks @semicolon

This is very similar to my original code, but I leaves me stuck with the exact same problem- how can you write a test for main which checks that it properly handles the case when get_some_json actually errors out?

You can unit test main, but you can't run integration tests on it. Which is why the prevailing recommendation is to make it as small as you possibly can and move the business logic to a library (which can also be integration tested). Test Organization - The Rust Programming Language (rust-lang.org)

... This is one of the reasons Rust projects that provide a binary have a straightforward src/main.rs file that calls logic that lives in the src/lib.rs file. Using that structure, integration tests can test the library crate with use to make the important functionality available. If the important functionality works, the small amount of code in the src/main.rs file will work as well, and that small amount of code doesn’t need to be tested.

Testing main itself is usually awkward because it might depend on the environment. E.g. a command line interface. Or it might open a window and run an event loop.

2 Likes

If it helps, remember the rule of thumb: unit tests should avoid accessing anything from the environment, ideally, while integration tests should test everything working together as it will in production.

Given that a main() can't do anything itself without accessing the environment (since it takes no arguments and doesn't return anything other than failure) there's really no reason to unit test it.

Philosophically, you should probably best think of unit tests as a way to pin down the interface between units. Integration tests are the ones that test actual functionality.

3 Likes