Is it possible re-use data sets with macros?

[couling]

For context: I'm new to rust and, most recently, come from Python and C. I'm looking for a way to achieve some of the matrix unit testing I am familiar with from Python's Pytest. In particular I'm interested in replicating Pytest's parameterized fixtures which very easily let you create test cases for a cartesian product of input values.

The project I'm working on has some stock data-sets which can be expressed as static arrays, each of 5-20 values. From experience in Python, there's real value in testing every permiatation where a function may be interested in 2-3 of these. Yes!, this results in approx 2,000 - 4,000 test cases in total.

I'm currently looking at the test-case crate which get's me a long way forward when I copy-paste the static arrays into the macro #[test_matrix()] arguments.

    // test-case documented usage...
    #[test_matrix(
        [-2, 2],
        [-4, 4]
    )]
    fn multiplication_tests(x: i8, y: i8) {
        let actual = (x * y).abs();

        assert_eq!(8, actual)
    }

What would be even better is if I didn't need to copy-paste the static arrays [-2, 2] and [-4, 4] above.

I realise that, by comparing pytest to test-case, I'm comparing a runtime framework with compile time macros and so there are some limitations.

What's the simplest way for me to pass in the same static array into several macro uses?

[zirconium-n]

Uhh just declare a static array?

static TEST_DATA: [[i8; 2]; 4000] = /* ... */;

[couling]

@zirconium-n yes, that's how you declare a static array...

The question is how to use that with a macro similar to test_matrix?

Perhapse I should have mentioned that attempting to pass an argument like TEST_DATA into the macro has resulted in type errors because it is expecting the argument type to match the test-function type...

... That's not what I'm looking for the test function type needs to be the type of just one element, not the type of the array itself. That is, the test function type needs to be i8 not [i8; 2] and not &[i8].

[jumpnbrownweasel]

@couling

The question is how to use that with a macro similar to test_matrix?

With python test_matrix, does it automatically pass each row as parameters to your test function? I don't know a way to do that automatically (declaratively) in Rust, so I would just write the for loop for each test. You can refer to the same static/const test data from multiple tests, if you declare that test data in the test module.

Perhaps it is possible to write a Rust macro that outputs the for loop, but I will have to defer on that to those who know more about Rust macros.

[couling]

@jumpnbrownweasel I've not interrigated the code, but for rust macro test_matrix AFAIK it's a macro that is writing out #[test] functions which individually pass arguments into the test function I've written. So the above snipit results in cargo test reporting on 4 tests.

Pytest in python is a runtime framework, so it's a lot simper to do the same thing there.

[drewtato]

Macros work on tokens, so in order to have m * n test functions generated, you need m tokens in one list and n tokens in another. You could do that by making your own macro that generates the test_matrix attributes. This would need to either be another procedural attribute macro, or a declarative macro that wraps the function. Probably something like this (using tokio::test since that's in the playground): Rust Playground

macro_rules! tm {
    ($name:ident, $($f:ident),* $(,)?) => {
        mod $name {$(
            #[tokio::test]
            async fn $f() {
                super::$f().await;
            }
        )*}
    };
}

tm! {
    some_tests,
    a, b, c
}

async fn a() {}
async fn b() {}
async fn c() {}

[couling]

I don't suppose it's possible to simply expand a macro before it's passed to another macro.

As an experiment I tried:

    macro_rules! foo {
        () => {
            [
                (0, &[0x00]),
                (127, &[0x7F]),
                (128, &[0x80, 0x01])
            ]
        };
    }

    #[test_matrix( foo!() )]
    fn bar(example: (u32, &[u8])) {
        // ...
    }

But the result was not the same as:

    #[test_matrix( [(0, &[0x00]), (127, &[0x7F]), (128, &[0x80, 0x01]) )]
    fn bar(example: (u32, &[u8])) {
        // ...
    }

It looks like the test_matrix is able to expand the [ ... ] of the second but not the first. Somehow for the first it seems to see just one token and does not expand the square brackets.

[jorendorff]

Macro expansion works from the outside in. It's the opposite of function calls. So the test_matrix thing is being expanded and passed foo!() as a string, not the expansion of foo!.

But setting aside all the details about macros for a second... consider writing some normal code for this, something like

#[test]
fn test_bar() {
    for test_case in TEST_CASES {
        bar(test_case);
    }
}

Later, once you've got a dozen of those, it'll be much more obvious how you should use macros to eliminate boilerplate.

[couling]

@jorendorff

But setting aside all the details about macros for a second... consider writing some normal code for this, something like

Ooof. No. That was my first thought and I've spent a couple of days searching for a way to avoid that.

The reason I don't write something like that is pretty much the same as why I don't write every single unit test in some monster monolithic test:

• Test performance with threading
• Finding every failed case in a single run, not just the first one
• Re-running individual cases rather than the whole suit
• (Cognitive) debugging speed when specific combinations fail
• ...

Copy-pasting test cases is sub-optimal but the result is actually much easier to live with.