Code works on rust playground but not in test code

Here is the code in playground: Rust Playground

Here is the code in test code:

#[cfg(test)]
mod serde_tests {
    // use super::*;
    use serde::{Deserialize, Serialize};
    use serde_json;

    #[derive(Debug, Serialize, Deserialize)]
    struct CurrencyRates {
        date: String,
        #[serde(flatten)]
        rates: Rates,
    }

    #[derive(Debug, Serialize, Deserialize)]
    #[serde(untagged)]
    enum Rates {
        Idr {
            #[serde(rename = "idr")]
            idr: CurrencyMap,
        },
        Usd {
            #[serde(rename = "usd")]
            usd: CurrencyMap,
        },
    }

    #[derive(Debug, Serialize, Deserialize)]
    #[serde(untagged)]
    enum CurrencyMap {
        Specific {
            #[serde(rename = "1inch")]
            one_inch: f64,
            aave: f64,
            ada: f64,
        },
        Partial {
            ada: f64,
        },
        Other {
            env: f64,
        },
    }

    #[test]
    fn test_serde() {
        let idr_json = r#"{
       "date": "2025-01-26",
       "idr": {
           "ada": 0.000062981114
       }
   }"#;

        let usd_json = r#"{
       "date": "2025-01-26",
       "usd": {
           "env": 0.22
       }
   }"#;

        let full_json = r#"{
       "date": "2025-01-26",
       "usd": {
           "1inch": 0.00019578941,
           "aave": 1.8799505e-7,
           "ada": 0.000062981114
       }
   }"#;

        let idr_rates: CurrencyRates = serde_json::from_str(idr_json).unwrap();
        let usd_rates: CurrencyRates = serde_json::from_str(usd_json).unwrap();
        let full_rates: CurrencyRates = serde_json::from_str(full_json).unwrap();

        println!("IDR Rates: {:?}", idr_rates);
        println!("USD Rates: {:?}", usd_rates);
        println!("Full Rates: {:?}", full_rates);

        // Serialize back to JSON
        let idr_serialized = serde_json::to_string_pretty(&idr_rates).unwrap();
        let usd_serialized = serde_json::to_string_pretty(&usd_rates).unwrap();

        println!("Serialized IDR Rates:\n{}", idr_serialized);
        println!("Serialized USD Rates:\n{}", usd_serialized);
    }
}

With error:

thread 'forex_impl::exchange_api::serde_tests::test_serde' panicked at src/forex_impl/exchange_api.rs:110:71:
called `Result::unwrap()` on an `Err` value: Error("data did not match any variant of untagged enum Rates", line: 6, column: 4)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test forex_impl::exchange_api::serde_tests::test_serde ... FAILED

failures:

failures:
    forex_impl::exchange_api::serde_tests::test_serde

Here is the toml:

[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"

[dev-dependencies]
serde_json = "1"
serde = { version = "1", features = ["derive"] }

Cargo version:

$ cargo --version
cargo 1.83.0 (5ffbef321 2024-10-29)

Why does this erring for the exact same code?

What versions of serde and serde_json are in use? (Read Cargo.lock or run the cargo tree command.) Perhaps a old one with a bug is being used?

However, you don't need to use untagged here. This should get the same result, but be more efficient and have better errors:

    #[derive(Debug, Serialize, Deserialize)]
    enum Rates {
        #[serde(rename = "idr")]
        Idr(CurrencyMap),
        #[serde(rename = "usd")]
        Usd(CurrencyMap),
    }

serde(untagged) should be avoided whenever possible, because it has poor error messages and it has to try every variant, which can be slow and also has to allocate temporary storage for all the data until it finds the right variant. It's much better to find a way to have a tag.

I've removed the untagged, still the same error on local test.

serde version:

$ cargo tree | rg "serde"
β”‚       β”œβ”€β”€ serde v1.0.217
β”‚       β”‚   └── serde_derive v1.0.217 (proc-macro)
β”‚       └── serde_json v1.0.137
β”‚           └── serde v1.0.217 (*)
β”‚   β”œβ”€β”€ serde v1.0.217 (*)
β”‚   β”œβ”€β”€ serde_json v1.0.137 (*)
β”‚   β”œβ”€β”€ serde_path_to_error v0.1.16
β”‚   β”‚   └── serde v1.0.217 (*)
β”‚   β”œβ”€β”€ serde_urlencoded v0.7.1
β”‚   β”‚   └── serde v1.0.217 (*)
β”‚   └── serde v1.0.217 (*)
β”‚   β”œβ”€β”€ serde v1.0.217 (*)
β”‚   β”œβ”€β”€ serde_json v1.0.137 (*)
β”‚   β”œβ”€β”€ serde_yaml v0.9.34+deprecated
β”‚   β”‚   β”œβ”€β”€ serde v1.0.217 (*)
β”‚       β”œβ”€β”€ serde v1.0.217 (*)
β”‚       β”œβ”€β”€ serde_spanned v0.6.8
β”‚       β”‚   └── serde v1.0.217 (*)
β”‚       β”‚   └── serde v1.0.217 (*)
β”‚           β”œβ”€β”€ serde v1.0.217 (*)
β”‚           β”œβ”€β”€ serde_spanned v0.6.8 (*)
β”‚   β”œβ”€β”€ serde v1.0.217 (*)
β”‚   β”œβ”€β”€ serde_json v1.0.137 (*)
β”‚   β”œβ”€β”€ serde_urlencoded v0.7.1 (*)
β”œβ”€β”€ serde v1.0.217 (*)
β”œβ”€β”€ serde_json v1.0.137 (*)
    └── serde v1.0.217 (*)
β”œβ”€β”€ serde v1.0.217 (*)
└── serde_json v1.0.137 (*)

Which versions are the buggy ones? and which version I should use?

using HashMap results in same error in local test, but run successfully on rust playground:

    #[derive(Debug, Serialize, Deserialize)]
    enum Rates {
        #[serde(rename = "idr")]
        Idr(HashMap<String, f64>),
        #[serde(rename = "usd")]
        Usd(HashMap<String, f64>),
    }

That was just speculation about what the problem could be. You are using the latest versions.

There must be something about your project that is different from running the same code on the playground, but we don't have enough information to say what that difference is. We need more code with which to reproduce the problem.

Make sure that your source code doesn’t contain any surprising invisible characters (e.g. zero-width spaces, left-to-right markers; subtle diacritics; variant selectors, and whatever else unicode might have to offer).

If any of the keys in the json are wrong, you get that error. (Without much additional context.) So invisible stuff would be quite subtle.

I suppose, a good way of checking this is to debug-format out the string literals. In a test, printing is a little tricky, so you could try with asserts like

    let idr_json = r#"{
       "date": "2025-01-26",
       "idr": {
           "ada": 0.000062981114
       }
   }"#;

    assert_eq!(format!("{idr_json:?}"), "\"{\\n       \\\"date\\\": \\\"2025-01-26\\\",\\n       \\\"idr\\\": {\\n           \\\"ada\\\": 0.000062981114\\n       }\\n   }\"");

    let usd_json = r#"{
       "date": "2025-01-26",
       "usd": {
           "env": 0.22
       }
   }"#;

    assert_eq!(format!("{usd_json:?}"), "\"{\\n       \\\"date\\\": \\\"2025-01-26\\\",\\n       \\\"usd\\\": {\\n           \\\"env\\\": 0.22\\n       }\\n   }\"");

Another possible approach to making sure the strings aren't "bad" might be this:

Perhaps try to copy back the input the other direction, into your local text file. Or do a copy-paste of the output of the re-serialized text for "Serialized IDR Rates" and "Serialized USD Rates", back into your source file for the definition of idr_json and usd_json.

Also… when you say "same error" with that change, do you mean literally the same error? Did you make sure that you actually save all changes and re-built the tests? E.g. double-check by testing that if you introduce a deliberate typo e.g. derive(Debug becoming derive(Debugg, you get the corresponding compilation error. Or better perhaps write a new panic!("some message") at the beginning of this test to see if that fires.

Out of interest: Are you running the tests from the terminal or an ide?

Turn out it was because of feature from a crate I used

rust_decimal = { version = "1.36", features = ["serde-arbitrary-precision"] }

That feature serde-arbitrary-precision cause it to fail. Im not sure why. This is from the docs:

Serialize/deserialize Decimals as arbitrary precision numbers in JSON using the arbitrary_precision feature within serde_json .

Removing that feature works fine.

1 Like

Let's see… it's a synonym for a serde-with-arbitrary-precision feature… which implies…:

Enables arbitrary_precision and std of serde_json.

Given serde_json is part of this code, and its arbitrary_precision feature isn’t enabled by default, it seems likely (to me) that is how this can have any effect.

Oh the confusion… I was about to post that what a coincidence someone had just submitted an issue about this slightly earlier today

Then I was confused why GitHub wasn't displaying "x hours ago" and the date instead.

My brain clearly hasn’t entered 2025 yet.

…but β€œwhat a coincidence, someone else has reported this issue exactly 1 year ago” still works, right?

6 Likes