Funny error message when trying to return Iterator

1

I have a type (Series) that hosts data of AnyValue, an enum with a long list of types that include numeric, string and others (see polars crate if curious). I have a function that is "generic" over a subset of the AnyValue variants, values that behave as numbers.

I created an enum type AnyNumber. When I try to write a function that returns an impl Iterator I get an error that seems to treat my values as types: "Expected struct UInt8Type, found struct UInt16 type"

fn numbers(input: &Series) -> Result<impl Iterator<Item = AnyNumber>> {
    match input.dtype() {
        DataType::UInt8 => Ok(input.u8()?.into_no_null_iter().map(AnyNumber::UInt8)),
        DataType::UInt16 => Ok(input.u16()?.into_no_null_iter().map(AnyNumber::UInt16)),
        DataType::UInt32 => Ok(input.u32()?.into_no_null_iter().map(AnyNumber::UInt32)),
        ...
        DataType::Float64 => Ok(input.f64()?.into_no_null_iter().map(AnyNumber::Float64)),
        _ => Err(eyre!(
            "The underlying type is not a AnyNumber: {}",
            input.dtype()
        )),
    }
}

The error:

error[E0308]: mismatched types
   --> /Users/edmund/Programming-Local/tnc-analysis/lib/src/matrix.rs:216:36
    |
216 |               DataType::UInt16 => Ok(input.u16()?.into_no_null_iter().map(AnyNumber::UInt16)),
    |                                   -- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected struct `UInt8Type`, found struct `UInt16Type`
    |                                   |
    |                                   arguments to this enum variant are incorrect

 ::: /Users/edmund/Programming-Local/tnc-analysis/polars/polars/polars-core/src/chunked_array/mod.rs:478:10
    |
478 |       ) -> impl Iterator<Item = T::Native>
    |  __________-
479 | |            + '_
480 | |            + Send
481 | |            + Sync
482 | |            + ExactSizeIterator
483 | |            + DoubleEndedIterator
484 | |            + TrustedLen {
    | |                       -
    | |                       |
    | |_______________________the expected opaque type
    |                         the found opaque type
    |
    = note: expected struct `Map<impl Iterator<Item = ...> + Send + Sync + ExactSizeIterator + DoubleEndedIterator + TrustedLen + '_, ...>` (struct `UInt8Type`)
            the full type name has been written to '/Users/edmund/Programming-Local/tnc-analysis/bin/target/release/deps/tnc_analysis_lib-a8155906914e5ad5.long-type-15334500443020542455.txt'
               found struct `Map<impl Iterator<Item = ...> + Send + Sync + ExactSizeIterator + DoubleEndedIterator + TrustedLen + '_, ...>` (struct `UInt16Type`)
            the full type name has been written to '/Users/edmund/Programming-Local/tnc-analysis/bin/target/release/deps/tnc_analysis_lib-a8155906914e5ad5.long-type-5313474775209893471.txt'
help: the type constructed contains `std::iter::Map<impl Iterator<Item = <UInt16Type as PolarsNumericType>::Native> + Send + Sync + ExactSizeIterator + DoubleEndedIterator + polars_arrow::trusted_len::TrustedLen + '_, fn(u16) -> AnyNumber {AnyNumber::UInt16}>` due to the type of the argument passed
   --> /Users/edmund/Programming-Local/tnc-analysis/lib/src/matrix.rs:216:33
    |
216 |             DataType::UInt16 => Ok(input.u16()?.into_no_null_iter().map(AnyNumber::UInt16)),
    |                                 ^^^-------------------------------------------------------^
    |                                    |
    |                                    this argument influences the type of `Ok`
note: tuple variant defined here
   --> /Users/edmund/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/result.rs:507:5
    |
507 |     Ok(#[stable(feature = "rust1", since = "1.0.0")] T),
    |     ^^

The into_no_null_iter() has the following type signature:

pub fn into_no_null_iter(
    &self
) -> impl Iterator<Item = <T as PolarsNumericType>::Native> + Send + Sync + ExactSizeIterator + DoubleEndedIterator + TrustedLen

Why might the compiler focus on a partial expression of the iterator i.e., ignoring the map expression that unifies the type? Is there a workaround other than using collect()?

Thank you!

2

It doesn't. Map must be generic over the mapping function, because it contains the mapping function. Therefore, your .map()s taking closures of different argument types are of different types.

For returning opaque heterogeneous types, the usual way to go is a trait object, i.e., Box<dyn Iterator>.

2 Likes
3

Sometimes you can unify Map types by making them contain function pointers instead of function items:

.map(AnyNumber::UInt8 as fn(u8) -> AnyNumber)

However, that doesn't help here because the input into_no_null_iter() iterator also changes its type and its Item type. It would only help if everything else in the chain was identical in type and only the behavior of the mapping function was being changed.

1 Like
4

Yep, and technically, Map could also contain a &dyn FnMut in order to avoid being generic over the mapping function, but realistically, always forcing dynamic dispatch on a core primitive would go very much against the principles of the language.

5

I was focused on the return type. The key was different argument types. Per usual, great catch. The memory required to execute each of the map functions is different despite the enumerated return value!...