Accepting Iterator<T> in addition to &[T]

help
1

Playground Link: Rust Playground

I have existing code:

fn transmit<T: Into<u32> + Copy>(self, data: &[T]) -> SingleShotTxTransaction<Self, T>
    where
        Self: Sized,

used very roughly like

for v in data {
  write((*v).into())
}

I'd like to change it to accept an iterator, so I can call it like ch.transmit(data.flat_map(...)). The iterator can't return &T because the iterator is generating values on the fly in the map. I can't just make the data be Into<u32> - it isn't a 1:1 mapping.

I can change the signature to:

fn transmit<D, R, T: Into<u32> + Copy>(self, data: D) -> SingleShotTxTransaction<Self, R, T>
    where
        Self: Sized,
        D: IntoIterator<Item = T, IntoIter = R>,
        R: Iterator<Item = T>,

used like

for v in data {
  write(v.into())
}

and change existing callers from channel.transmit(&buffer) to channel.transmit(buffer.iter().copied()) (and then IntoInterator isn't really needed)

But I'd like to do this without needing existing callers to change. (That's why I had the IntoIterator above) I wonder if it is possible, given the constraint that my iterator can't return a reference?

I thought core::borrow:Borrow might help, eg

    fn transmit<D, I, B, T: Into<u32> + Copy>(self, data: D) -> SingleShotTxTransaction<Self, I, B, T>
    where
        Self: Sized,
        B: Borrow<T>,
        D: IntoIterator<Item = B, IntoIter = I>,
        I: Iterator<Item = B>,

used like

for v in data {
  write((*v.borrowed()).into())
}

And calling that with the an Iterator<u32> works fine, but calling with &data (eg existing code) won't compile:

   Compiling esp-hal-smartled v0.12.0 (/home/ted/src/esp-hal/esp-hal-smartled)
error[E0283]: type annotations needed
    --> /home/ted/src/esp-hal/esp-hal-smartled/src/lib.rs:191:23
     |
191  |         match channel.transmit(&self.rmt_buffer).wait() {
     |                       ^^^^^^^^ cannot infer type of the type parameter `T` declared on the method `transmit`
     |
     = note: cannot satisfy `_: Into<u32>`
note: required by a bound in `transmit`
    --> /home/ted/src/esp-hal/esp-hal/src/rmt.rs:1005:29
     |
1005 |     fn transmit<D, I, B, T: Into<u32> + Copy>(self, data: D) -> SingleShotTxTransaction<Self, I, B, T>
     |                             ^^^^^^^^^ required by this bound in `TxChannel::transmit`
help: consider specifying the generic arguments
     |
191  |         match channel.transmit::<&[u32; BUFFER_SIZE], core::slice::Iter<'_, u32>, &u32, T>(&self.rmt_buffer).wait() {
     |                               ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Why can't it infer T calling it this way? Is there any way that would work? I'm guessing I should be expressing the types differently, but I'm not sure how.

2

You're making it more complicated than it needs to be.

fn foo<I>(iter: I)
where
    I: IntoIterator<Item = i32>,
{
}

That will accept any kind of iterator yielding i32s.

5 Likes
3

If I remove the Into<>, eg:

    fn transmit<D, I, B>(self, data: D) -> SingleShotTxTransaction<Self, I, B, u32>
    where
        Self: Sized,
        B: Borrow<u32>,
        D: IntoIterator<Item = B, IntoIter = I>,
        I: Iterator<Item = B>,

Then I can compile with both &[u32] and Iterator<u32>, but other existing code that relies on the &[Into<u32>] conversion fails.

The Into<u32> allows a degree of on-the-fly mapping, but only works for 1:1 mapping, and the new need is mapping that isn't 1:1 and also avoiding a buffer. Avoiding the buffer is important because it will need ~200kB on a microcontroller that only has ~400kB total.

If I need to break one case, I'd probably break callers relying on Into<>, but I feel I should be able to do this without breaking any existing callers.

4

If I'm writing new code, you're right. I'm making it too complicated.

I'm trying to figure out how to not break existing callers. And maybe that's not possible, but I'd like to try. IntoInterator<Item=u32> doesn't work though I need &[u32], and specifically &[Into<u32>]

I should have given an example you can play with. Sorry about that. Here's a Rust Playground

I want revised to replace existing allowing the new case of an iterator input. And I'd like existing code to not change, so the two calls to existing should also work with revised.

5

Because Borrow is generic. A type can implement Borrow<T> for many choices of T, all of which may be Into<u32> + Copy. There's no way for the compiler to know which one you meant.

Fundamentally, your problem is that you want both a specific and a generic impl; that generally isn't possible due to a potential overlap.

6

From what I understand you now have two cases: former &[T] and newer IntoIterator<Item = T>. First one implements iterator which gives out &T, but new ones cannot be restricted to giving out &T as those are generated values.

So the problem is not wanting specific and generic impl, it is wanting to work with both T and &T. How about adding new trait which is implemented on &U where U: Into<u32> + Copy and u32:

fn transmit1<T: Into<u32> + Copy>(_: &[T]) -> ()
{
    todo!()
}

pub fn do1() {
    transmit1(&[32u32]);
    transmit1(&[32u16]);
}

trait Convert {
    fn convert(self) -> u32;
}

impl<'t, T: Into<u32> + Copy> Convert for &'t T {
    fn convert(self) -> u32 {
        (*self).into()
    }
}

impl Convert for u32 {
    fn convert(self) -> u32 {
        self
    }
}

fn transmit2<I, T: Convert>(_: I) -> ()
    where I: IntoIterator<Item = T>
{
    todo!()
}

pub fn do2() {
    transmit2(&[32u32]);
    transmit2(&[32u16]);
    transmit2((0u32..2).map(|v| v * 2u32));
}

playground

(Sorry, I have created playground before you have given your own.)

I have not written any implementing code for transmit as this is a problem “how do I write code which typechecks” and there is no need to have anything, except function signatures in this case.

2 Likes
7

But yet it can infer that when I pass it an iterator, or even a slice, just not a reference to a slice in the Rust Playground - it only needs the hints when given a reference to an array.

I don't understand what is so different borrow.rs - source looks pretty similar for T vs &T

@ZyX-II has a solution with a different indirection that looks like it solves the problem.

It feels like perhaps there is a limit of the number of layers of generics that the compiler can derive?

8

You can imagine it going something like this.

  • revised([0_u32])

    • What's I with obligation I: IntoIterator<Item = B> ?
      • It's [u32; 1] with new obligation B = u32
    • So assume B = i32
    • What's T with obligation u32: Borrow<T>
      • There's only one solution in this crate, T = u32
    • Use this solution (even though it could become ambiguous in the future if more solutions are introduced)

  • revised(&[0_u32])

    • What's I with obligation I: IntoIterator<Item = B> ?
      • It's &'_0 [u32; 1] with new obligation B = &'_0 u32
    • So assume B = &'_0 u32
    • What's T with obligation &'_0 u32: Borrow<T>
      • There are at least two solutions for T, &'_0 u32 or u32
    • Give up: it's ambiguous

  • revised([&0_u32])

    • Very similar to the last case (requires turbofish)

(Note that in your turbofish, you need only specify T.)

You're wishing it would soldier on and reject &'_0 u32 due to failing &'_0 u32: Into<u32>, but it doesn't. (There's probably a number of reasons such as "too slow" and "too fragile".)

1 Like
9

A non-reference type T only implements Borrow<T>, so the choice is ubambiguous, there's only one possible impl that applies.

A reference &U impls Borrow<U>, but of course, the blanket impls applies to references, too – they also impl Borrow<&U>.

10

Note that this has a drawback of either making you write a bunch of implementations of Convert or making whoever writes those iterators do something like adding .map(Into::<u32>::into): you cannot have blanket implementation for T and any other implementation, including &T as it is possible for the implementation for T apply to its reference. (Maybe you could in the future with specialization, but I would not be optimistic about specifically T and &T. You definitely cannot do that with nightly and #![feature([min_]specialization)] now.)

I have chosen specifically this solution because with the problem as described you do not break existing callers in this case and writing new implementation for any concrete type T: Into<u32> if you need that for convenience is as easy as writing impl Convert for ConcreteT { fn convert(self) -> u32 { self.into() } } which can be done with a macro.

(BTW, link to the playground was fixed.)

11

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.