GAT Iterator with mapping function

Hello.
I've got a lifetime/soundness question, hoping some Rust expert can help a newbie !

Rust playground - API skeleton only, function impl does not matter here

I'm trying to use GAT with a Streaming/Lending iterator to test some real world use cases I have - I need to iterate a file without allocating a buffer for each line/chunk.

I have an iterator trait that makes use of GAT and it's based on the rfc doc, very basic, it does work fine.
Now I'm trying to add a simple 'map' function to the trait hoping to be able to wrap the buffer reference in some arbitrary struct, this wrapper type would be used only within the lifetime of the 'next' call just like the buffer ref would.

I know there are simple ways of getting around this(like just get the buffer reference inside the loop like you would normally would and put it inside a struct), but I'm trying to improve my Rust skills.

A more simple way of putting it:
I want to create an iterator that allocates a byte buffer only once, iterate over the lines and wrap each line into another type where the type will be only valid for the duration of the 'next' call and must be done using a 'map' function.

Appreciate any advice / help here as I'm stuck on this for weeks and it's making me frustrated.

#![feature(generic_associated_types)]

trait FnMut1Arg<A>: FnMut(A) -> <Self as FnMut1Arg<A>>::Output {
    type Output;
}
impl<F: ?Sized, A, O> FnMut1Arg<A> for F
where
    F: FnMut(A) -> O,
{
    type Output = O;
}

fn main() {
    let iterator = FileIt;

    // doesn’t work because type inference for closures is dumb…
    // let mut wrapped_item = iterator.map(|e| Wrapper { e });

    // let’s help inference out
    fn help_inference_out<F: FnMut(&[u8]) -> Wrapper<'_, [u8]>>(f: F) -> F {
        f
    }
    let mut wrapped_item = iterator.map(help_inference_out(|e| Wrapper { e }));
    while let Some(item) = wrapped_item.next() {
        println!("{:?}", item);
    }
}

#[derive(Debug)]
struct Wrapper<'a, T: ?Sized> {
    e: &'a T,
}

struct FileIt;

impl GatIterator for FileIt {
    type Item<'n>
    where
        Self: 'n,
    = &'n [u8];

    fn next(&mut self) -> Option<Self::Item<'_>> {
        todo!()
    }
}

trait GatIterator {
    type Item<'n>
    where
        Self: 'n;

    fn next(&mut self) -> Option<Self::Item<'_>>;

    fn map<F>(self, f: F) -> Map<Self, F>
    where
        Self: Sized,
        for<'n> F: FnMut1Arg<Self::Item<'n>>,
    {
        Map { it: self, f }
    }
}

#[derive(Debug)]
pub struct Map<I, F> {
    it: I,
    f: F,
}

impl<I, F> GatIterator for Map<I, F>
where
    I: GatIterator,
    for<'n> F: FnMut1Arg<I::Item<'n>>,
{
    type Item<'n>
    where
        Self: 'n,
    = <F as FnMut1Arg<I::Item<'n>>>::Output;

    fn next(&mut self) -> Option<Self::Item<'_>> {
        self.it.next().map(&mut self.f)
    }
}
2 Likes

As a bonus, here’s the same thing without using #![feature(generic_associated_types)]

trait FnMut1Arg<A>: FnMut(A) -> <Self as FnMut1Arg<A>>::Output {
    type Output;
}
impl<F: ?Sized, A, O> FnMut1Arg<A> for F
where
    F: FnMut(A) -> O,
{
    type Output = O;
}

fn main() {
    let iterator = FileIt;

    // doesn’t work because type inference for closures is dumb…
    // let mut wrapped_item = iterator.map(|e| Wrapper { e });

    // let’s help inference out
    fn help_inference_out<F: FnMut(&[u8]) -> Wrapper<'_, [u8]>>(f: F) -> F {
        f
    }
    let mut wrapped_item = iterator.map(help_inference_out(|e| Wrapper { e }));
    while let Some(item) = wrapped_item.next() {
        println!("{:?}", item);
    }
}

#[derive(Debug)]
struct Wrapper<'a, T: ?Sized> {
    e: &'a T,
}

struct FileIt;

impl<'n> HasItem<'n> for FileIt {
    type Item = &'n [u8];
}
impl GatIterator for FileIt {
    fn next(&mut self) -> Option<Item<'_, Self>> {
        todo!()
    }
}

trait HasItem<'a, __ = &'a Self> {
    type Item;
}
type Item<'a, This> = <This as HasItem<'a>>::Item;
trait GatIterator: for<'n> HasItem<'n> {
    fn next(&mut self) -> Option<Item<'_, Self>>;

    fn map<F>(self, f: F) -> Map<Self, F>
    where
        Self: Sized,
        for<'n> F: FnMut1Arg<Item<'n, Self>>,
    {
        Map { it: self, f }
    }
}

#[derive(Debug)]
pub struct Map<I, F> {
    it: I,
    f: F,
}

impl<'n, I, F> HasItem<'n> for Map<I, F>
where
    I: GatIterator,
    F: FnMut1Arg<Item<'n, I>>,
{
    type Item = <F as FnMut1Arg<Item<'n, I>>>::Output;
}

impl<I, F> GatIterator for Map<I, F>
where
    I: GatIterator,
    for<'n> F: FnMut1Arg<Item<'n, I>>,
{
    fn next(&mut self) -> Option<Item<'_, Self>> {
        self.it.next().map(&mut self.f)
    }
}
2 Likes

Thank you very much @steffahn ! I appreciate your help.
Do you know any resources I can read on this subject ? where I can understand the details on why this happens and the places I might stumble on it again ?

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.