Having trouble trying to return an iterator from two functions that take two different inputs to create the iterator from

help
#1

I have a seemingly simple problem.

I have two functions in my module, from both of which I want to return an iterator called Tokens:

fn do_from_text<'a>(text: &'a str) -> Tokens<'a> {
    Tokens { text }
}

// This one accepts a file path. I will read the file content, possibly do some processing on it and get the text which will be what I'll give to Tokens.
fn do_from_path<'a>(path: &str) { ... }

struct Tokens<'a> {
    text: &'a str,
}

// Implement iterator for Tokens here.

My problem is with the do_from_path function. Here's what I intend to do:

fn do_from_path<'a>(path: &str) -> Tokens<'a> {
    let text = fs::read_to_string(path).unwrap();
    do_from_text(&text)
}

This obviously doesn't work, and I have no problem understanding why. But:

  • I don't want to have to clone the text (so that I can have a text: String in Tokens). This string might be very large and I might have to call this many times.
  • I don't want to create multiple Tokens struct (one that accepts a slice and another that owns a string), I want just the one.
  • And most importantly, I don't want to make it so that the consumer has to do the reading logic of the file (in which case this would be solved because the consumer would own the string and they could just give me a reference to it, so returning the Tokens iterator would be no problem as well).

It feels like a pattern that should have a solution but I can't figure it out. Am I out of luck here or is there a solution that satisfies my goals above?

#2

If you want to store either a borrowed str or an owned String, you probably want a Cow<str>:

use std::borrow::Cow;

fn main() {
    let borrowed = Tokens { text: Cow::Borrowed("text")};
    let owned = Tokens { text: Cow::Owned("text".to_string()) };
    
}

struct Tokens<'a> {
    text: Cow<'a, str>
}

Another option might be to make Tokens generic like so:

fn main() {
    let borrowed = Tokens { text: "text" };
    let owned = Tokens { text: "text".to_string() };
    
}

struct Tokens<S: AsRef<str>> {
    text: S
}
2 Likes
#3

Oh this makes sense! I can't believe I haven't thought of AsRef, but I'm fairly new to rust so I only had shallow knowledge about Cow. Cow seems to be exactly what I want here. Thanks!

1 Like