Troubles creating generic function interface for iterables & their references


#1

Hi. I’m trying to implement join filter in templating library called askama. The library compiles templates into Rust code during compile-time, so there are bits of codegen with syn involved. Given that, the function that represents join filter should be able to accept different types of iterables and references to them. The problem I’m having can be seen by running this gist in playground. Because of codegen we can’t avoid double ref (&&slice). Is there a way to get around this while keeping the code on 27th line as much unchanged as possible?

P.S someone on IRC suggested this, but it doesn’t play well with type inference, has Copy it in (that can probably be avoided?) and therefore doesn’t suit our needs.


#2

To state this problem in a little more detail: we want to accept all things that can iterate over a Display items, so we figured that we could type that as IntoIterator<Item = std::fmt::Display>. This seems to accept something like &["foo", "bar"] just fine, as expected, but it rejects &&["foo", "bar"]. This is surprising because rustc generally seems to try dereferencing as part of its coercion strategy.

Indeed, I found this StackOverflow answer that seems to suggest the rule is something like “dereference as many times as possible, then reference as most once”. So I’m at a loss why this is not dereferencing the double reference.

In the past when I ran into problems similar to this, it often turned out that there was some subtle mismatch in the expected and actual type that made the inferencer get it wrong. Can’t figure out what it is, though! Might even be a compiler bug?


#3

How much flexibility do you have around the callsites to join? Are those part of codegen as well? You can force deref chain to peel off references by calling a method that triggers it (i.e. the “dot” operator triggers it). For instance: https://play.rust-lang.org/?gist=0a87ffc4f4d09f080f4586a701ce9a8d&version=stable


#4

I’m certain this is a fairly well-known limitation of generics. Here’s a simple example of the issue: https://play.rust-lang.org/?gist=41c02c542afb3a2d5cb2799cf30f2eea&version=stable

fn foo<T, F: Fn(T)>(x: T, f: F) {
  f(x);
}

fn g(x: &i32){}


fn main() {
  g(&&3i32); // ok
  foo(&&3i32, g); // error
}

My understanding is that:

  • deref coercion works on expressions, between the types available at that location in the AST.
  • while rust does seem to capable of dispatching on the argument type of a generic Fn parameter (e.g. fn foo<T, F:FnMut(T)>(f: F) {} compiles and can be used… somehow), I think it generally prefers not to, since the arguments to F are a type parameter of the trait.

This also makes for a crucial difference between, say, .map(|x| x) and .map(identity) (where identity is the appropriately-defined generic function). In the former, x explicitly appears as an expression, making deref coercion possible. The latter is pretty much guaranteed to do nothing.


#5

I think another way to put it is auto deref does not occur at the type system level.


#6

Thanks folks!
I managed to implement what @vitalyd suggested, we’ll probably go with that.