Will 'impl Future' replace return types of combinators, like map and and_then?


#1

I have noticed that map for the future returns Map object, the similar stuff is going on for and_then, and other combinators. As a result of long chain of futures you end up with something like this:

And this is only 2 combinator functions applied in the chain. I understand that it is Intelij specific, but I find these type hints very-very helpful, and decoding the type of a variable without these hints would be even more problematic.

However, my brain is too much overloaded when I need to know what Map, AndThen and so on mean and what they also implement, etc… And all of this is only for knowing that full_body is really of a type equivalent to Future<String, Error>.

Why do I need to know that map returns Map<...> but not Future<Result, Error>? Is it because returning the Box requires an allocation?

Will impl Future feature fix it?
Or alternatively, would not it be better to design all future combinators to return Box<Future<...>> instead and have preallocated, expandable memory pool for frequent operations with boxed futures?


What does Rust need today for server workloads?
What does Rust need today for server workloads?
#2

I am also curious whether impl Future will replace the many nested variants of Future with a singular type.


#3

Allocation is one problem. The second problem is loss of monomorphization, which translates to worse performance (irrespective of the allocation part).

impl Trait feature will help, though existing APIs will likely maintain their concrete-type signatures for backwards compatibility. But new code will use it.


#4

The best way to understand more what impl Trait is capable of is to try it. e.g. on https://play.rust-lang.org/

A Trait is not a data type. A limitation of the function stack is variables (inc args and return) have to be a fixed size known by the compiler. When you do &Trait or Box you tell the compiler to make a data type, (goes by names Trait object or fat pointer,) consisting internally two pointers, to the structure and to a vtable.

Box already fixes it so as to hide the internal details. impl Trait allows the compiler to optimize which it can’t do with Box.

impl Trait is not a single type. (Same only for same function.) Think of it like the compiler generating for each function;

struct AbstractAnonymousAPI(InternalStructureImplTrait);
impl Trait for AbstractAnonymousAPI {...}

There will be backwards compatibility but maybe a chance that the proposed addition of “Epoc” will allow API change.


#5

I understand this. It is also valuable for a programmer to know that certain variable of known size, but this information should not be delivered to a programmer through the deep nested chain of types, like in the example I provided. Although, the compiler should keep tracking of the full necessary metadata to make it optimizations.

This is good to know some internals behind, but these should not be exposed beyond of the function signature. Consider the following code:

fn get_forbes_quote_of_a_day() -> impl Future<Item=String, Error=Error>> 
...
let quote = get_forbes_quote_of_a_day()

What should I, as a programmer, know about quote variable?

  • That it is of type Future<String, Error> + Sized. If it is more than this, it becomes unnecessary details, which will distract my brain from other more important stuff in the code.

What compiler should know about quote variable?

  • As much as it needs to know in order to make all possible optimizations.

I hope impl Future will deliver on the above expectations?

Next, consider the following follow up on the code above sample:

let quote_length = quote.map(|r| { r.len() })

What should I, as a programmer, know about quote_length variable?

  • That it is of type Future<usize, Error> + Sized. That is it. Again, if it is more than this, it becomes unnecessary details, like it is now Map<Future<String,Error>, fn(String) -> usize>>.

What compiler should know about quote_length variable?

  • Again, as much as it needs to know in order to make all possible optimizations.

The delivery on the above expectations is not possible if the signature of the map function is not changed to something like the following:

fn map<F, U>(self, f: F) -> impl Future<U>
        where F: FnOnce(Self::Item) -> U, Self: Sized;

Hence, it was my question: will it actually change after impl Future is stabilized?


#6

That’s kind of sad seeing that all futures-related crates are at version 0.something.


#7

Exactly! Go and scala had futures designed really well, which helped to their success to some extent. Hopefully, rust futures will improve.


#8

There’s going to be one aspect with impl Trait that I suspect you’ll still find clunky - only a single type can be returned. So if you have, say, a match or if/else arms that return closures, it won’t compile.

I’m also unsure whether compiler error messages will really be any better - there’re still concrete types being layered (when applying a series of combinators), they’re just anonymous/unnamed.

Biggest boon for impl Trait is it’ll allow skipping boxing for cases where you must do it today - so performance should be better. There’s also the benefit of not having dozens of (public) simple named structs floating around. But from a “type soup” perspective, I’m not sure it’ll change anything.


#9

And this can be leveraged by the map, and_then and other combinator functions exactly for the reason I highlighted initially. If it does not happen, well… it would be sad.


#10

You’re preaching to the choir - I think everyone is in agreement with why impl Trait is good - heck, that’s the whole reason the feature is being implemented :slight_smile:. But the Rust community takes a strong stance on backwards compatibility so rolling this out to existing code is tricky. I’m sure all new code will use it without thinking twice.


#11

This is great. However, futures crate is so much fundamental for everything else built on top. This non-backward compatible change would worth the benefit (see my original point in this thread). Also as it was mentioned before by @bestouff , it is at versions 0.1.x, so kind of expected that non-backward compatibility changes are possible. E.g. hyper and reqwest are going to move it’s main definitions to http crate, when hyper is already on the version 0.11.x. Although I am not sure if it will break backward compatibility, but I guess it will.

Anyway, whatever it will be. I wished to provide my feedback and highlight things which could be done better. I am evaluating feasibility of Rust for the development of mission (life) critical web API services, and issues like this are not adding pluses to enable me to obtain “easy buy-in” from everybody…


#12

My honest advice for now if you haven’t done so already is to bind a key to toggle the type hints.

In CLion, the option shows up as Toggle parameter name hints under Keybindings. It toggles both the types and parameter names. Personally, I keep them off by default since they mess with alignment and my sense of code width, but they are definitely one of IntelliJ Rust’s killer features, barring cases like the sample you posted.


#13

The active hints in the case like this are also beneficial, because they highlight the things which might require improvements in libraries and apis. This one definitely does require. Disabling hints would make it even harder to figure out the type.


#14

It is my hope as well that the futures crate will adapt impl Trait.

Could there even be some possibility to expose both APIs in some automated way?


#15

I did not know you can do that. Awesome!


#16

Unfortunately, the “necessary metadata”/“as much as necessary” exactly is the concrete, nested, loooong type. Nothing less will do.
Believe me, the rust designers are minimal in the extreme (just look at the std lib, tiny!). If they could have gotten away with less info, they would have!

Currently the concrete type must be explicit, because there is nowhere else to put it. impl Trait will hide it behind the anonymous impl Future type, like @jonh explained.
I’m not sure I understand why this is not exactly what you need. Can you elaborate?


@vitalyd: you seem to be better versed in the impl trait discussion than I am. I also detect some hints of language barriers and not understanding eachother.
Could you clarify what you mean with “existing API will likely maintain their … signatures”?
If I understand correctly, you mean that the existing, “old” API will stay available as is, for backwards compatibility. However, that shouldn’t hinder a second API being built next to it right? Something like “futures 2.0” or a wrapper crate like futures_but_with_impl_returntypes.
That “new” API would then be more what @avkonst is looking for. Correct?


Please keep in mind that Rust is very young for a programming language. Only 2,5 years since 1.0! (comparable to C++ in 1987 or Java in 1998, when it was at J2SE 1.2)
This talk about “life critical”, and your other post in the “server workloads” topic indicate you are looking for a very mature platform, since “[HTTP server] at the level of Jetty or Vert.x” is apparently an essential requirement.

I believe we can take it as a compliment that you are considering our young language at all for apparent life-or-death situations. As you have discovered, the core rust language is already pretty awesome, and I hope that you can keep in mind that much of the ecosystem is still being built. (Much the same as for C++ in 1987, and just look at how much better java has gotten since 1.2, when Jetty was at 3.x, which is currently considered “fossilized”).
We may simply not ready for your (very demanding!) use-case today, but please stay around to see what we’ll have built by tomorrow/next year! The pace is incredible! :racing_car: :smiley:

You bet they will! Thanks for providing some input into the discussion on making that happen! :smiley:


#17

How about opening an issue for this ?


#18

Correct.

Note that I’m expressing my opinion only - I don’t know what the various lib maintainers intend to do.


#19

Thank you for your answer. I understand that. Compiler needs to track all of this, a programmer might not need to. That was my initial point.

That is great! One of the things I was looking into is attitude and “culture”. So, far I like it very much. Reminds me a lot of what I have seen in scala community.

Great! I understood this, and asked if futures are going to leverage it.

This is exactly what I would like to see adopted on at least fundamental libraries like Futures and libraries built on top, like Hyper.

Correct!

I understand this, but I see great potential. It’s approach to some of the problems (like data races and memory management) is likely to be as revolutionary as the adoption of the garbage collection based runtimes, like JVM.

Life critical software aims for defect-free (to the extreme possible), responsiveness and high-availability (eg. preference of survivability over crash in unexpected cases), as almost any other service-level software, but maybe with greater focus. In this case, reduction of complexity, static analysis, runtime analysis, tests are all applicable practices regardless of the language. I see Rust outperforms C++ already in the areas of reduction of complexity and static analysis. What I have tried so far is impressive. I am sure high-availability and further reduction of complexity will come eventually, but it depends on attitude.

Maybe, but I have got many other problems, where I see Rust almost already well applicable. I think I will stay and follow.

Thank you for your clarifying answer.


#20

So just to put it out there, you can hide the many concrete types by putting boxes around them. This allocates and prevents some compiler optimizations but does put a leash on the type complexity. This isn’t satisfactory from a performance (and ergonomics) standpoint but is something you can (and sometimes must) do today.