Removing dynamic dispatch that always makes the same choice

help
1

TLDR

How good is the compiler at optimizing away dynamic dispatch that picks exactly the same function each time around a loop?

Details

I have code which injects a function into an inner loop, where that single function will be called repeatedly on items of a huge data set. Exactly what that function does, depends on a number of configuration parameters.

The need for new configuration parameters is constantly emerging, and the chain or tree or web of logic which combines the parameters into the definition of an appropriate function is becoming more and more complex, and involves more steps. Initially I was passing impl Fn* along the stack towards the inner loop, but this is leading me to a proliferation of generic functions with type parameters whose only purpose is to encode (at compile-time) some aspect of the configuration parameter set, all for the sake of preserving static dispatch.

The code would be much simpler if some variety of dyn Fn* were passed around instead, but before I start rewriting it all, I want to be at least slightly confident that there won't be a meaningful performance degradation.

I have two concerns

  1. Some of these functions do trivial amounts of work per iteration, hence my concern that injecting the cost of dynamic dispatch might be significant. On the other hand, in a single run, whatever function runs on the first iteration will also be called on all subsequent iterations, so I hope the compiler can figure this out and not waste time on dispatching over and over again.

  2. At the other end of the complexity spectrum, some of these functions are composed of chains or trees of closures calling closures. I fear that turning all these from statically into dynamically dispatching ones, might prevent the compiler from inlining the components that make up the function or doing some whole-program optimizations and thus result in significant performance degradation.

What sorts of things should I bear in mind when considering replacing the impls in my code with dyns?

Should I consider some other approach (enums, any, ...)?

2

Devirtualization is definitely a thing, and so is branch prediction. But reasoning about performance upfront is very hard; you'll probably have to measure a non-trivial example to obtain any sort of confidence in the compiler's and CPU's ability to optimize away unnecessary dynamism.

1 Like
3

Yes, that's what I was trying to do, but the code has already grown to be so complicated that the amount of work necessary to get two meaningful things to compare, is daunting. So I'm looking for some hints that might help me decide how high a priority this deserves.

4

You can make a high level decision without a comparison. Run a performance profiler on the code with the dyn call, and if you don't see that call as a hotspot, then don't worry about it.

2 Likes
5

Unfortunately I can't do that: the motivation of the question is that there are no dyns in my code, and I don't want to spend a lot of time and effort on putting those dyns in there, if there is a significant risk that it might slow the code down.

If I were thinking of removing dyns from my code for the sake of performance, then, of course, I would profile first to see whether there is any point. But I have the reverse problem: I'm wondering about the potential implications of dyns which aren't there yet.

6

Fair enough. Can you manually introduce the dyn at some inner level just to start testing that? Maybe you can evaluate that, and start pushing that outward if it looks ok, until you get to the full point where you reap the benefit of code simplification that you're hoping for.

1 Like
7

That's what I'm trying to do: find some tractable portion to modify and evaluate first.

It's not easy: the pressure to produce results in the short term has caused quite a bit of technical debt to have accrued here. But if I let it continue like this, it will be even more difficult to clean up later. So I should persevre.