That is true.
But I would like to note, however, that if user doesn't use ECS, then one is probably not on the hunt for performance in general. Or, at least, not at that exact moment.
But if engine requires use of ECS (and, suppose, the state is not ECS-friendly), then rebuilding the entire ECS-dataset is the easiest solution. And if engine doesn't allow for that (or if we're avoiding that), then there are two options left -- either update stale entities "by had", which is really ad-hoc-y and error-prone (in my experience), or do something akin to VirtualDOM, which could be even more error-prone, will definitely be more ad-hoc-y (because entities are typed structs and not heterogeneous data-structures), will definitely be, performance-wise, even worse than rebuilding.
If engine could expose some more primitive APIs (that could as well be used internally by ECS-aware render, sound-engine, etc) that would, hypothetically, be a win for everyone.