Semver for refactoring change

[geebee22]

@H2CO3

You might say that it's an improvement, after all.

An improvement seems more like a minor version than a patch. Ultimately, I think the answer is probably "it does not matter" (in most circumstances). But this is what is bugging me, you still have to make a choice and there is no clearly correct answer.