@dthul is on the money in terms of the way to avoid the issue, but conceptually, there's no issue in allowing references to be compared by the sorting routine - those references are dropped after it determines the order between any 2 elements it's looking at.
This is explored a bit in this github issue and this forum thread.