Do whatever results in the cleanest code downstream.
The performance difference will be negilible unless you are creating millions of these things, and even then you'll gain more performance by using smarter algorithms (e.g. O(log n) instead of O(n2)) or inlining the Box to reduce indirection.
I'll mention a couple of advantages of this. First and foremost, in the common case where it is passed by reference, there is a single layer of indirection. You can allocate this on the head with a simple Box:: new(X { .. }). And in general is like users of my code to be able to make their own choices about allocation, e.g. they might prefer an Arc or Rc over a Box, and this gives them that flexibility without requiring an additional layer of indirection.
Edit This is also beneficial if the want to use these structs in a collection that is already head allocated like a Vec or a HashMap.
It's easier to understand, and still does get null-pointer-optimized if you put in an option (playground).
Though I would still suggest @Michael-F-Bryan's suggestion instead.