Cargo unfortunately doesn't try to unify major versions and will keep making that duplication by default.
You can downgrade the dependency only for your lockfile
cargo update -p arrow@54.0.0 --precise=53.3.0
Some helpful commands:
cargo tree -d
cargo tree -i arrow