I think this is more of an interpretation issue rather than something explicitly declared.
The main advantage of MessagePack over ProtoBuf (or others) is that it will not require a schema, so we reduce implementation overhead.
Following your standard, we prepend an extra byte to make it an array, while the Rust static type declaration as source of truth is sufficient (the application objects here are the Rust native types).
The goal of msgpacker was always to reduce network overhead by minimizing the serialization time and amount of bytes.
What I'm trying to understand is if there should be a fix on the library or not. I don't think it's the case since the implementation is spec compliant (from the link above)
While that interpretation is certainly possible, frankly, I don't think it's the proper way to use the msgpack format.
msgpack aims to be "It's like JSON" (It lets you exchange data among multiple languages like JSON), and is designed with inter-language use in mind. Its advantage lies in the self-describing nature of its binaries, allowing for schema-less exchange.
However, your argument assumes the use of Rust types as the schema. While this could be called a broad definition of an msgpack binary, it undermines its original purpose. If you're going to use Rust types as the schema, formats like bincode or rkyv would be more appropriate.
Of course, you're free to decide how your library should be implemented. However, the current behavior doesn't seem to be the desirable behavior for an msgpack serializer.
Rust by default is not compliant with other languages. If you want to be able to deserialize a Python struct in Rust, you need to create a wrapper type that will statically represent any native type, as described in here.
To see that demonstrated, you can run a test with the following valid MessagePack Python/TypeScript/etc representation:
[146, 0, 203, 66, 120, 167, 66, 234, 244, 144, 0]
Following your approach, we should tag each element in the collection, making it an array instead of native representation. That would work, but you incur some serious performance overhead if you do so.
The way I see, libraries should offer support for such scenarios, but also offer alternatives that are more performant in case the user is not expecting non-uniform collections; then it becomes more of an architectural choice/implementation detail.
The rationale of not using bincode is because it was very expensive to use it, and rkyv is due to its zero-copy serialization focus that is not helpful on network communication context. Back then, the only viable alternative was to write a new library (now we have great alternatives on the ecosystem).
MessagePack libraries always need to be smallest/fastest. You did a great job with the serialization implementation btw. As for the deserialization, we need to focus heavily on performance as servers will often be bloated by any overhead.
That's the intended design of msgpack. It requires type headers because all values are self-descriptive.
The way I see, libraries should offer support for such scenarios, but also offer alternatives that are more performant in case the user is not expecting non-uniform collections;
msgpack provides an ext types for such scenarios. This allows for the addition of performance-optimized binary representations. For example, the MessagePack for C# implementation includes extensions that copy struct arrays to memory without encoding and extensions that incorporate LZ4 compression, which work very effectively.
While performance-oriented choices are important, they should be opt-in. Defaults should adhere to specifications and standards.
This version fixes an issue where the build would fail when using no_std. Additionally, the c_enum option has been added, allowing you to serialize c-style enums as integer types!
#[derive(FromMessagePack, ToMessagePack)]
#[msgpack(c_enum)]
#[repr(u8)]
pub enum Status {
Ok = 0,
NotFound = 4,
InternalServerError = 5,
}
This version adds the as_bytes option. This is similar to serde_bytes and allows you to specify whether types like Vec<u8> should be serialized as binary. (It defaults to true for performance reasons and to maintain compatibility with previous versions.)
Additionally, we have added FromMessagePack/ToMessagePack implementations for &[T] and Cow<T>. We also fixed an issue where the serialization format for ext8/ext16/ext32 was incorrect.