Golang binary.Uvarint in rust

Is there any crate or function that is the same as golang binary.Uvarint?

The implement seems easy

func Uvarint(buf []byte) (uint64, int) {
	var x uint64
	var s uint
	for i, b := range buf {
		if b < 0x80 {
			if i > 9 || i == 9 && b > 1 {
				return 0, -(i + 1) // overflow
			}
			return x | uint64(b)<<s, i + 1
		}
		x |= uint64(b&0x7f) << s
		s += 7
	}
	return 0, 0
}

but I cant find such function in rust-lang

Well, for one thing, int and uint are not types in Rust. Do you mean isize and usize, the size of an address on the host machine?

There are some crates for variable-length integers:

https://lib.rs/crates/vint64

https://lib.rs/crates/leb128

Golang has everything-goes standard library, but Rust keeps std minimal. It's normal to rely heavily on crates from crates.io as an extended "standard library".

2 Likes

It looks like the golang binary package implements a golang-specific serialization format. Depending on what you need, I could recommend a few different rust crates.

First - what are you looking for? Something which can read data encoded by golang's binary module, or something equivalent (but not necessarily binary compatible) in Rust?

If you need to read things that have been encoded in that format, I recommend using the byteorder crate and re-implementing the functions you need.

As for equivalent things, there are a few crates. I can attest that bincode is great for encoding lots of numbers into a stable, efficient binary format, and byteorder (linked above) is good for implementing your own encoding doing things byte-by-byte.

I don't know of any crates which offer just a bunch of functions to serialize individual values into a package-specific format, though. It seems like this would be a compromise, kind of the worst of both worlds - you'd need to write your own serialization functions for serializing data grouped together, but you wouldn't get bitwise control of how, for instance, integers are encoded.

If you are looking to serialize a bunch of different bits of data into a binary stream, and you're willing to use bincode's format for that rather than implement your own, you can use it like this:

#[derive(serde::Serialize, serde::Deserialize)]
struct MyData {
    some_int_1: u64, // will be encoded with varint
    another_int: u64, // encodes multiple fields adjacent in the output
    some_string_data: String, // can encode more complicated things too, if you need
}

let data: Vec<u8> = ...;

let my_data: MyData = bincode::deserialize(data);

If you need to have more control, byteorder can be used like this:

use byteorder::{LittleEndian, WriteBytesExt};

let data: Vec<u8> = ...;
// always consumes 4 bytes
// this could be used to implement your function above
let val: u64 = LittleEndian::read_u64(data);

Or the varint crates @kornel mentioned could offer similar interfaces for manual varint serialization.

2 Likes

unsigned-varint implements the format unsigned-varint specified by Multiformats. It seems compatible with Go uvarint.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.