Rustler and Elixir bitstring and non-utf8 data types

help
1

Hello.

I am using Rustler to investigate adding some Rust performance into an Elixir application. One of the data types is BitString and not sure what type I can use in Rust for this.

iex(1)> content = :zlib.gzip("<p>Hi. I am some content</p>")

content |> i
Term
  <<31, 139, 8, 0, 0, 0, 0, 0, 0, 19, 179, 41, 176, 243, 200, 212, 83, 240, 84, 72, 204, 85, 40, 206, 207, 77, 85, 72, 206, 207, 43, 73, 205, 43, 177, 209, 47, 176, 3, 0, 55, 217, 16, 5, 28, 0, 0, 0>>
Data type
  BitString
Byte size
  48
Description
  This is a binary: a collection of bytes. It's printed with the `<<>>`
  syntax (as opposed to double quotes) because it is not a UTF-8 encoded
  binary (the first invalid byte being `<<139>>`)
Reference modules
  :binary
Implemented protocols
  Collectable, IEx.Info, Inspect, Jason.Encoder, List.Chars, String.Chars

It suggests it is non-utf8 data. I was looking at using Vec<u8> or &[u8] but I think these would be utf-8 only.

When using the vector of "Vec" at runtime I get:

** (ArgumentError) argument error
   (zlibrs 0.1.0) Myapp.myfun(<<31, 139, 8, 0, 0, 0, 0, 0, 0, 19, 179, 41, 176, 243, 200, 212, 83, 240, 84, 72, 204, 85, 40, 206, 207, 77, 85, 72, 206, 207, 43, 73, 205, 43, 177, 209, 47, 176, 3, 0, 55, 217, 16, 5, 28, 0, 0, 0>>)
   iex:2: (file)

When using the byte slice of "&[u8]" I get an error when compiling.

error[E0277]: the trait bound `&[u8]: Decoder<'_>` is not satisfied
  --> src/lib.rs:4:1
   |
4  | #[rustler::nif]
   | ^^^^^^^^^^^^^^^ the trait `Resource` is not implemented for `&[u8]`, which is required by `&[u8]: Decoder<'_>`
   |
   = help: the following other types implement trait `Decoder<'a>`:
             &'a Path
             &'a T
             &'a str
             ()
             (A, B)
             (A, B, C)
             (A, B, C, D)
             (A, B, C, D, E)
           and 34 others
   = note: required for `&[u8]` to implement `Decoder<'_>`

Any help on this is appreciated. Thank you.

2

No, that's why there's a separate String (and str) type. An u8 is an arbitrary byte, and a Vec<u8>/[u8] is a collection of arbitrary bytes.

(You will not be able to use &[u8], because you can't deserialize into a reference type – there has to be some owner for the data that the FFI glue code creates.)

I'm not familiar with Elixir. What does that error mean?

Isn't the problem that the binding expects a collection of bytes, but you are passing a string to it?

3

It seems converting the data item to a list seems to allow the function to work.

content = :zlib.gzip("<p>Hi. I am some content</p>") |> :binary.bin_to_list()

It must have been expecting a list instead of a different type. Thank you @paramagnetic for the suggestion about not using [u8]. It works with Vec.

1 Like
4

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.