Packaging assets in a library crate

help
1

I'm writing a small library crate that provides exposes an resource catalog to Rust programs. The catalog is a set of JSON files that are exported from another system. The code part is easy; it's just a few macros to define some structs and parse the JSON in, and is working well. My question is; how do I package the JSON files into the crate and how does my library find them?

Directory structure looks something like:

cataloglib
├ Cargo.toml
├ src
│ ├ lib.rs
│ └ foo.rs
└ catalog
  └ foo.json

Where foo.rs contains code to read foo.json and return Rust objects for its content, and searching methods and so on. In its singleton constructor.

cargo package will copy the catalog/ dir into package just fine, but if another project lists cataloglib = { path = "/path/to/cataloglib" }, those files are never copied into the local project when it's built (distribution will eventually be via git, not crates.io, so packaging might not be in play anyway).

Even if they were included, I don't see any way to find out where the crate is "installed" - no environment variables or similar.

I figure this is the same problem that would be faced by someone wanting to make a library of images or other assets, for example. How is this being handled elsewhere?

And if not, what's the alternative? The only think I can think to do is to compile the JSON files into Rust code and package it that way, but that makes things a bit harder for ongoing deployment. Not insurmountable, but annoying, especially since this seems like a thing that should be possible.

1 Like
2

Maybe std::include_str! can help you.

5 Likes
3

I think you need to be clearer about what it is you actually want.

I mean, you say "where the crate is installed", but crates don't get installed. Which suggests you're on a different page to everyone else.

Are you expecting the catalog directory itself to follow the crate around somehow? Are you expecting it to be installed somewhere? Do you just want to embed its contents into the crate? As an archive, or just a bunch of array literals? Does it need to be modifyable at runtime? At compile time of the final binary crate?

4

Yep, that's exactly what I want. There's also std::include_bin! for non-texty things.

(I was also pointed to this Stack Overflow post which said the same thing).

Thanks!

2 Likes
5

A question about that: would it be possible then to decode the JSON itself at compile-time, since the input is fixed?

6

Doubtful. First of all, you can't call functions at compile-time. Secondly, both JSON representations I'm aware of are owning; they expect things to be on the heap, and you can't heap-allocate at compile time.

You could probably have a build script parse a JSON file, then spit out Rust code that constructs a static JSON tree structure, using your own types.

7

Yeah, build scripts work, though they're a bit hacky. I use one to naively parse an .ini of supported Windows system classes and generate Rust definitions for them, which includes creating static nul-terminated UTF-16 strings for Windows FFI:

The definition of SystemClass is here (where the generated file is include!()'d):

https://github.com/cybergeek94/wingui/blob/master/src/ffi/class.rs#L104

8

For another example of a build script that generates code for static data, you can have a look at GitHub - kaj/rust-pdf: Generating PDF files in pure Rust , where src/build_metrics.rs generates code (from .afm font metrics files) that is included in src/fontmetrics.rs.

(Now that I look at it with fresh eyes, in that case using include_str!() seems a bit unnecessary and probably slows down the build. Since the build_metrics code is actually executed in the build state and generates rust source, it could just open and read the data files.)