Internationalization

I've been using the "internationalization" crate. Functionality is OK, but it has a major bug. If I change the translations in the ".json" file, I have to run "cargo clean", or I get "missing translation" errors. Internationalization is a build time thing, and its dependency management is broken.

That crate has a small number of downloads. There must be a more mainstream crate.

I don't think there is a consensus here yet. I use the fluent crate with i18n-embed and i18n-embed-fl with some success.

2 Likes

Found the bug in "internationalization". See here. It's looking for the translation files in the wrong place.

So I did a local "git clone" to get a local copy to work on. Then, to use that, I provided, in my own project, I changed

internationalization = "^0.0.2"

to
internationalization = { path = "../rustcode/internationalization-rs", package = "locales" }

(The package = "locales" is because the package is called "internationalization" in crates.io, "internationalization-rs" to Github, but "locales" in its own Cargo.toml.)

But, when compiled this way, CARGO_MANIFEST_DIR is not the value for the main project. It's the directory containing of the library's Cargo.toml file: /home/john/projects/rustcode/internationalization-rs/. Which, according to the Rust documentation, is what it should be. If fetched from Github, though, while it ought to be somewhere off in the working directories where the code was loaded from github, it's (usually?) the directory containing the root CARGO_MANIFEST_DIR.

Looking at the environment variables available to a build script, there's no way to get the directory that's the root of the whole build. Yet the whole point of build scripts is often to depend upon something external to the current package. What am I missing here?

The "internationalization" crate should probably not have a build script that tries to collect the translations files of the project using it, it should provide either a library function to be called from the build.rs of your top-level crate, or (nicer) a macro to be used in the top-level crate for collecting the translations.

1 Like

Right, a build dependency on a parent is kind of flaky. Rust documentation for build scripts should say that, so people don't make crates that try to do it and create messes like this. Still, the fact that build scripts that depend on PWD yield nonrepeatable builds is just wrong.

On internationalization, all I need is something simple, for menu items and buttons such as "File", "Open", etc. The internationalization crate has the functionality I need. But it's abandoned; no changes in 2 years.

The fluent crate says it's unfinished, and it's last change was also two years ago. The fluent-bundle crate seems more active, but its documentation also says it's unfinished. Also, the fluent system seems to do a lookup on each use. Syntax is

bundle.get_message("hello-world")

I'm using egui, which needs each string on each frame. So a run-time lookup system without memorization is far too slow.

Looking at the crates, it looks like this area got attention up to two years ago, and then everybody gave up on internationalization crates.

Update: the internationalization crate cannot work as currently designed. See this discussion.

So I looked into using memoization to wrap some kind of key/value store.No good. The memoization crate is broken.. Undefined behavior once allowed is now detected and panics.

All this stuff seems to be abandoned. No changes in years.

I feel like I'm trying to build on sand today.

1 Like

I believe cargo-i18n is currently the best "entry point" for translating a rust project. It does leave the actual translations up to either fluent (which may be kind of immature, but does not seem abandoned, there was changes in GitHub - projectfluent/fluent-rs: Rust implementation of Project Fluent as recently as 4 days ago) or gettext (which is a c language dependency, which may or may not be a problem).

For lookups beeing to slow to load on every frame in a gui toolkit, I think that is more a problem in the (use of) the gui toolkit than in the translation tool. It should be possible to look up the string when creating the widget (or changing its state) rather than on rendering.

Doesn't really work for an immediate-mode system such as egui. Look at the egui samples.

This needs a local static variable, something you can do easily in C/C++, but may not be able to do at all in Rust. You want to code something like

menu(t!("Menu_word",lang))

and have there be some static variable with the appropriate translated string, looked up once on first use. This is straightforward in a language with local statics. Is it even possible in Rust?

I'm starting to see why all the internationalization crates are either clunky or broken.

OK, solved my own problem.

I didn't realize until today you could have a block-local static in Rust. It's somewhat frowned upon. But here it's useful for memoization.

Why a local static? Sure, for some menu items etc it could be a static, but for others widgets, the translation lookup will be parameterized and the result will need to be an owned String. Immediate mode rendering does not mean the entire world has to be recreated for each frame, quite the contrary; it means you must have a good representation of the current state of the world to be able to render it fast enough. I see no reason translated strings can't be part of that representation.

I didn't write egui. It efficiently rebuilds the menus on every frame, using a code representation of the menus is faster than a data representation. Given that, I need cheap in-place translated strings.

So I wrote my own internationalization library. This uses the same syntax and file formats as the internationalization crate, but doesn't have the problems mentioned above.

Yes, it "rebuilds" the menus on every frame. But you control what it builds them from, don't you? I'm quite sure you can build them differently based on some state that you controll, so why not put the translations in that state? I'm not saying the widget should own the state, but clearly your application can have state, and that state can affect the widgets in the gui. The From<&str> imp for WidgetText does not specify that the str needs to be static, it can be borrowed from a String.

Yes, you can declare lots of variables, have structs full of translations, and such. That gets ugly fast. Egui already has style problems, because it encourages more nesting than Clippy likes. The internationalization crate had good ergonomics, so I reused their syntax for a new implementation.