I'd like to revisit a question I asked a while back: I'm looking for a configuration file format.
I will continue to use toml for simpler things, but I'm finding it increasingly frustrating to use toml when configurations become more complex.
I tried using ron, and I genuinely like it -- but it suffers from being a serialization format, and not a configuration file format.
The format I want to use is kdl, but the official kdl crate doesn't support a Deserialize like interface. There are other projects, like knus, which do -- but they don't support the kdl 2.0 spec, and they also generate absurdly long symbol names, causing the modern macOS linker to trigger an assertion.
The facet-kdl crate seems to be based on the official kdl crate (i.e. supports the latest kdl spec), and also support a derive macro using the facet ecosystem. So this is basically what I'm looking for .. but last time I checked, facet was still quite experimental.
Anyone using facet-kdl and can write about their experiences with it?
This may be a controversial opinion, but IMO anything hierarchical is a bad choice for a configuration file format. We once used JSON for configuration at a place I worked and it was painful teaching users how to make changes to it. Meanwhile, .ini is just key=value, which is easy to parse for both computers and humans.
I use JSON. It's certainly not perfect, doesn't allow comments and you should care of certain special symbols in values. Selection a right format will depends on:
Do you plan to manually create/modify it
How frequently are you going to modify it
Should it be portable and easy convertible in other formats
Can you provide custom formatting for easy readability
Probably you can extend the set of questions, and then you can select a right format. Regarding kdl 2.0 spec, it looks too powerful for most of my cases.However facet looks cool, so just use it.
The trouble is that some data is just inherently hierarchical, as seems to be the case in OP.
I'd experiment with facet, see if facet-kdl does what you need. If it does, consider contributing back to the upstream when you find issues. Thanks for reminding me about facet, I think I'll use it instead of serde for my current project.
I really want to use KDL, and had some good luck with the knuffle crate (what knus was forked from) but pretty much ran into all the same things you mentioned, plus editor support is also really messy.
For now I've been sticking with tasteful use of YAML, which does have it's own issues (not counting Norway, which is just using an ancient spec problem), but you do have to be careful to handle a bunch of "it seems like this should work" cases that the syntax implies (think all the GitHub Actions workflow trigger syntax variations)
As an aside, I also want to say serde is theoretically just the wrong tool for a config format a human will write, as you can't report semantic error locations or round-trip edits, but I don't have a great alternative other than manually driving a parser. Seems like there's a library there...
In general I agree with that. The issue I have is that this configuration file needs to be hierarchical. I used toml up to a point, but ended up needing to use magic naming conventions to glue parts of the tree together. I switched to ron, which solved those issues, but after that update I got complaints from an end-user about (among other things) "the comma requirement" in the configuration file.
For my part, I'm happy with toml for flat configurations and ron for the more complex ones. But in this case I am forced to listen to user feedback..
The root issue is that this is an end-user complaint. While I'm looking to solve this for a specific project, I'm also looking to standardize around a format for future projects. I should have stated that in the original post: I'm looking for a configuration file format that non-developer end-users don't hate.
In this particular project users need to iterate over configuration changes -- they change some parameters, run tests, change more parameters, run more tests, and repeat as many times as needed. Once the system is fine-tuned, they should "never" need to touch the configuration file again (at least for that specific system). But in some cases it can take a few days to get it right, and I understand the frustration of users who just want to run the tests, and it keeps nagging them about (what seems to them) minor issues like missing commas.
I sent a draft kdl of how the configuration might look to the user who complained about commas, and they were much happier with that. The only problem is that kdl has been more of a pain for me, as the developer.
At this point I basically used up the entire "Would you accept a configuration file format change?" good-will budget for this project's entire life-time, so I want to get it right, and, on a personal note, I also want to never have to go through this switching of formats again.
I was quite hyped about Pkl when it was first announced, and a bunch of Pkl crates immediately popped up on crates.io. I thought Pkl was the final format that I would standardize around. But none of the crates seemed make any progress and it seems like the steam completely went out of Pkl for non-Apple-supported languages.
Yeah, this is the route I'll take.
This was the major "a-ah!" moment I got from last time I asked about configuration formats. I explicitly stated "And obviously I want serde support!", and someone remarked "Do you though?". I was so wedded to the idea of reusing the custom type parsers I have written for serde that it never occurred to me that other frameworks would have their own ways of supporting custom parsers (just as facet and knus does). And up until that point it had never occurred to me that the reason serde sometimes gives non-optimal errors messages is because it's designed for data serialization/deserialization.
I'm not sure, but I think it was 1024 bytes or characters.
The error message says:
ld: Assertion failed: (name.size() <= maxLength), function makeSymbolStringInPlace, file SymbolString.cpp, line 74.
A workaround is to use the -ld_classic switch (which has its own drawbacks). I googled that error and ld_classic and found some forum post where someone who seemed to know what they were talking about stated it was 1024, but I never bothered to dig any deeper than that, so it may be completely wrong. Given that Apple actually does open source some of their stuff, I should probably have gone to the source, but at the time I figured that because it's new it may not be open sourced yet.
There have been substantial changes to facet, and at some point facet-kdl stopped getting updates. I don't think the intention was to abandon it completely, but there's a note in its repo about it being moved into the facet monorepo, but the kdl extension does not live there.
I don't have the time to revive facet-kdl, but it's what I would like to do.
It's a good point, but, for example, when I deal with Android development, it's always easier for me to apply configurations change directly to the Gradle file, than finding a GUI screen to do it more comfortably. You can be surprised how many people still use a terminal based tool for everything, when tons of nice GUI based tools are available. It reminds me the situation with tube/vinyl listeners, when Spotify is available at one click at your phone.