Serde-yaml deprecation. alternatives?

Hi,

With the recent deprecation of serde_yaml, is there an alternative? Make sense to update serde documentation to point out that YAML support is deprecated?

(my project works fine with current version of serde-yaml, but I think this could become a technical debt in future updates of other dependencies in the future)

https://crates.io/crates/serde_yaml/0.9.34+deprecated

7 Likes

According to the advisory there is a maintained fork.

edit: I might have been too quick in connecting dots, early morning. The advisory is talking about the yaml-rust crate, while you were talking about serde_yaml. Sorry about that.
Forking is always an option, perhaps just providing the serde wrappers?

1 Like

I believe there should have been a discussion with the maintainer about first asking if there is anyone open to maintaining the project further, instead of just archiving it all of a sudden on a whim.

I, on the other hand, think that we should be grateful for all of the work that @dtolnay has donated to the community in the form of this project (and others); we have no right to expect future donations of the same kind.

Even updating the repository to let everyone know that it’s no longer maintained was not strictly necessary— Lots of maintainers simply disappear with no notice or warning when other things become a higher priority in their life.

20 Likes

I created a fork on GithHub, serde-yaml-bw and expect to provide some basic maintenance, like moving ahead the version numbers of used packages, bug fixes and maybe some small enhancements that do not break the backward compatibility. I do not think we should abandon the open source project just because it lost the maintainer.

2 Likes

I'm wondering why nobody pointed out the obvious but: did you know you can just keep using the package even though there's a scary sticker in its README? It won't bitrot. You won't get the shiny new features of YAML, if any (but that's probably for the better, given the history of Billion Laughs and the like).

4 Likes

My initial concern was to depend on a library that holds updating other dependencies. I am using YAML as a configuration file for my own services and jobs. The easy alternative is to change them to a TOML file format (I also try to keep compatibility with JSON for parsing parameters, but I prefer to use as a default format one that supports comments). But for reading YAML from other sources (other systems, k8s...) the current library is very good (and a fork that keep up with the rest of dependencies is even better :slightly_smiling_face:).

And to add, serde/serde_json/serde_yaml are very good libraries. Thank you all. :heart:

Just marking deprecated is probably a better idea in a post-libxz attack world. Passing the built up reputation of a mega popular library to an unvetted maintainer is just asking for a supply chain attack.

The closest to a solution to this sort of thing is something like the Apache Software Foundation, which provides a common trust base to diffuse ownership across; but I don't think that thing happens easily, especially in a post GitHub world!

Forking seems like a perfectly reasonable option in comparison.

9 Likes

Because it has been added to RUSTSEC, so cargo audit will complain. If you're making a binary you probably can ignore that, but if you're making a library other people will complain that you're using a dependency flagged by cargo audit and will ask you to stop using it.

2 Likes

But that's dumb. "dtolnay hasn't touched this in two years" is not any more dangerous than "dtolnay hasn't touched it in one year". (Let alone passing maintenance to some random untrusted guy.) This really shouldn't be a "security advisory". It has nothing to do with security, this is pure, useless bureaucracy.

I would just mark any such complaints as WONTFIX with an explanation.

1 Like

To be clear, the issue was more with yaml-rust than serde-yaml (only the latter was maintained by dtolnay), whose maintainer has been unreachable for 4 years. And the RUSTSEC advisory doesn't even list it as a security issue, just as "info" (see RUSTSEC-2024-0320: yaml-rust: yaml-rust is unmaintained. › RustSec Advisory Database). Many people however won't take their time to understand what that means and will instead just panic and go around asking dependencies to not depend on that crate.

dtolnay then just took the opportunity to mark their own crate as unmaintained as well. Note though that's there's no RUSTSEC advisory for that yet.

5 Likes

I understand your problem with the lack of trust but currently cannot offer a solution. I tried to contact the author of the package but so far received no reply so posting here one more time, maybe he is reading. If the you think the project absolutely should not be continued by me, let me known, I will delete my repository on GitHub. My goal was to contribute to open source and Rust community, not do something that just causes disrespect.

This has very little to do with respect and a lot with security. I think it's better to deprecate than to transfer ownership to an untrusted party. serde_yaml is downloaded over 100k times a day. Lots of projects and people depend on it. Giving control over it away to a stranger on the internet would be a huge security risk. You know you don't have the intention to use that kind of reach to infect other devs' computers with your ransomware, but how should the current owner know that? Granted, best would be to keep ownership and add a co-maintainer whose work can be reviewed till trust is created, but that is nothing that can be asked lightly from a FLOSS maintainer, especially with a portfolio like dtolnay's, and any decision they make must be respected by the community that freely benefited from their work.

8 Likes

This may be potentially resolved by somebody more trusted in Rust community contacting me personally, verifying my identity, checking my CV and things the like. Maybe Rust foundation could do something if the maintainer is not willing to talk? Until that, I think I will simply keep the minimally maintained fork as a dedicated dependency for my own project to avoid annoying messages about deprecated dependencies.

It's not the community's or the Foundation's decision to make, though - it's dtolnay's, since it's their library we're talking about. Publishing it for others to use does not, socially, entail that they have an obligation to take community input over its maintenance.

The crates.io maintainers do have a say, but they have repeatedly affirmed the position that crate names are first-come-first-serve, other than outright abuse or legal issues. For them to reverse course and overrule a maintainer by handing over their crate name to someone else would be a striking and substantial deviation from that policy, and would severely undermine developers' trust in the crates.io repository.

You want the crate maintained. That's a valid want. I think the solution you've settled on - maintaining it for yourself - is the best possible solution in the situation we have today. Dtolnay's licensing choices for serde-yaml allow this solution, and even allow you to go on developing your version of the library and publishing it (subject to very relaxed requirements around attribution), if you like. They have already authorized this, by choice, probably at least in part as a solution to continuity for the project should they let it go.

3 Likes

This is a common misconception. Software may not degrade on its own, but in an environment that changes, it may as well bitrot. Ossification is just as troubling to manage.

3 Likes

If they don't spend the time to even understand the issue why should anyone spend time (other than WONTFIX) about their complaints?

If info-level notices cause them to panic then there's some misconfiguration or misunderstanding going on that shouldn't be everyone else's problem.

TL;DR - yaml-rust2 is the way to go.

You say, Many people however won't take their time to understand what that means. Ok, I did. I have discovered an interesting history behind the implementations of Yaml processing in Rust.

While stepping over serde-yaml soure code with debugger, I have finally arrived to some strange and ugly code, with pointers, with lots of unsafe, something clearly autogenerated. After digging out more, I found that, indeed serdes-yaml 0.9 is internally relying on unsafe-libyaml version 0.2 that is auto-translated code of libyaml, a C library. I see no reason of using such a drastic approaches in the code that merely parses some text input. Among other things, this unsafe-libyaml is also set read only on GitHub so we have one more unmaintained package. In C.

I also found that there is an open source pure Rust YAML parser available, yaml_rust. The code of this parser have been sitting in GitHub for three years without development and looks like with the broken build as well. It still does. However it has been forked and continued as yaml_rust2. I do not know much about the reasons and people behind this forking. The yaml-rust2 fork really took off only on March this year and currently has 53,936 downloads on crates .io.

I initially assumed that unsafe-libyaml is some early hack from the dawn of Rust when lots of critical functionality was missing and the time was scarce. However going over Git history revealed for me the more mysterious story: serde-yaml has actually been using yaml-rust in the past! Yaml-rust has been replaced by the auto-translated C code on July 18, 2022, by dtolnay (see this commit). I am not sure why this has been done. There are three pull requests making these changes , but none of them has any kind of discussion. They were all both submitted and merged by dtolnay. There are also two closed issues related to this replacement but also without any discussion. Hence looks more like single sided decision of the maintainer which he obviously has right to make on his own project but there may still be various views towards the change itself.

As a result of all these findings, I am switching into yaml-rust2 and recommend others to do the same. Using this library may be a little bit less straightforward with less things happening automagically but anyway all migration only took for me less than an hour and this includes including learning yaml-rust2 from scratch. Not a big deal.

Sorry for all noise I created here about the alternative maintenance, I needed some forensic not just in the code of serde-yaml but also in few other projects to understand the actual situation. I do not longer recommed using my own fork of serdes-yaml. The amazing world of Free Software is still full of mysterious legends worth Linux vs Minix. And the world is full of wonderful problems waiting to be solved.

P.S. The first version of this posting included many hyperlinks documenting my findings, but as a new user I am only allowed to post two links per post so I was forced to remove most of them.

12 Likes

Though of course it doesn't implement serde

1 Like

(Edited to add the above quote)

There might be the potential to find a vetted maintainer and officially handover the crate.

I think it would be a fork in GitHub but the ability to publish new versions as serde_yaml would be part of the handover. It signifies that dtolnay no longer has responsibility for the crate, but avoids churn in the ecosystem from switching dependencies.

Of course there's a huge question mark over what such a handover looks like and how a maintainer could be vetted and who's going to do the vetting. Something to perhaps work towards as a goal if the libs team are enthusiastic for it.

I expect it will be too late for serde_yaml to have such a blessed solution, which is a shame but sooner or later a fork (or 3) will appear if there's enough demand for it.

I'm gradually building up to contributing some maintenance effort for the rust ecosystem but I have no real proficiency yet. It's a difficult thing to start - I would feel obligated to work hard and worry I'd struggle with time/motivation to do so