Runtime tokio compatibility errors

Hello folks,

Recently I encountered an issue while trying to update an application to tokio 1.0. The issue is solved (I'll describe it in a moment in case it helps anyone else), but I have a curious question that resulted from experiencing and addressing the issue.

I'm sure experienced tokio devs can guess what happened: I updated the tokio dependency and related libs, made all the code changes that were necessary for build + tests to run, and all looked good ... but when I ran the app I got the following runtime panic:

thread 'main' panicked at 'not currently running on the Tokio runtime.', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.24/src/runtime/handle.rs:118:28

Once I'd got past the initial surprise I noticed the tokio-0.2.24 version in there, and quickly realized that one small part of the code was using a library that was still pulling in tokio 0.2, in a way that could not readily be addressed (there was not a usable update of that library with tokio 1.0 support).

A bit of searching around found me the tokio_compat_02 crate, and I was easily able to fix up the bit of code in question to integrate with the tokio 1.0 runtime. This was quite a relief, because until I discovered that I thought I had come across a major (and nasty) hole in the "automatically bring in multiple versions of dependencies if needed" behaviour that cargo supports.

However, that led to me wondering: why was this error not detectable at compile time?

I wonder if anyone with greater experience in the tokio framework could offer some thoughts on that: I'm anticipating that there might be some interesting things to learn from the explanation.

Thanks and best wishes,

   -- Joe

Thanks for posting this. I hadn't seen tokio_compat_02 before, outside of tokio-rs github. I wonder why this wasn't released as tokio-compat 0.2 (latest is 0.1.6)?

Duplicate dependencies is considered a feature of cargo, and its helpful in certain cases unlike this one of yours, so its not surprising IMO that no warning or error is raised.

Oh, I agree that in principle duplicate dependencies is a feature. What surprised me was that the requirement to insert compatibility wrappers was not detected and reported at compile time.

You need* type collision to have compile time detect conflict between two versions. (* shared libs ie -sys crates also stop at compiling.)

It is sometimes good thing there is the enough isolation that the developers allow use more than one major version.

Seen your problem bite too with log crate. Unexpected missing logs, not such a big a deal.

If there was a reliably way of somehow detecting that, I suspect tokio (1.1) would consider a PR for it. I'm not aware of a way to do that.

It is just how cargo works :woman_shrugging:

We are working on updating the panic messages to include the Tokio version, so it is easier to spot the mistake.

2 Likes

Hello folks, thanks for all the follow-up.

I think this is what I was surprised to not see: I was expecting that if some piece of code returns a Future of a kind not supported by Tokio 1.0, that there would be a compile time error, because it would be detected as a different/incompatible type.

I guess I'm seeking clarity here on what is actually happening under the hood. I assume that it's something like, callbacks under the hood of the tokio-0.2-derived futures call into some global inside the tokio 0.2 lib that expects to be responsible for the runtime, and the compat layer replaces those with callbacks into the tokio 1.0 runtime?

(Yes, I can just dive deeper into the code, but I hope if we discuss this here publicly it can provide something discoverable for other people too. :slight_smile: )

That's great to hear. It would be particularly helpful if the panic message could also refer to the compat libraries so as to give a clear hint of how to address the problem.

All runtimes share the same Future type, which is defined in the standard library. The Tokio runtime does indeed use a thread-local variable for its context, and the compat library indeed messes with this global.

1 Like