TRust-DNS Resolver 0.9 and Client/Server 0.14

I'm torn on that at the moment, i.e. the other option would be to just remove the function on Windows, but I decided this tack instead. We'll see as people use it and what feedback there is.

Ah yes, just removing it was also the first thing that came to my mind. On the one hand that means you realise the missing impl at compile time, on the other hand the current error-message is far nicer than suddenly getting "no such method" when cross-compiling.

Purely on principle, I think consistent compile-behaviour + nice error message trumps deferring the error to runtime.

But it would be really nice to have this as compile-time error with a nice message.

I felt like there should be a way to have our cake and eat it too, so I went searching. Look what @alexCrichton stabilised just a few days ago: compile_error!. Coming in rust 1.20 :heart:
Thanks @wesleywiser for implementing it!

4 Likes

It looks small, but this version brings some big features. DnsSec validation was merged in a couple of weeks ago, but I held off on a release until I could build a cache because it seemed inappropriate to put something out there that would increase the number of requests to the DNS network. The LRU cache will store successful DNS queries, defaults to 32. This can be improved, and currently doesn't cache NXDOMAIN responses, but it's good to get it into peoples hands for some feedback.

There's one breaking change to the LookupIp interface due to the nature of cached records. LookupIp formerly implemented the Iterator interface directly. The LookupIp::iter function should be used instead to get at the IPs returned.

See: cache size and validate for enabling these features.

0.4.0

Removed

  • breaking impl Iterator removed from LookupIp result type, see LookupIp::iter for replacement

Added

  • Support for DNSSec validation
  • LRU Cache
1 Like

In this release the Resolver gains the ability to perform generic record resolution. The Resolver also gained specialized methods for other specific record types, all described here: trust_dns_resolver::Resolver - Rust

Specifically, in addition to the existing lookup_ip, the resolver now has specialized methods for:

  • lookup for any type
  • lookup_service a convenience for properly constructing an SRV lookup
  • reverse_lookup for IP to Name lookups
  • mx_lookup for mail exchanges
  • txt_lookup for TXT records

Release notes:

Resolver 0.5.0

Changed

  • breaking LookupIp now returns an iterator over owned data (IpAddr is Copy + Clone ref not necessary)
  • breaking Resolver::lookup will now return an Err on NxDomain and NoData responses
  • rewrote much of the caching and lookup functionality for generic RecordType lookups
  • removed &mut from resolver fn interfaces, make it easier to use

Added

  • Generic record type lookup
  • reverse_lookup for IP to Name lookups
  • ipv4_lookup for looking up only ipv4 (lookup_ip has options for dual-stack)
  • ipv6_lookup for looking up only ipv6 (lookup_ip has options for dual-stack)
  • mx_lookup for querying mail exchanges
  • srv_lookup for service records and also a specialized form for ease of use lookup_service
  • txt_lookup for text record lookups

Client 0.11.3

Added

  • lookup to ClientHandle, simpler form with Query
  • query to Query for ease of Query creation
5 Likes

Initially, this was going to be a small release, but then a bunch of contributions brought a whole bunch of features! There are a few major things in this release.

On the Client side, which also is used by the Resolver, @briansmith did a significant amount of refactoring to have *ring* perform the DNSSec proof validation, making OpenSSL completely optional. This means that *ring* can be used for RSA, ECDSA and ED25519 validations. There is more work going on here to cleanup the library and remove OpenSSL as a dev dependency as well, but this is a huge step in that direction.

Moving up to the Resolver, and a big thanks to @liranringel, the Resolver can now read the system DNS configuration from the Windows registry (only 64bit support at this time). This was a huge contribution, which is based off @liranringel's new ipconfig crate for Windows. This is an awesome new addition. It can be used with the Resolver::from_system_conf method.

Additionally, @cssvision added support for reading the /etc/hosts file (on Unix's) first for lookup_ip resolutions. This can be disabled with the ResolverOpts::use_hosts_file option, and is on by default.

For myself, the NameServerPool now first prioritizes UDP for lookups prior to attempting TCP. This should fix some resolution issues where TCP connections might not actually be available. In addition to that, on truncated responses from upstream servers, the Resolver will promote to TCP and continue the Resolution (mostly important for DNSSec). This helps with large response record sets. Also, CNAME chains will be fully resolved, which was a limitation of the Resolver in the past.

Thank you to all the contributors in this release! Also, thank you to the users who've reported issues in the various TRust-DNS libraries, these reports are important to the growing stability of the libraries. Full release notes below:

0.6.0 Resolver

Changed

  • Split UDP and TCP into different NS pools, prefer UDP lookups first
  • On truncated UDP responses, promote to TCP for resolution

Added

  • 64bit Windows support for reading DNS configuration! (@liranringel)
  • CNAME chain resolution (where CNAME results are not returned in the same query)
  • Resolution prefers /etc/hosts before querying (@cssivision)

0.12.0 Client & Server

Fixed

  • Server was not properly signing zone after fresh start

Added

  • RSA and ECDSA validation with ring for DNSSec, removes dependency on openssl (@briansmith)
  • lookup to ClientHandle, simpler form with Query
  • query to Query for ease of Query creation

Changed

  • Large celanup of signing and verification paths in DNSSec (@briansmith)
  • breaking changed TrustAnchor::insert_trust_anchor to more safely consume PublicKey rather than Vec<u8>
5 Likes

TRust-DNS Client & Server 0.13, Resolver 0.7

There are a big batch of bug fixes in this release. There are a number of large changes here. Through some discussions with @briansmith it was decided to fork off central portions of the DNS library to a new trust-dns-proto crate. Brian also spent a lot of time reviewing the library and found some potential bugs in different areas. This release includes fixes for what were the largest issues (see the changes below). During the refactor to extract the Proto crate it was attempted as best as possible to leave in place backward compatible exports, though that wasn't possible in all cases. For anyone that has issues upgrading, please reach out to me and I can help point you in the right direction. Specifically, DNSSec is now disabled by default, this can be enabled with either the dnssec-ring or dnssec-openssl features. This was all done to better harden the Resolver library. A huge thanks is owed to Brian for the in depth review and help in refactoring different components.

On the Client and Server front, with a lot of help and patience from @Darkspirit, the Server is now better in compliance with NxDomain and empty NoError responses. There were also some response issues fixed related to Supported Algorithms, where not all RRSIGS were returned. In addition to that, expectations around self-signed RRSIGs for the ZSK DNSKEY were fixed to pass standard DNSSec validation tests. Through this process we also improved the logging output to make it easier to understand what happens with a request as it's handled by the Server.

There are also some cleanliness things that were added to the project in the way of Clippy and Rustfmt CI passes. @little_dude did a lot of work for that, thanks!

Thank you to all the contributors for the ongoing support. This release took a little longer to come together than anticipated, thanks for the patience! Here are the changelogs:

Resolver 0.7.0

Changed

  • Resolver no longer depends on Client
  • breaking Resolver no longer returns io:Errors, use From<ResolveError> for io::Error
  • Resolver is now Send
  • DNSSec now disabled by default in Resolver, see dnssec-ring or dnssec-openssl features #268
  • CNAME chaining was cleaned up #271 (@briansmith)
  • On hostname parsing to IpAddr, return without lookup #302 (@cssivision)
  • Change default LookupIpStrategy from Ipv4AndIpv6 to Ipv4thenIpv6 #301 (@cssivision)

Added

  • ResolveError and associated types

Fixed

  • Cleaned up CNAME chained lookups, better TTL enforcement, etc #298

Client and Server 0.13

Added

  • TRust-DNS Proto crate to separate server management from base operations #222
  • TRust-DNS Util crate for dnssec management tools (@briansmith)
  • Integration tests for Server to validate all supported DNSSec key types
  • breaking Common features dnssec-ring, dnssec-openssl, and dnssec across all crates (replaces openssl and ring features)
  • Clarified tls feature with tls-openssl, and tls in server (in preperation for tls-rustls)
  • Support for rfc6844, CAA record type #234
  • Support for rfc6698, TLSA record type #285
  • Clippy validation in CI #288 (@little-dude)

Changed

  • DNSKEY is now self-signed
  • Internal API changes to client calling into proto for actual implementations
  • Large refactoring of internal APIs to more cleanly support *ring* and OpenSSL features (@briansmith)
  • ClientHandle::send moved to trust_dns_proto::DnsHandle::send (internal API)
  • Many interfaces moved from client::ClientStreamHandle to trust_dns_proto::DnsStreamHandle
  • Message::sign has been renamed and change to the more general method Message::finalize
  • Some io::Errors have been converted to trust_dns_proto::ProtoError
  • SyncClient and SecureSyncClient are now Send + Sync #245
  • Unknown RecordTypes and RDatas will no longer error #294

Fixed

  • Server: signing issues when loading from persistence
  • Server: When SupportedAlgorithms (rfc6975) not supplied default to returning all signatures #215
  • Proto: u16::from(DNSClass) now enforces OPT is greater than/or 512 per spec #303
  • Improve usage of Rand for message ids and port assignment #291 & #292
  • NxDomain and empty NoData responses to be compliant #286 (lots of help from @Darkspirit)

Removed

  • Removed the NativeTls and OpenSSL ClientConnection variants, use the Rustls impls or the tokio based TlsClientStream instead. This was required for SyncClient being Send + Sync
  • Server: no longer auto-generate keys on startup #218
  • All deprecated APIs removed from -proto #262
  • Server: removed deprated RSA config loading options, see reference test cargo.tomls #276 (@briansmith)
10 Likes

TRust-DNS Resolver 0.8

This is a minor release. It's mostly oriented around some performance improvements. The biggest new feature is support for IDNA, punycode, Names. I had wanted to get mDNS (multicast DNS) support completed for this release but the flu had other ideas. Another note, while I enjoyed learning and working with LALRPOP, it proved to be too expensive of a parser for the fairly simple resolv.conf file. @little_dude worked with @tailhook to get the resolv-conf crate to be compatible with TRust-DNS' needs and @cssivision finished up the work to integrate it into the Resolver. Thanks for all the help here! Users should see a big improvement to compile times.

A big thanks to this release's contributors (PRs listed below): @cssivision, @little_dude, @oherrala, @neosilky, and liranringel, jannic

Resolver 0.8.0

Changed

  • Updated trust-dns-proto to 0.3, which brings in better Name and Label impls
  • Dropped LALRPOP resolv.conf parser in favor of the resolv-conf #335 (@cssivision & @little-dude)
  • Improved message serialization #311 (@little-dude)
  • Many serialization improvements #317
  • Dependencies updated #334 (@oherrala)

Added

  • Name and Label now support idna, punycode, see Name::from_str
  • Clippy added to build #304! (@neosilky)
  • from_system_conf on now supported on Windows 32bit targets (previously just 64bit) #313 (@liranringel)

Fixed

  • octal escapes fixed in Name parsing #330
  • NULL record type incorrectly valued at 0 to proper 10 #329 (@jannic)
9 Likes

TRust-DNS Resolver 0.8.1

Subsequent to releasing 0.8.0, two bugs were discovered; both are fixed in 0.8.1. I consider these to be critical enough that I'll be yanking the trust-dns-resolver 0.8.0 and trust-dns-proto 0.3.0 Crates. The first and most major was #339 which was a panic due to an edge condition during compression and serialization of Name. The second and less serious was #340 which would panic on an assertion when trying to lookup an empty list of names, this has been changed to an error instead. The patches all have proper regression tests in place so that these don't happen again.

Sorry for any inconvenience this may have caused.

Resolver 0.8.1

Changed

  • Make read_system_conf() function public #338 (@oherrala)
  • Hosts map was not properly reference counted #342

Fixed

  • Panic in edge case of label compression #341 (@SAPikachu)
  • Fix localhost lookup and no longer panic on no names #343
2 Likes

TRust-DNS Resolver 0.9

The big announcement here is DNS-over-TLS is finally in the Resolver! Thank you to cloudflare 1.1.1.1 and quad9 9.9.9.9 for finally giving us some endpoints with which we can use DNS-over-TLS. TRust-DNS Client and Server have supported DNS-over-TLS for over a year, but we never had a recursive resolver we could use for our configuration in the Resolver, so I held of implementing it until there were some 3rd parties to validate with. This requires one of the DNS-over-TLS features to be enabled for use. dns-over-rustls will use the *ring* based rustls library, and if you're also using DNSSec would work best with dnssec-ring for the greatest overlap in library usage. The other feature for suggested use would be to use dns-over-native-tls, which will use the host platforms default TLS implementation. There is dns-over-openssl as well, but that requires a bit more to configure the root CA's, etc, so I don't recommend it's usage unless you know what you're doing. If you have one of those features enabled, then you should see in the docs for the trust_dns_resolver::ResolverConfig these functions: cloudflare_tls and quad9_tls, see config::NameServerConfigGroup if you're interested in a method of combining the NameServers from both into a single NameServerPool.

The next big announcement is that all the libraries have been ported to the new tokio. I had very little to do with this effort, and it was massive. We owe a huge debt of gratitude to @Keruspe and @justinlatimer who did a substantial amount to get this done. Thank you! In the process we noticed many of the internal types in the libraries were not Send, this has been fixed, and should make things a little easier to use.

In the vain of making things a little easier to use, after working through some issues with @zonyitoo it became apparent that there is a gap in the library. We need a global resolver people can address and use statically (probably will require a background thread). Before that lands we put together and example in the library for creating one before a version is added to the library: https://github.com/bluejekyll/trust-dns/blob/master/resolver/examples/global_resolver.rs. Any feedback on this can go into the related issue, https://github.com/bluejekyll/trust-dns/issues/462.

There were many other improvements as well, like TTLs being exposed from Lookups by @hawkw and other fixes and improvements, see below. Thank you to all the trust-dns contributors, https://github.com/bluejekyll/trust-dns/graphs/contributors!

0.9 Resolver

Added

  • DNS-over-TLS configurations (requires one of dns-over-native-tls or dns-over-rustls features) #396
  • Experimental DNS-SD, service discovery (RFC 6763, mdns feature required) #363
  • Experimental mDNS, multicast DNS, known issues persist (RFC 6762, mdns feature required) #337
  • Exposed TTLs on Lookup objects @hawkw #444
  • Added global resolver example #460

Changed

  • Use tokio-timer (part of tokio upgrade) @justinlatimer #411
  • Backtrace now optional @briansmith #416
  • Upgrade to tokio-tcp (tokio upgrade) @Keruspe #426
  • Upgrade to tokio-udp (tokio upgrade) @Keruspe #427
  • Upgrade to tokio-executor (tokio upgrade) @Keruspe and @justinlatimer #438
  • Always reattempt nameserver reconnections regardless of time #457
  • Defaulted type parameter for LookupFuture, removed InnerLookupFuture #459

Fixed

  • BinEncoder panic on record sets of extreme sizes #352
  • Panic when oneshot channel receiver goes away #356
  • Incorrect IPv6 configuration for Google nameservers #358
  • Properly yield on failure to acquire lock #372
  • Correct order of search list with ndots variable #410
  • Send (Sync where applicable) enforced on all DnsHandle::send and other interfaces #460
  • Properly track max query depth as a task_local not thread_local #460, #469
  • IPv4 like name resolution in lookup_ip with search order #467

Removed

  • usage of tokio-core::Core @Keruspe #446

TRust-DNS Client and Server 0.14

The Client and Server libraries have also been updated. There are fewer big things to announce here, but it also benefited from the tokio upgrade. Thanks!

0.14.0

Changed

  • Updated trust-dns-proto to 0.3, which brings in better Name and Label impls
  • rusqlite updated to 0.13 #331 (@oherrala)
  • Many serialization improvements #317
  • Use tokio-timer (part of tokio upgrade) @justinlatimer #411
  • Backtrace now optional @briansmith #416
  • Use tokio-tcp (part of tokio upgrade) @Keruspe #426
  • Use tokio-udp (part of tokio upgrade) @Keruspe #426
  • Upgrade to tokio-executor (tokio upgrade) @Keruspe and @justinlatimer #438
  • Send (Sync where applicable) enforced on all DnsHandle::send and other interfaces #460
  • ClientHandle api return Send @ariwaranosai #465

Added

  • Name and Label now support idna, punycode, see Name::from_str
  • trust_dns::rr::ZoneUsage for detecting restrictions on Names and their associated zones

Fixed

  • octal escapes fixed in Name parsing #330
  • NULL record type incorrectly valued at 0 to proper 10 #329 (@jannic)
  • BinEncoder panic on record sets of extreme sizes #352
  • Panic when oneshot channel receiver goes away #356
  • Hung server on UDP due to bad data #407

Removed

  • usage of tokio-core::Core @Keruspe #446
6 Likes

Awesome! It's nice to see this project growing :-). btw what would be necessary to implement standalone resolver daemon? Something like Unbound or Knot Resolver.

This was a project I was planning on starting soon. Basically I think we take the named/catalogue from the server crate and integrate the Resolver into it as a forwarding agent.

Then we can improve from there...

1 Like

@bluejekyll Benjamin, first of all, thanks for great work.

I am trying to use trust-dns-resolver="0.10.0-alpha.2" in async mode as global variable. I do not quite understand AsyncResolver::new return result. Why does it return background task? Is it for cases when resolver is used in non-Tokio application and task context is not initialized, so you create one just in case?
If I am going to call resolver from within tokio-based application, do i still need this background task?

Thanks,
Vadym.

1 Like

Apologies for the late response.

In the next release, 0.10, we're trying to clean up some issues in usage of the library, especially across threads and executors. One issue was that when registering the i/o stream handlers in the AsyncResolver, there were a lot being automatically spawned in the background. The new interface is to expose this to the consumer of the AsyncResolver such that the executor on which the background DNS i/o handlers are being run is under the control of the caller. What this means is that prior to using the AsyncResolver's handle, you must spawn the background future.

This example shows the new usage: trust_dns_resolver - Rust

This example is a global resolver: https://github.com/bluejekyll/trust-dns/blob/master/resolver/examples/global_resolver.rs (after reviewing this code, I think it can actually be a little simpler now, but I haven't gone back and done that).

Does that make sense? I'm open to feedback on how we might make this cleaner and/or clearer. Also, a warning. I wanted to get these changes out early, which is why there is an alpha release, but I do expect interfaces to change in breaking ways before 0.10 is fully released.

Thanks for the great question!

edit: I reread your question,

No, this is still highly dependent on tokio, trust-dns is currently highly dependent on tokio, so this is not about making an abstraction for that case.

1 Like

Thanks for the replay. I do understand that both tokio and trust-dns are moving targets and I am happy to provide early feedback :slight_smile:

I am still in process of rewiring my brain to rust way of async so this might be source of some of my questions.
About global resolver. It uses lazy_static macro. Can I achieve the same result with std::sync::Once?
And another question, in examples, runtime is created (Runtime::new(), then spawn and then shutdown_on_idle. I see that is it the same what tokio::run seems to be doing. Is this the simplification you mean?

The simplification I meant was that in the global_resolver example, there is an Arc<Mutex<Option<AsyncResolver>>, which is used to extract the AsyncResolver handle from the thread being spawned to execute the Resolver in the background. This was necessary before b/c the background task wasn't a "pure" future (my term, maybe someone has one better). What I mean by that is that before poll was called on the future, it actually starting some i/o work. Futures should do no work until polled, and that was a rule that I didn't "know" or follow when doing the original work. That has been cleaned up, and so I think this example is probably more complex than it needs to be (but it still works), in that the AsyncResolver can be constructed, and then the background passed into the thread and then handle stored in the global static.

lazy_static! uses Once behind the scenes. I would recommend using that as it handles a bunch of edge conditions, if you want this to be global. Is there a reason you don't want to use that?

What's happening here is that the background task for the AsyncResolver is being run to "completion" when it is spawned. In this case it actually will run until all copy's of the handle are removed. This is because an mpsc queue is used for sending DNS requests to the background task, as long as that queue still has a reference, it will keep the background task running. The shutdown_on_idle will then shutdown the Runtime (if nothing else was spawned to it).

Hopefully that helps!

Thanks, that answers my questions about trust-dns. I was asking about lazy_static! simply because I was not sure what its status (recommended/outdated/hack). I have more questions, but they are more futures/tokio related.
Right now I have 2 concerns. In big application, resolver singleton would not be in main module, so I'll have to remember to init resolver and hook it to main tokio runtime in every test or main(). And second one is shutdown. As I understand, there is no notion of "backround" task, meaning the one which would not block application from exiting. If I am writing a library, I have to instruct application developer to drop resolver, otherwise application will hang?

If it's hack status, then I missed the memo. With more things becoming "const fn" in the compiler and stdlib, there are definitely cases where lazy_static! won't be necessary anymore, but that won't be an option for trust-dns since most of that has to happen at runtime, not compiletime.

This is a great question. While I haven't used this function myself, this Runtime::shutdown_now should allow you to cancel any futures running in the background.

I think you're also raising a point about the operation of the trust-dns resolver as well, which is related to it's runtime... and I think there is a valid question there, about how it should manage work in the background task. To make it idle itself when there are no active requests, perhaps... We should probably review the code and see what we might actually want to happen in that case. If you want to file an issue against that project for this, feel free.