IP multicast with Tokio on all interfaces

I would like to use IP multicast with Tokio to communicate with all instances of a program running on the same (sub)network. I already figured out that there are some platform-dependent issues, such as that Windows requires me to bind to the unspecified address (0.0.0.0 or [::]) when I want to listen to multicast, while Unix-like platforms allow me to directly bind to the multicast address.

Additionally, I face problems with selection of the correct interface. Looking into the docs of tokio::net::UdpSocket::join_multicast_v4 and join_multicast_v6, I must specify which interface to work on. Unfortunately, "an appropriate interface" doesn't always get chosen when I pass std::net::Ipv4Addr::UNSPECIFIED or 0, respectively, as second argument to these functions: I ended up having a virtual network interface selected that was just connected to a virtual machine, instead of listening to multicast packets on the interface where the default route is set.

I got to the conclusion that I must manually find out which IP addresses and interface number my system has. I would like to do that in a cross-platform manner and to support both IPv4 and IPv6. Note that just getting the IP addresses isn't sufficient, as I think I also need to get this "interface" u32 number that I need to pass to join_multicast_v6. Where do I get that?

The nix crate seems to be an idiomatic choice here? See nix::net::if_ and nix::ifaddrs. But of course it only runs on (U)nix-like platforms. Are there other alternatives which work on all platforms? Or is there an easy way to do this under Windows, so I can use nix on #[cfg(not(windows))] and use the (unsafe) windows crate otherwise?

I would like to avoid pulling in a bunch of extra dependencies or relying on years-old code where I don't know whether it's still maintained. For example, I think the winapi crate could be replaced by the official windows crate, but a lot of crates still use the winapi crate. I would like avoid adding that as a dependency.

Any help is appreciated.

P.S.: Maybe there's an easier way to listen to / transmit on all interfaces when doing multicast, so I don't really need to gather all the interfaces first?

Probing interfaces is a little annoying, because not all libraries return the same entries. (Because their view of what is relevant differs). I was interested in manually constructing ethernet packets and putting them out on the wire, and found some issues when trying to use existing crates. Not sure you'd run into the same problems if you're working on IP level.

I ended up writing my own crate for it (it is unpublished). The unix case is easy, the problem was on Windows.

If you want to go down the route of probing the interfaces yourself: What you may be looking for, on Windows, is GetIfTable2(), MIB_IF_TABLE2, et al.

I can DM you some snippets that lists all interfaces (including names and indexes). Getting this to work was more work than it was worth, and I don't recommend you muck around with it unless you like to punish yourself.

Edit: Perhaps I'm making it more complicated than it needs to be. (I did this a while back). There are plenty of ways to get interface lists on Windows, but I needed the internal device name, which is why I ended up using the GetIfTable family of calls. You don't need it if you're just looking up the interface index.

1 Like

I guess I can use the nix crate for it? However, I still see a problem in the unix case because interfaces usually have a lot of IPv6 addresses. Which one to use? Do I send a packet for each address? That may be a lot of overhead.

In the receiving case, I might use a single socket bound to std::net::Ipv6Addr::UNSPECIFIED and then use the numeric u32 interface IDs passed to join_multicast_v6. But for transmitting, it seems like I can only select the interface by binding to a specific IP address. And with IPv6 that is a lot of addresses (at least on my system).

Moreover, I might need to handle interfaces being added or removed if the system is a mobile/portable device connecting to and disconnecting from WLAN while being moved. Do I poll the interfaces in a regular interval? What if I miss a disconnect followed after a reconnect? Would my sockets still listen on that interface (or still have joined the multicast group) afterwards? Does this depend on the platform? Likely.

A lot of open questions and I think it's difficult to find information on all these issues.

I'll take a look and message you if needed, thanks for the offer.

I feel like I might get away with just obtaining the IP addresses and, in case of IPv6, use 0 as the interface ID (hoping when binding to the IP address, the device will be selected properly).

The remaining problem would be though to pick the "canonical" IPv6. But which address is that? The one that would be used when sending a packet to the default router? Or the one that has prefer_source (see FreeBSD ifconfig) set on my system. This flag doesn't get reported by nix::ifaddrs::InterfaceAddress::flags, for example.

Yeah, I get the feeling this is "ein Fass ohne Boden" (a bottomless pit) to get into. :weary:

Anyway, if someone has somemore hints or advice on the overall (multicast) problem, or on device probing in general, I'm happy to get some more advice / hints.

Another problem I'm facing when using link-local addresses:

fn main() {
    let _: std::net::Ipv6Addr = "ff02::1%lo0".parse().unwrap();
}

(Playground)

std::net::Ipv6Addr doesn't include any information for the interface. Thus I can't use link-local addresses at all. How am I supposed to transmit IPv6 multicast packets to a particular interface using Tokio then?


There is tokio::net::UdpSocket::bind_device, but:

Available on Android or Fuchsia or Linux only.

edit: And:

Note that this only works for some socket types, particularly AF_INET sockets.

So I guess it's not supported for IPv6 at all.

The idea with multicast is to precisely to avoid things like that. You join a group, and multicast to it and let the network stack do what it needs to most efficiently transmit packets to multiple recipients. (If you dump the multicast packets you'll see that it does things like send data out as ethernet broadcast).

As for why the group join uses an IPv4 address and IPv6 interface index -- I'm uncertain, but I have some vague memory that it goes something along the line of: Multicast uses its own subnet, which is not your regular lan subnet -- which means it can't use your existing routing table to know where to output packets to the wire. This is why you need some kind of identifying information for it to know where to send packets. On IPv4 the addresses are considered to be somewhat static, so it can map an address to an interface. In the IPv6 world you have temporary addresses and other fun stuff, so you need to specify an explicit interface -- and the most platform-agnostic way to identify an interface is using the interface index.

Well, yes. :slight_smile:

We spent a day here experimenting with multicast a while back and we came out of the experience with more questions than answers.

1 Like

I'm tempted to use broadcast instead of multicast, and maybe even require the broadcast IP to be manually set. However, broadcast doesn't work with IPv6. I could use ff02::1%interface_name. This works on my command line using socat:

On one end:

% socat -v STDIO UDP6-LISTEN:1234 < /dev/null

On the other end:

% echo Hello | socat STDIO 'UDP6-SENDTO:[ff02::1%interface_name_goes_here]:1234'

But as explained above, it looks like Tokio doesn't allow me to select the interface when binding and the parser for IPv6 addresses will choke on %ifname. Any solution for that?


Unrelated to Rust but to my problem and/or a possible workaround:

Trying to see if I could use IPv4 broadcast to solve my problem, instead of IPv4/IPv6 multicast, I made another weird experience with Windows.

When you plug out the Ethernet cable but have WLAN connected, you won't be able to send broadcasts to 192.168.xxx.255 anymore (assuming both Ethernet and WLAN have/had an IP in the 192.168.xxx.0/24 range). This also is the case when I create a new socket (tested with ping 192.168.xxx.255 on cmd/PowerShell). When we disabled the Ethernet interface on Windows, then packets get sent out on WLAN again. Re-enabling the Ethernet interface doesn't change anything: WLAN is still used if the cable isn't plugged in. But plugging in the cable, makes the (new) socket use the cable again. And disconnecting the cable again will cause the packets to be lost again then (until the interface is disabled manually).

This is all crazy! Why do packets silently get dropped? I don't even get an error report when doing the corresponding calls. :face_with_symbols_over_mouth: (Upset with Windows here.)

It looks like windows expects you to pass the interface index as a 0.0.0.X IP address for ipv4

I believe you can get the list of interfaces with GetAdaptersAddresses. My read of those docs is that the list is returned to you in sorted order by which should be preferred.

The default-net crate looks like it might be a reasonable cross platform solution for querying network interfaces. And appears to depend on windows rather than winapi


This stack overflow answer also has some interesting details that sort of resemble the problems you seem to be having, at a glance

1 Like

Thanks a lot. I think that's exactly what I need to solve my problem, even though there are still a lot of obstacles. Thanks a lot for pointing me to that crate.

Side note: It appears like they use a fairly primitive mechanism to determine the default IP, see source here (just connecting a socket to a 1.1.1.1 peer and checking the local address of the socket), which is then used there. Sort of brilliantly simple, but I think this mechanism will fail when there is no IPv4 configured on a host (or where the IPv4 default route goes somewhere else than the IPv6 default route). I might use a slightly modified variant of this mechanism and then call default-net code from there.

Yeah I did notice that, but a brief search of other solutions used basically the same technique, or were platform specific so :person_shrugging: if it works it works I guess

At this point, I'll use any solution that works, I guess. :sweat_smile:

(It's not really that bad: there is no real outside connection, of course, as UDP is connection-less and there is no packet sent.)

1 Like

Limiting myself to the "default" interface (instead of working on all interfaces) and assuming I can get IPv6 to work by binding to my own IPv6 and using 0 as interface parameter to join_multicast_v6 (and hope this will select the proper interface then), I might get away with just determining my own "default" IP address (where default means the one used for the default route).

Inspired by the default-net crate, I came up with the following code that does not require any extra dependencies:

use std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddrV4, SocketAddrV6, UdpSocket};

pub fn default_ip4addr() -> Option<Ipv4Addr> {
    const DUMMY: Ipv4Addr = Ipv4Addr::new(1, 0, 0, 0);
    let socket = UdpSocket::bind(SocketAddrV4::new(Ipv4Addr::UNSPECIFIED, 0)).ok()?;
    socket.connect(SocketAddrV4::new(DUMMY, 1)).ok()?;
    match socket.local_addr().ok()?.ip() {
        IpAddr::V4(addr) => Some(addr),
        _ => None,
    }
}

pub fn default_ip6addr() -> Option<Ipv6Addr> {
    const DUMMY: Ipv6Addr = Ipv6Addr::new(0x20, 0, 0, 0, 0, 0, 0, 1);
    let socket = UdpSocket::bind(SocketAddrV6::new(Ipv6Addr::UNSPECIFIED, 0, 0, 0)).ok()?;
    socket.connect(SocketAddrV6::new(DUMMY, 1, 0, 0)).ok()?;
    match socket.local_addr().ok()?.ip() {
        IpAddr::V6(addr) => Some(addr),
        _ => None,
    }
}

(Playground)

(Note that Playground apparently doesn't have network support, so these functions will return None on Playground.)

AIUI multicast packets generally don't cross networks, which might cause problems if you have for example Ethernet as the default interface on a computer but all the other devices expecting to participate in the multicast communication are using a Wifi network.

In non-enterprise networks generally the network hardware will treat Ethernet and wifi as participating in the same network so hopefully that won't cause problems even if it does come up

Well, multicast can be routed too (see IPv6 Multicast Address Scopes, for example). But that's a complex issue.

My original idea was to send out multicast packets on every interface, but at this point I'm considering to just limit myself to the default interface. This will have limitations on the use, unfortunately.

Often WLAN and Ethernet will be bridged. But you are right, in some enterprise networks, this might not be the case. I'll keep that in mind.

1 Like

What seems to work finally is:

  • For sending out multicast packets:
    • Use this technique mentioned above to find out the local IP associated with the default route (or any other route if I replace the DUMMY address with a target address inside whichever local network I want to operate on) and bind a socket to it. A new socket can be created on a regular basis, so that even if the interface changes (ethernet cable vs WLAN), the sent-out packages will go out on the right interface.
  • For receiving multicast packets:
    • Bind the socket to the unspecified address (0.0.0.0 and/or ::). Then regularly scan all available interfaces (currently using default-net) and join the multicast groups in regard to each interface, while ignoring any errors (there will be some link-local IPs reported, for example, where I can't join the multicast group). In order to make this work, I must omit the "pick any interface" choice (0.0.0.0 or 0, respectively) as the interface. The advantage here is that I don't need to ever "close" and re-open the socket, so unless network interfaces change, I won't lose any packets.

I find it somewhat ugly, but it seems to work.

I'm not sure if I will (want to) listen on all interfaces, or if the default interface is enough. (Note that in either case, I would bind to the unspecified address and only use the interface when joining the multicast group.[1]) If the default interface is enough, I could (almost) get rid of the default-net dependency, but I still need to do a conversion from an IP to the u32 interface number (as used here by Tokio).

I wonder if there's an easier way to obtain that interface number from a given IP. Maybe the solution would be to go through default-nets source code to see how they do it. But I wonder if there's an easier way.

Note that default_net::interface::get_interfaces seems to be a somewhat expensive operation.


  1. This means I will also receive packets not sent to the multicast group, e.g. I will also receive packets sent to my unicast IP or to a broadcast IP if the port number matches. But Windows doesn't allow binding to a multicast address anyway. ↩︎

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.