Network Deployment
This article is intended for network systems administrators and engineers deploying Ditto in their specific network environment and provides essential information about different transports and configurations, networking concepts, and various wireless protocols.
The Ditto local area network (LAN) uses both Multicast Domain Name System (mDNS) and DNS Service Discovery (DNS-SD) to automatically discover and communicate with other Ditto peers. For more information, see the official documentation for mDNS and DNS-SD.
When you have multiple network segments and want to ensure Ditto peers automatically discover each other, configure the mDNS multicast frame as follows.
IPv4 Multicast Address | 224.0.0.251 |
|---|---|
IPv6 Multicast Address | ff02::fb |
Port | UDP 5353 |
DNS-SD Service Type | _http-alt._tcp |
mDNS Service Announce Message Example | ditto<hashed-app-id>_announce._http-alt._tcp.local |
Ditto uses various transports to sync data, all of which utilize the 2.4 and 5 GHz 802.11 device antenna, commonly known as Wi-Fi.
However, there are important differences between these transports to consider when configuring and debugging your network.
Apple Wireless Direct Link (AWDL) and WiFi Aware are peer-to-peer links which do not require any 802.11 Access Points, Switches, or Routers. In these transports either data is sent on 802.11 frames directly or one or more devices act as a wireless access point. Importantly, no switch, router, or firewall is present on the networking path between devices. Instead device discovery occurs through direct observation of bluetooth, 2.4GHz, or 5GHz 802.11 network traffic. AWDL and WiFi Aware are not implemented or configured by the Ditto SDK, but are services provided by the underlying operating system.
Thus, if any AWDL service is working properly (i.e. AirPlay), then Ditto peer discovery and data sync via AWDL should also work provided permissions are correctly configured. Note: AWDL can only discover other AWDL (iOS & macOS) peers and WiFi Aware can only discover other Wifi Aware (Android) peers. Without a LAN transport, the only common link between these devices is Bluetooth Low Energy (BLE) which has limited bandwidth for data sync. When possible, we recommend permitting LAN transport for best performance.
The LAN and WebSocket transports also use the device’s 2.4 GHz or 5GHz antenna at the physical layer and so are also considered “WiFi”, but these peer devices are part of a Layer 3 IPv4 or IPv6 network which adds important packet routing and filtering capabilities. Wireless Access Points, Switches, Routers, Firewalls, Gateways, DHCP servers and other networking hardware are present on these networks and responsible for assigning IP addresses to Ditto Peers. In more technical language, the “LAN” transport is actually a stack of IP + mDNS + TCP + mTLS with an optional extension to web sockets when DNS resolution of peers is desired. There are some key points to bear in mind:
- Ditto Peer discovery occurs via mDNS packets sent over UDP port 5353 to a multicast IP address
- Each Ditto Small Peer *will* listen for and respond to mDNS queries on UDP port 5353 by default. These UDP datagrams provide the IP address and TCP port that each small peer is listening for LAN (TCP) client
- Both the mDNS Query and mDNS response datagrams are required to discover a peer Many network switches do not forward these frames because they can lead to network congestion if misconfigured. *Forwarding of these multicast frames must be enabled to allow automatic Ditto Peer discovery*
- Further IP-based networks may be segmented into subnets and VLANs, preventing devices from discovering each other
- Both TCP and Websocket transports implement mTLS encryption within the peer-to-peer connection
- You should consult your access point and network switch documentation how to enable multicast forwarding. This may also be called “avahi-proxy” or “mDNS proxy”
- If you are concerned about mDNS security, forward only the ditto multicast frames. They look like `ditto<hashed-app-id>_announce._http-alt._tcp.local`
- Stale mDNS announcements are typical, but Ditto SDK does ignore stale announcements so duplicate messages to unavailable devices are avoided
- If mDNS peer discovery is not permitted on a network, then Ditto may be configured to use either static IP addresses or traditional DNS discovery. This will need to be provided by your DHCP and DNS servers. This is generally not scalable for large numbers of dynamic peer devices, but may be a suitable fallback for some applications.
Each Ditto Peer is capable of acting as both a Client and Server for each Ditto transport.
This means a Ditto Peer can listen for TCP and Websocket Connections from a particular IP address on a particular port. This is configured in the Ditto SDK through the TransportConfig struct<link>.
By default, a Ditto Peer will attempt to connect to the Ditto Big Peer via Websockets by resolving wss://<app-id>.cloud.ditto.live on TCP port 443. Peers will not listen for incoming Websocket connections by default, but this can be enabled. You can manually add other peers acting as servers either using their TCP/IP address or a websocket URL.
A Ditto Peer is unable to send an mDNS update message when it is turned off or goes out of wireless network range, so stale mDNS announcements are typical.
Network firewalls need to permit outgoing web socket connections on TCP port 443 if the Ditto Big Peer is in use.
Outgoing HTTPS traffic to the Ditto Auth Provider; as in, https://<app-id>.cloud.ditto.live on TCP 443 is also required.
You can connect to Ditto peers running on a local 802.11 IPv4 network externally by permitting incoming TCP or Websocket connections. The target (listener) device typically needs to have a static public IP address, a DNS-resolvable IP address, or appropriate Network Address Translation configured in the network Gateway. The default web socket port 443 is privileged on most operating systems, so selection and configuration of an unprivileged port is generally advisable.
Note that this device will be exposed to the public internet which may not be part of your security threat model.
You should take appropriate security precautions. For this reason the WebSocket transport may be preferable or better supported. Both TCP (LAN) and WebSocket transports provide mTLS-on-the-inside for authentication and encryption after the transport connection is established.
WiFi is very sensitive to interference and the number of clients. While Ditto does its best to be bandwidth-efficient, meshes are inherently more bandwidth-intensive than a typical hub-and-spoke server model. For larger meshes (eg. more than 30 devices), LAN issues can start to really play a role in performance degradation. Through our own testing, we’ve learned a couple of useful tips:
- Avoid 2.4GHz when possible. It’s a very congested band.
- Optimize your radio channel selection. Check that you’re not accidentally operating in an already congested space. Enable automatic channel selection if it makes sense for your environment.
- Design your WiFi infrastructure to handle about 30 clients per radio. To clarify, a typical WiFi AP has 2 or 3 radios (2.4GHz, 5GHz, and recently 6GHz). If you have 100 devices, plan on setting up 4 Access Points in your building, and ensuring that the devices balance out connections across the APs (something that can be obnoxiously hard to orchestrate automatically…).
- Ensure that the connections between the APs and the switches are high-bandwidth and low latency, as the APs will be communicating large amounts of traffic between each other to reach other peers.
- Experiment with enabling multicast-to-unicast conversion for your WiFi infrastructure, often called “Multicast Enhancement” or something similar. Multicast traffic over WiFi is a complicated beast, and in most circumstances, it’s actually beneficial to convert multicast packets to unicast before sending them over the radio. See this StackExchange explanation for more explanation.
Well-known wireless manufacturers will advertise their APs as capable of supporting “100 clients”, but what they really mean by this is an ideal case of 34 clients on 2.4GHz, 33 on 5GHz and 33 on 6GHz. 100 clients all on 5GHz is a severely degraded experience.
The easiest way to verify your WLAN is working properly is to install a Ditto Demo App such as Sky Service on several test devices. The SkyService App can be downloaded from the Apple and Google app stores. These should be connected to the site’s wireless access point and have Bluetooth disabled.
From the Settings menu, you’ll find the presence viewer which will show discovered peers. Verify that peers connected to different wireless access points can discover each other and are visible in the presence viewer.
Wireshark You can use the network traffic capture and analysis tool to directly observer the Ditto announce messages on a variety of interfaces. This powerful tool is available on Windows, Mac, and Linux. Generally, it is sufficient to capture filter udp port 5353 and use the built-in mdns display filter. You may need to make multiple captures at different points in your network.
Here is an example of a Ditto mDNS Query in Wireshark:
And here is the associated Ditto mDNS Response. You must see both the request and the response. If the response is missing, it is likely network hardware is blocking your mDNS request.
