Leader Election Pattern in Ditto
Basic end-to-end leader-election example.
Leader election is a well‑known pattern in distributed systems: a set of peers selects one peer to run tasks that must execute exactly once (for example, kicking off a batch job or publishing a heartbeat).
Ditto supplies the key building blocks for many leader‑election schemes: shared collections to hold state, presence information to identify peers, and low‑latency sync so everyone sees changes quickly.
The example that follows shows a small, end‑to‑end leader‑election flow. It keeps the logic simple—no advanced clock‑synchronization tricks or Byzantine safeguards—so you can grasp the essentials and then adapt the pattern to your own requirements for fail‑over speed, clock skew, and trust.
Design Assumptions
The example that follows relies on a few baseline assumptions. Feel free to tune them—whether it’s acceptable clock skew, the maximum time you can run without a leader, or the trust model—so they align with your own system’s requirements.
- All peers connect under the same Ditto AppId.
- Each peer holds read/write permission to the
leader_election
collection. - We treat every peer as trusted (no actively malicious participants).
- Device clocks stay within 60 seconds of one another—a safe assumption for most modern mobile hardware.
- The system tolerates up to 60 seconds without a leader.
Implementation
Each peer writes a document to the shared leader_election
collection, creating a common pool of leadership candidates. Every peer then runs the same selection algorithm: it reads the pool, ranks the entries, and elects the top candidate as leader. If a peer discovers that it now holds the top spot, it immediately assumes the coordinator duties.
Every five seconds all peers repeat the check, confirming the leader is still alive or electing a replacement when necessary.
Joining the Leader Election Pool
To join the leader election pool users submit a candidate application by adding a document into the leader_election
collection with the following schema:
In the example below our peer is submitting themselves as a candidate by adding a document to the leader_election
collection.
To ensure we don’t have a leader or candidate that drops out of the network, every candidate will use a keep alive heartbeat that the other peers can check to make sure we are still active.
Peers will filter candidates that don’t have a heartbeat timestamp within the last 60 seconds.
This is done by updating the heartbeat_timestamp
for their own record.
Determining the Leader
Leader election can be determined in many ways. We’ll use the following basic sequence to determine the leader.
- Sort by
priority
descending, higher wins (10 high, 1 low, 0 ineligible) - Break ties with earlier
initial_timestamp
- Break further ties with lexicographic order of
_id
In the example below we’ll use Ditto’s DQL to search and return a sorted list of peers based on our rules as defined above.
Checking for Expired Leaders
In the code below we setup a timer to re-run leader election every 5 seconds to ensure that the leader hasn’t changed. The main thing we are looking for is that we are not the new leader. If we are the leader then we need to assume the duties
Opting out as a Candidate/Leader
If a candidate or the leader knows they will not be eligible, we want to provide this information to the group to avoid any delays in new leader election. If the leader just leaves users will wait up to 60 seconds before they determine who the next leader is.
We can streamline the process by either setting priority = 0
or deleting the document entirely.
Complete Code
Was this page helpful?