Leader election is a well‑known pattern in distributed systems: a set of peers selects one peer to run tasks that must execute exactly once (for example, kicking off a batch job or publishing a heartbeat).
Ditto supplies the key building blocks for many leader‑election schemes: shared collections to hold state, presence information to identify peers, and low‑latency sync so everyone sees changes quickly.
The example that follows shows a small, end‑to‑end leader‑election flow. It keeps the logic simple—no advanced clock‑synchronization tricks or Byzantine safeguards—so you can grasp the essentials and then adapt the pattern to your own requirements for fail‑over speed, clock skew, and trust.
Design Assumptions
The example that follows relies on a few baseline assumptions. Feel free to tune them—whether it’s acceptable clock skew, the maximum time you can run without a leader, or the trust model—so they align with your own system’s requirements.
- All peers connect under the same Ditto AppId.
- Each peer holds read/write permission to the
leader_election
collection.
- We treat every peer as trusted (no actively malicious participants).
- Device clocks stay within 60 seconds of one another—a safe assumption for most modern mobile hardware.
- The system tolerates up to 60 seconds without a leader.
Implementation
Each peer writes a document to the shared leader_election
collection, creating a common pool of leadership candidates. Every peer then runs the same selection algorithm: it reads the pool, ranks the entries, and elects the top candidate as leader. If a peer discovers that it now holds the top spot, it immediately assumes the coordinator duties.
Every five seconds all peers repeat the check, confirming the leader is still alive or electing a replacement when necessary.
Joining the Leader Election Pool
To join the leader election pool users submit a candidate application by adding a document into the leader_election
collection with the following schema:
// Sample leader candidate document schema
{
"_id": "myDeviceId123", // device identifier (we use Ditto's Peer Key below)
"priority": 5, // 10 = high priority, 1 = low, 0 = not eligible
"initial_timestamp": 1692001234567, // when election was joined
"heartbeat_timestamp": 1692001234567 // last liveness ping
}
In the example below our peer is submitting themselves as a candidate by adding a document to the leader_election
collection.
const myDeviceId = ditto.presence.graph.localPeer.peerKeyString;
const currentTimeStamp = Date.now();
const leaderCandidateEntry = {
"_id": myDeviceId,
"priority": 2,
"initial_timestamp": currentTimeStamp,
"heartbeat_timestamp": currentTimeStamp
};
await ditto.store.execute(
"INSERT INTO leader_election DOCUMENTS (:leaderCandidateEntry)",
{ leaderCandidateEntry });
To ensure we don’t have a leader or candidate that drops out of the network, every candidate will use a keep alive heartbeat that the other peers can check to make sure we are still active.
Peers will filter candidates that don’t have a heartbeat timestamp within the last 60 seconds.
This is done by updating the heartbeat_timestamp
for their own record.
const HEARTBEAT_UPDATE_INTERVAL_MS = 10000; // check every 10 seconds
// Setting up a heartbeat
setInterval(async() => {
await ditto.store.execute(`
UPDATE leader_election
SET heartbeat_timestamp = :now
WHERE _id = :myDeviceId`,
{
now: Date.now(),
myDeviceId
});
}, HEARTBEAT_UPDATE_INTERVAL_MS);
Determining the Leader
Leader election can be determined in many ways. We’ll use the following basic sequence to determine the leader.
- Sort by
priority
descending, higher wins (10 high, 1 low, 0 ineligible)
- Break ties with earlier
initial_timestamp
- Break further ties with lexicographic order of
_id
In the example below we’ll use Ditto’s DQL to search and return a sorted list of peers based on our rules as defined above.
// Filter peers where priority < 0 and/or the heartbeat is old
// Use DQL to return a sorted list of peers based on our rules above
const heartbeatCutoffTimestamp = Date.now() - HEARTBEAT_TIMEOUT_MS;
await ditto.store.execute(
`SELECT *
FROM leader_election
WHERE
priority > 0
AND heartbeat_timestamp > :heartbeatCutoffTimestamp
ORDER BY
priority DESC,
initial_timestamp ASC,
_id ASC`,
{ heartbeatCutoffTimestamp });
Checking for Expired Leaders
In the code below we setup a timer to re-run leader election every 5 seconds to ensure that the leader hasn’t changed. The main thing we are looking for is that we are not the new leader. If we are the leader then we need to assume the duties
const HEARTBEAT_TIMEOUT_MS = 60000; // 60 seconds
const LEADER_CHECK_INTERVAL_MS = 5000; // check every 5 seconds
let currentLeader = null;
const myDeviceId = ditto.presence.graph.localPeer.peerKeyString;
setInterval(() => {
checkAndElectLeader(); // proactive check
}, LEADER_CHECK_INTERVAL_MS);
async function checkAndElectLeader() {
// Exclude 0 priority & stale heartbeat candidates
const response = await ditto.store.execute(
`SELECT *
FROM leader_election
WHERE
priority > 0
AND heartbeat_timestamp > :heartbeat_cutoff_timestamp`,
{
heartbeat_cutoff_timestamp: Date.now() - HEARTBEAT_TIMEOUT_MS
})
const currentCandidates = response.items.map(a => a.value);
const newLeader = electLeader(currentCandidates);
const newLeaderId = newLeader?._id || null;
// return if the leader hasn't changed
if (newLeaderId === currentLeader?._id) return;
if (newLeaderId) {
console.log(`New leader elected: ${newLeaderId}`);
} else {
console.warn(`No eligible leader found.`);
}
if (newLeaderId === myDeviceId) {
// I'm the new leader, assume duties
executeLeaderDuties();
} else if (currentLeader?._id === myDeviceId) {
// I'm no longer the leader, pause leader duties
pauseLeaderDutiesIfRunning();
}
currentLeader = newLeader;
}
Opting out as a Candidate/Leader
If a candidate or the leader knows they will not be eligible, we want to provide this information to the group to avoid any delays in new leader election. If the leader just leaves users will wait up to 60 seconds before they determine who the next leader is.
We can streamline the process by either setting priority = 0
or deleting the document entirely.
const myDeviceId = ditto.presence.graph.localPeer.peerKeyString;
await ditto.store.execute(
`UPDATE leader_election
SET priority = 0
WHERE _id = :myDeviceId`,
{ myDeviceId });
// (alternative) Delete candidate document from the collection
// (supported in 4.10 and later)
const myDeviceId = ditto.presence.graph.localPeer.peerKeyString;
await ditto.store.execute(
`DELETE FROM leader_election
WHERE _id = :myDeviceId`,
{ myDeviceId });
Complete Code
const HEARTBEAT_TIMEOUT_MS = 60000; // 60 seconds
const LEADER_CHECK_INTERVAL_MS = 5000; // 5 seconds
const HEARTBEAT_UPDATE_INTERVAL_MS = 10000; // 10 seconds
let currentLeader = null;
const myDeviceId = ditto.presence.graph.localPeer.peerKeyString;
// Create a heartbeat task that updates every 10 seconds
setInterval(async() => {
const currentTime = Date.now();
await ditto.store.execute(`
UPDATE leader_election
SET heartbeat_timestamp = :currentTime
WHERE _id = :myDeviceId`,
{ currentTime, myDeviceId });
}, HEARTBEAT_UPDATE_INTERVAL_MS);
// Create a leader check task that checkes every 5 seconds
setInterval(() => {
checkAndElectLeader();
}, LEADER_CHECK_INTERVAL_MS);
// Check to see who's the current leader and assume duties if elected as leader
async function checkAndElectLeader() {
// Exclude 0 priority & stale heartbeat candidates
const heartbeatCutoffTimestamp = Date.now() - HEARTBEAT_TIMEOUT_MS;
const response = await ditto.store.execute(
`SELECT *
FROM leader_election
WHERE
priority > 0
AND heartbeat_timestamp > :heartbeatCutoffTimestamp
ORDER BY
priority DESC,
initial_timestamp ASC,
_id ASC`,
{ heartbeatCutoffTimestamp });
// Our first document will be our leader
const newLeader = response.items[0]?.value
const newLeaderId = newLeader?._id || null;
// If the leader hasn't changed return
if (newLeaderId === currentLeader?._id) return;
if (newLeaderId) {
console.log(`New leader elected: ${newLeaderId}`);
} else {
console.warn(`No eligible leader found.`);
}
if (newLeaderId === myDeviceId) {
// I'm the new leader, assume duties
executeLeaderDuties();
} else if (currentLeader?._id === myDeviceId) {
// I'm no longer the leader, pause leader duties
pauseLeaderDutiesIfRunning();
}
currentLeader = newLeader;
}
// Given an array of candidates, return the leader candidate
function electLeader(candidates) {
return candidates.sort((a, b) => {
if (b.priority !== a.priority) {
// Sort by priority descending (higher wins)
return b.priority - a.priority;
}
if (a.initial_timestamp !== b.initial_timestamp) {
// earliest timestamp wins
return a.initial_timestamp - b.initial_timestamp;
}
return a._id.localeCompare(b._id);
})[0];
}
// Opt out of the leader election
// call this if the application is shutting down or the user is unable to perform
// leader tasks. (Eg. device is low on battery)
async function optOut() {
await ditto.store.execute(
`DELETE FROM leader_election
WHERE _id = :myDeviceId`,
{ myDeviceId });
}
// Apply candidate application to be a leader
// If the user has submitted before we'll override the previous values
// Needs to be called on startup
async function registerAsCandidate() {
const currentTimeStamp = Date.now();
const leaderCandidateEntry = {
"_id": myDeviceId,
"priority": 2,
"initial_timestamp": currentTimeStamp,
"heartbeat_timestamp": currentTimeStamp
};
await ditto.store.execute(
`INSERT INTO leader_election
DOCUMENTS (:leaderCandidateEntry)
ON ID CONFLICT DO UPDATE`,
{ leaderCandidateEntry });
}