Glossary
In this glossary, you'll find definitions and explanations for key terms, technologies, and features related to Ditto.
An Apple-developed proprietary technology that establishes a point-to-point Wi-Fi connection between two Apple devices. If available, Small Peers use AWDL to form a mesh network connection and sync data among peers.
A logical container, or namespace, that holds all the data associated with your application, or app for short.
- Small Peers are limited to utilizing one app concurrently, a specification included in the Identity. On the contrary, the Big Peer can support multiple apps simultaneously. App IDs consist of strings, with lower-case UUIDs being the prevalent choice.
- A Small Peer can use only one app at a time, specified as part of the identity. The Big Peer can service multiple apps at a time.
The Ditto component for storing large binary files for end users (associated with a document as an ATTACHMENT data type).
Attachments use a different transfer mechanism from regular sync, so, although Ditto references attachments by documents, they must be explicitly downloaded.
An HTTP service that you, the developer building a Ditto-enabled app, host. This service receives credentials and responds with metadata about your end user and which permissions they should have.
Ditto's identity service then uses this information to dynamically produce the required peer certificates, configurable in the portal.
The underlying key-value store that Ditto uses for database-like data persistence.
Broadly refers to the Ditto cloud deployment in its role as "just another peer in the mesh."
A Big Peer is unique; it has highly durable and scalable storage and typically greedily syncs all data from Small Peer devices.
A Big Peer is commonly implemented to manage an entire organization's or app's dataset, which may be substantially larger than the partial dataset any given Small Peer might operate on.
An internal component offers general blob storage that any internal Ditto components use when needing to persist "files."
Files are in quotes (" ") because the blob store does not necessarily require a true filesystem and might operate solely in-memory.
A public service offered by the Ditto SDK for establishing raw byte streams between peers connected in the Ditto mesh.
This technology enables many use cases beyond Ditto's document sync. For example, to include voice chat or game data; syncing third-party (non-Ditto) databases over the mesh network or virtual private network (VPN) over Ditto; and so on.
Internally, though, it is a thin wrapper enclosing communication channels.
See change data capture.
A bidirectional message-oriented data flow that runs on the top of a VPN connection. Following are the key characteristics:
- Offers reliable or lossy delivery characteristics.
- Opened in a mutual or client-server type relationship:
- Mutual relationship — Both client and server initiate the connection and the other responds.
- A client-server relationship — Either the client or server initiates the connection and the other responds.
- Protocol-agnostic; that is, it's independent of any specific communication protocol similar to the Transmission Control Protocol (TCP) and User Datagram Protocol (UDP).
- Consumed by services running on the mesh network to exchange data.
Also known as the mesh chooser, a stateful algorithm that decides which outgoing connections to make at any given moment.
This decision is based on the peers currently connected to, the peers whose advertisements have been detected by transports, and past failures.
The cryptographic root of trust for all identities and certificates within a Ditto-enabled app, which consists of a standard X.509 certificate authority (CA) for peer TLS certificates, including JSON web token (JWT) and in-band certificate signatures.
By knowing the public keys of the CA, an offline peer can verify the authenticity of another offline peer to determine if it is permitted to participate in the mesh and sync data.
A grouping of documents (the fundamental Ditto type) under a name. Loosely equivalent to a table in SQL terms. An app may have many collections.
An advanced class of data type designed to manage and replicate data changes in a way that allows multiple distributed peers to make updates concurrently without the need to reach a consensus to form a single meaningful value for merge.
For more information, see Ditto's blog post: An Inside Look at Ditto's Delta State CRDTs.
The Small Peer's implementation of the Ditto store.
Represents a structured data object that contains information to be transmitted from one peer to another.
A schema-flexible unit of data contained in a collection; analogous to a row in a table.
An SDK-layer type that targets some peer in the mesh for a Ditto Bus connection or similar. It opaquely wraps whatever identifying information is needed.
A custom OSPF-esque, shortest-path-first interior routing protocol that, using Presence data as a data source to determine routing, routes multihop packets.
See data transfer object.
A concept Ditto uses to identify whenever a peer has changed in some fundamental way that obsoletes our prior knowledge of them. In other words, it's a concept sync uses to determine a version of a peer's CRDT knowledge.
An epoch changes each time a peer performs a data eviction operation on the local Ditto store.
The process of purging data only from the local Ditto store so, although no longer available on the Small Peer where the eviction occurred, the data remains accessible elsewhere in the platform.
Eviction is important for use cases like cabin crew apps where data from the last flight is not needed on the next flight.
Formed by combining physical and logical clock portions, tracks when mutations occur to a CRDT or component thereof.
The physical clock portion is the local timestamp on a peer as unix milliseconds.
The logical clock portion is a number unique to each peer and increases by one with each change they make.
See hybrid logical clock.
A complex parameter passed to Ditto at startup which defines how you will authenticate yourself and verify who other peers are, and whether they’re using the same Application as you. Modes include OfflinePlayground, OnlinePlayground, OnlineWithAuthentication, and SharedKey. Some of these rely on the application having a common CA.
The data which the CA asserts about an participant in the mesh. It includes their Peer Key, site ID, permissions, and other arbitrary data returned by the authentication webhook such as email address, job title, and so on.
The part of the Big Peer that handles login requests, invoking Authentication Webhooks if required, and generating the cryptographic material for peers to authenticate each other by acting as the CA. In code, this basically means AuthServer.
A bespoke binary format that a Certificate Authority (CA) can sign to assert that a given Identity Data is valid and matches a particular key. Plays the role of a TLS X.509 certificate in non-TLS situations.
As per the Random Slicing algorithm, we think of the keyspace as the range 0 to 1. We take the capacity of the cluster, and divide 1 by it. This determines how much of the keyspace each partition owns.
An encrypted connection between two peers who are not directly connected, with traffic routed via intermediate peers. Used for multihop sync via Query Overlap Groups, and Ditto Bus.
A faster mode of Bluetooth LE transport. This is at a lower level/complexity than GATT. Platforms that support it have much faster speeds (~20 kB/s).
- In a CRDT context, metadata refers to the field by the same name in a CRDT Document. CRDT metadata tracks the actor which the current site uses when performing mutations, and tracks all the summaries for the CRDT.
- In a replication context, metadata is all the information which replication uses to keep track of the state of other peers in a Ditto mesh. A large part of this consists of CRDT metadata as above.
- In a storage context, metadata might refer to the internal abstraction which provides key-value storage optimized for metadata using any supported Ditto backend. These metadata stores can be created standalone, but every also comes with a special associated metadata store for tracking backend-specific info (transction numbers, version info, etc.).
A synchronous machine inside a Virtual Connection to a single remote peer, performing packet fragmentation and reassembly for all of the Physical Connections arriving from various transports.
A type of Identity where peers do not need unique credentials to log in and everybody has read and write access to everything.
Any mechanism for establishing direct WiFi connections between devices without needing a router in between. AWDL is one such technology for Apple devices, and WiFi Aware is another for Android devices.
A P-256 private key generated and persisted on every device that runs Ditto. This is the primary unique identifier for each peer and its ECDSA signatures are used to prove authenticity. It is unrelated to CRDT actor IDs.
A specification of which documents a given peer can read or write. This is presented as a list of specific collection names, and a sublist of query strings for each collection. Documents must match one of those DittoQL (DQL) queries to be readable or writable. If using an online authenticated Identity these can be locked down; other identity types have no CA and let everybody access anything.
Self-service website (https://portal.ditto.live/) used to create and manage Applications hosted on Ditto’s cloud.
The part of Ditto responsible for building a picture of all peers in the surrounding mesh, including rich peer info such as SDK version, device names, and which transports are active. This information is used as the basis for routing and can be visualized with the presence viewer.
A query that runs on connected peers that results in changes being sent back to the initial peer when changes are made to the remote peers' local SmallPeer database.
Handles Channels to provide a particular functionality. The most important example is the Replication service, which performs document sync.
A unique identifier for a peer which identifies any CRDT document mutations they author. Also has been used as unique peer identifier generally, but this is being phased out in favor of the Peer Key.
A query from the replication perspective, whereby one peer requests any data that matches this query from other peers to synchronize.
The physical transport mechanisms, such as Bluetooth Low Energy (LE), WebSocket, AWDL, LAN, and so on, that Ditto uses to discover other Ditto peers and establish a secure peer-to-peer mesh connection between them.
The unique identifier assigned to every write transaction indicates its order in the sequence of events, a causal relationship to other events, conflict resolution mechanisms to handle concurrency conflicts, and enable non-blocking reads.