MAP
An alternative approach to using a JSON object as a REGISTER to represent and organize data in a hierarchical structure within a document is to use a MAP.
The MAP type uses the add-wins approach to resolving concurrency conflicts.
So instead of choosing a single definitive value to merge and distribute across peers like the last-write-wins strategy utilized by the REGISTER and ATTACHMENT data types, choose them all — even if the data is already contained within the MAP object.
The MAP data type in DQL forms a tree-like, parent-child relationship within a document's dataset. This is similar to a JSON object, which functions as a single unit. However, a MAP allows:
- Nesting of various DQL data types:REGISTER, MAP, and ATTACHMENT as values for its keys.
- Each key-value pair within a MAP can be modified independently without affecting other pairs.
- Utilization of an add-wins merge strategy for conflict resolution. When conflicting changes occur during sync, all conflicting changes are retained rather than converging on only one. This ensures that no data is lost.
The response structure of a MAP is a JSON-like object; however, each field in a MAP has a corresponding data type. This means that each field within a MAP has an independent merge definition the same as fields on the top-level document.
The MAP type is useful when creating a list of non-conflicting items and updating items over time within a document.
The following snippet demonstrates a Ditto document with an embedded MAP my_map:
The REGISTER is the default data type in Ditto; therefore, there is no need for explicit declaration — it's implicitly assumed.
However, to use a MAP (and ATTACHMENT) data type, you must explicitly declare it within your DQL statement using type definition syntax.
A type definition is a description informing Ditto that the given set of key-value pairs embedded in the document as a nested structure is distinct from the default REGISTER type. This definition specifies how Ditto should handle the dataset within that nested structure.
To declare a data type a MAP, you use the COLLECTION identifier prefix, followed by the collection name enclosed in parentheses alongside a type definition. All done inline as follows:
For more information and how-to instructions, see Ditto Query Language > Types and Definitions
The MAP type has an add-wins merge behavior. This means that the adding or updating of a field always wins over the removal (tombstone) of the same field when performing a merge across two peers.
If multiple fields are added across Small Peer devices, the result will be a MAP with all added fields. This is because each field within a MAP is associated with a distinct data type; that is, each can be represented as a REGISTER, MAP, or an ATTACHMENT.
That means, unlike a JSON blob object, which acts as a REGISTER functioning as a single unit, the merge strategy for each field within a MAP object depends on each of their data type representation.
In the following example, USER 1 inserts item1 and USER 2 inserts item2 to an items MAP:
The resulting document contains an items MAP with both item1 and item2:
The nested JSON object functions as a REGISTER and closely resembles a nested MAP structure within a document. However, their merge strategies and supported types for nested values are significantly different.
Understanding these distinctions is crucial when selecting the appropriate data type for expressing your data structure.
For example, consider a document with a properties field of the REGISTER data type storing information about a car in an embedded JSON object.
You want to change only the color field for a car identified by document ID 123 to blue.
Since a REGISTER acts as a single object, to update just the color, you'd need to update the entire object by providing the entire set of nested key-value pairs, including the modified color , as follows:
However, if the properties field is of the MAP data type instead, to update the color without altering the other nested key-value pairs in the set, assign the properties.color field to blue as follows:
If a tombstone (deletion flag) and a modification (INSERT) operation occur across two peers simultaneously, the resulting state depends on the causal order of these changes.
Causal order refers to the logical sequence in which events occur, considering the cause-and-effect relationship between them.
Modifications to the value of fields within a MAP do not count as modifications to the MAP itself.
Following is a truth table showing "if a MAP is deleted" based on "if the peers are connected" and "when the tombstone operation occurs relative to the modification."
PEER Status | Tombstone First | Tombstone Last |
|---|---|---|
Connected | FALSE | TRUE |
Disconnected | FALSE | FALSE |
Syncing large documents can significantly impact sync performance.
Consider using an ATTACHMENT instead of a regular document when handling a deeply embedded or large document object.
When internet connection is limited or using only Bluetooth Low Energy (LE) to sync across connected peers, syncing documents containing large amounts of data is slow.
For instance, when using only Bluetooth LE to sync a document of a typical size, the rate of replication is 20 KB per second maximum. Given this, a document of 250 KB or larger may require 10 seconds or more to sync for the first time between Small Peer devices.
A slow replication rate results in a loading spinner displayed to your end users until the sync process fully completes. This is due to the callback being unable to render the returned data; the end-to-end sync process requires that documents be broken down into smaller parts before being synced across the mesh, and then, until the client receives all of the smaller parts and reconstructs them, the document is not returned.
Instead of using a single document to encode large datasets, use a series of smaller documents.
- If a document exceeds 250 KB, a stdout warning prints.
- Any documents exceeding 5 MB do not sync to other peers.