CRUD Operations
Ditto's query engine and API methods allow you to perform traditional create, read, update, and delete (CRUD) operations. In Ditto, you use queries to read data from local Ditto stores, and various API methods to perform create, update, and delete operations.
This topic provides an overview of the operations you can perform in Ditto. For more comprehensive information, how-to instructions, and code snippets, see any of the following:
For optimization techniques and common implementation errors to avoid, see Best Practices for CRUD.
If you're using Ditto's optional Big Peer cloud deployment, for an overview of the HTTP API for data egress, as well as the LiveQuery Webhook for real-time bidirectional change data capture (CDC) with a third-party database, see the Platform Manual > Big Peer.
When interacting with the platform, you'll notice that Ditto's query language is quite similar to what you'd construct in most if statements.
You'll use familiar techniques like using an update statement to modify existing data, dot notation to navigate levels in a hierarchy, upon other intuitive query condition operators typical for performing standard CRUD.
The Ditto SDK provides a comprehensive set of methods and functions to facilitate data interaction and perform a wide range of operations in your app:
- For read operations, use the find and observeLocal methods.
- For modifications, use the update, upsert, evict, and delete methods.
At a high-level, queries operate on collections rather than individual documents. This approach to data management enhances retrieval efficiency and ensures seamless data replication among connected peers.
For an overview of the syntax you can use to filter your queries, see Ditto Basics > Syntax .
The following table provides an overview of the three types of queries you use to fetch documents and set up listeners in your app:
Type | Description | Use Case |
Local query |
| You want to quickly search or retrieve specific data, such as an attachment, recent activities, and so on, that is stored within a particular local Ditto store. |
Live query |
| You want to display real-time updates to your end user any time you receive an incoming remote change sent from another connected peer. |
Replication query |
When these conditions are observed within the mesh, Ditto automatically triggers a subscribe callback object that executes your subsequent processes and actions in your app. | You want to receive notification whenever changes are made by any peer participating in the mesh, so you can respond as appropriate to ensure real-time collaboration with remote peers. |
There are two methods you can use to retrieve documents stored to your local Ditto store:
- find— Returns all local documents that match your criteria.
- findById— Returns only the local document assigned the _id you provide.
If you're interested in actively listening for changes and events in the data so you can react accordingly, convert the local query to a live query by adding the observeLocal method in the enclosure.
When working with data subject to change at runtime, instead of building or interpolating query strings, you can query with runtime variables through an $args map object.
For more information, see Finding and Observing.
Unlike theobserveLocal method, which establishes a local listener, the subscribe method establishes a mesh-wide listener so you can stay up-to-date with data changes happening across remote peers.
Once you've initiated data replication by invoking the startSyncmethod, you can set up a subscription to signal which data you're interesting in automatically receiving updates for, as well as the subsequent actions for Ditto to perform in your app.
Given this subscription-driven approach, you eliminate the need to perform traditional resource-intensive and time-consuming methods of data retrieval in your app, such as repeatedly executing HTTP requests and polling a queue for data updates.
For more information, see Finding and Observing.
There are two ways you can write changes to Ditto: upsert the changes or update the changes. Choosing which to use depends on what you're trying to achieve.
- With the update method:
- You can query a property locally, and then make changes to each individual property only if that property has changed.
- You can create fine-grained changes to a single document or make batch changes to a set of documents.
- Due to Ditto’s conflict-free eventual consistency data model, there is no concept of “insert”; instead, each device assumes that their write transaction already exists somewhere in the broader Ditto system. Therefore, with the upsert method:
- You can upsert only the fields within the document that have changed.
- If all of the fields in the document are new, Ditto creates an entirely new document object and, unless manually supplied, generates and assigns that document a unique identifier, the _id.
For more information, see Upserting and Updating.
There are two approaches removing data in Ditto, and the decision of which to choose depends on your specific requirements and use case:
Invoking the remove method results in irreversible data loss.
Once a document is removed, it is permanently eliminated and can never be restored.
- remove — Permanently deletes one or more specified documents from a collection throughout the entire Ditto platform.
- evict — Physically eliminates the specified documents from its Ditto store to free up local memory or on-disk storage, prevents any pending changes affected by the eviction from being sent across the mesh network, and leaves the documents stored in remote Ditto stores in tact so the remain accessible to all connected peers.
- Soft-delete pattern — Adds a "forget" flag to the specified documents indicating that, although still accessible to remote peers for inspection and viewing, write operations, are restricted.
For more information, see Evicting and Removing.
You can model relationships between your data using foreign‑key relationships, key-value pair relationships by way of embedded CRDTmaps , as well as CRDT arrays.
Ditto does not support nesting documents within documents.
The following table provides a complete overview of the different relationships you can form in Ditto, as well as a brief description, list of possible approaches you can take, and links to related content:
Relationship | Description | Approaches |
One-to-many | Associates a parent element with children elements to establish a hierarchy. |
|
Many-to-many | Associates multiple entities in one collection with multiple entities in another collection. |
|
Many-to-one | Associates two or more collections, where one collection refers to the primary key of another collection to create a meaningful relationship between the datasets. |
|
You can represent a one-to-many or many-to-one relationship between two or more collections, in which one collection refers to the primary key of another collection to create a meaningful relationship between the datasets.
For example, the following snippet demonstrates a foreign-key relationship between documents in the cars and people collections, in which the reference to susanId serves as the foreign key establishing a relationship between cars and people:
A key-value relationship establishes a parent-child hierarchy between embedded data elements where the key serves as the parent and the associated values the children to link related data items as well as organize them in a structure that enhances efficiency of lookup.
When managing data that requires unique identifiers and relationships, instead of using an array to encode your data, use a map with unique string keys and object values instead.
To help you make the most out of Ditto's capabilities and ensure a frictionless experience for your end users, implement optimization strategies to enhance efficiency, speed, and precision of your operations.
This article offers a variety of optimization techniques that you can implement to improve the performance of your application.
To reduce the number of network roundtrips needed for each individual operation, ensure data consistency, and simplify your code, use the write method to perform multiple upsert, update, remove, and evict operations across two or more document collections within a single Ditto call.
Additionally, you can incorporate logic that facilitates the retrieval of a document's current state and, when necessary, initiate a conditional update operation based on that state.
Initiating a write transaction within another write transaction can cause Ditto to hang indefinitely, resulting in a deadlock.
If a deadlock situation occurs, both transactions are unable to complete, the log messages at the ERROR level persist in a forever loop, and you’ll need to review your app’s codebase.
Do not combine remove and upsert functions for the same document within a single write transaction.
If you remove a document as part of the operation, potential concurrency conflicts may occur for it is impossible to update a document that no longer exists in Ditto.
The following snippet demonstrates a batch operation in which two write transactions are contained in a single transaction enclosure:
All write transactions are event-driven, or asynchronous, but locally scoped by default.
When you have many documents to write to a peer, you can initiate a transaction in a way that avoids blocking or slowing down the main thread of your app by starting the transaction asynchronously; as in, it can occur concurrently with other Ditto operations.
For example, you can use DispatchQueue.global,as follows:
To avoid deadlocks, it is important to ensure that nested write transactions do not initiate within the scope of another write transaction.
A deadlock is a situation that occurs when a conflicting code pattern blocks two or more threads or processes from proceeding with their execution because each is waiting for a resource that the other holds (and never receives), resulting in a standstill of circular dependency.
In order to prevent potential conflicts that could arise from concurrent write operations, do not initiate a new write transaction to modify data until the current clause or write transaction is complete.
If you have a queued write transaction that remains blocked, the following message appears in your logs:
For more precise representations of date strings in queries for comparison operations, use the International Organization for Standardization (ISO) 8601 standard format. For an overview of operators and string functions and formats, see Ditto Basics > Query Tools.
ISO-8601 is an international standard for representing dates, times, and durations. The format adheres to specific patterns, such as the following standard formatting for date and time:
Date | Time |
YYYY-MM-DD | HH:mm:ss |
For more information about ISO 8601, see the official ISO documentation > ISO 8601 Date and time format.
When invoking the find and observeLocal methods to query values that may change dynamically during runtime, you can declare a top-level args variable that defines the dynamic values, and then pass the the $args identifier within your query conditions. That way, the query engine separates your query logic from the data so you can easily define and pass values as needed to adapt without having to change the query structure itself.
Using strings to filter dynamic data may impact the maintainability, security, and performance of your app by introducing various issues such as syntax errors in your code.
The Ditto SDK offers write strategies that you can incorporate into your code to control how Ditto makes changes and resolves any potential concurrency conflicts, helping you manage your data more effectively.
A write strategy is effectively a set of instructions that tell Ditto how to handle specific changes and conflicts when you are updating and upserting data to the local Ditto store.
The Ditto SDK provides the following write strategies that you can use when performing update and upsert operation in your app:
The insertIfAbsent write strategy ensures that a new document with the same _id value is inserted only if there is no existing document with that ID in the given collection.
If a document with the same ID already exists, the upsert operation does not perform any changes and instead maintains the existing document.
The insertIfAbsent write strategy is useful for scenarios where you want to add new fields only when they are missing; if the fields exists, but the values are changed, the upsert operation does not update the changed fields.
The insertDefaultIfAbsent write strategy, when implemented, ensures that the given document upserts only if a document with the same ID does not already exist in the local Ditto store.
If the document does not exist and it is is upserted, it will be timestamped with a value of 0, making it appear as if it was performed before any other action. This approach helps manage concurrency conflicts and ensure that the data is inserted only when necessary.
The following snippet shows an example of the writeStrategy: .insertDefaultIfAbsent parameter passed as an argument to the.upsert() method:
When upserting initial data from an external source to all peers at app startup, you must pass the enumeration value: writeStrategy: 'insertDefaultIfAbsent' as an argument in the upsert function.
Initial data is common data like sample chat messages from a central backend API that is accessible to end users at app startup.
Failing to indicate initial upserted data may risk future overwrites due to unbounded metadata, impacting storage, app performance, and stability.
For example, consider the concurrency conflict scenario and resolution outcome: Both Device A and Device B must retain the the change that occurred after Device B downloaded the common data — specifically, Device A’s change executed in step 2 at time = 1:
Device A downloads common data from a central API, and then upserts it as a document at time = 0:
Device A then upserts changes to the document at time = 1:
Device B syncs with Device A at time = 2:
Device B downloads the same common data, but from a backend API instead, and then upserts it as a document at time = 3.
The concurrent conflict that results from Device B’s upsert results in the previous upsert synced at time = 1 to be overwritten instead of being preserved:
