Skip to main content

Writing

Ditto is a distributed database that optimizes for availability, which means that you can always write to your local database even if you are disconnected from the internet. Writes from all peers will synchronize and Ditto will resolve any conflicts.

Ditto uses a delta state CRDT, which means Ditto only replicates data if it has changed. This makes Ditto very fast.

Upserting

Ditto doesn't have a concept of "inserting" data, only "upserting" data. This means that each device must assume that their operations are modifying documents that may already exist on some other device in the network. This is because it is possible that the current device just hasn't synchronized that document yet.

Creating a document

If you do not supply an id, Ditto will create one for you and return it. Note that the id is immutable. This means that you cannot change the id once you have created the document.

do {    // upsert JSON-compatible data into Ditto    let docID = try ditto.store["people"].upsert([        "name": "Susan",        "age": 31    ])} catch {    //handle error    print(error)}

As an example, let's say we have a document in the people collection that looks like this:

{  "_id": "123abc",  "name": "Sam",  "age": 45,  "isOnline": false}

When you upsert a document, only the supplied fields will be modified. Existing fields are left unmodified. 

do {    // upsert JSON-compatible data into Ditto    let docID = try ditto.store["people"].upsert([        "_id": "abc123",        "name": "Susan",        "age": 31    ])    XCTAssertEqual(docID, "abc123")} catch {    //handle error    print(error)}

This results in the name and age changing, but isOnline is untouched:

{  "_id": "123abc",  "name": "Susan",  "age": 31,  "isOnline": false}

Default Data

Default Data can be thought of as "placeholder" data. Default Data is useful when your app needs to load "starter" data from an external data source, like from a backend API, on multiple devices. Ditto's approach to conflict resolution orders changes by time. In most situations, this leads to predictable behavior. However, if your application is upserting the same initial data into multiple devices, such as common data from a central backend API, this could result in overwriting later changes:

  1. Device A upserts a document {"firstName": "Adam"} at time = 0 after downloading from a central API.
  2. Device A updates the document to {"firstName": "Max"} at time = 1.
  3. Device B synchronizes with Device A receiving the document {"firstName": "Max"} at time = 2.
  4. Device B downloads the same document from the backend API {"firstName": "Adam"} and upserts at t = 3, which overwrites the previous change synced at time = 1

In the above example, both Device A and B want to preserve the change by Device A that occurred after downloading the common data. To do so, Ditto offers an additional parameter: isDefault.

warning

If you insert the same data on each peer without using the default data API, each device will accumulate unbounded metadata, which could cause the local disk space to grow infinitely.

do {    let docID = try ditto.store["people"].upsert([        "name": "Susan",        "age": 31    ], writeStrategy: .insertDefaultIfAbsent)} catch {    //handle error    print(error)}

Updating existing documents

Updating an existing document is different depending on the type being updated

Register

  • Set - sets value for a given key in the document
  • Remove - removes a value for a given key in the document

Counter

  • Replace with counter - will convert a number value for a given key into a counter
  • Increment - unlike a number, a counter allows for increment operations (decrement is performed by incrementing by a negative increment) and these operations will converge

Map

  • Set - sets value for a given key in the map
  • Remove - removes a value for a given key in the map
do {    let docID = try ditto.store["people"].upsert([        "name": "Frank",        "age": 31,        "ownedCars": DittoCounter()    ])
    ditto.store["people"].findByID(docID).update { mutableDoc in        mutableDoc?["age"] = 32        mutableDoc?["ownedCars"].counter?.increment(by: 1)    }} catch {    //handle error    print(error)}

What is the difference between upsert and update?

  • upsert will create updates to all fields even if those fields already exist. Every call to upsert will create new metadata for each field, even if the field has not changed on the local store.
  • update allows you to query the field locally and make changes to each individual field only if that field should change. This allows you to create fine-grained changes to a single document, or make batch changes to a set of documents. Using the update API means that you will only synchronize the minimum data necessary to enforce all peers to converge on one view of the data.

Batching multiple updates in a single transaction

A write transaction allows a device to perform multiple operations within a single database call, that will be synchronized at the same time to other devices. You can perform insert, update, remove or evict operations on multiple collections more efficiently using this method.

warning

You cannot combine remove() and upsert() on the same document in the same write transaction. This is because you cannot update a document that has been removed.

ditto.store.write { transaction in    let cars = transaction.scoped(toCollectionNamed: "cars")    let people = transaction.scoped(toCollectionNamed: "people")    let docId = "abc123"    do {        try people.upsert(["_id": docId, "name": "Susan"] as [String: Any?])        try cars.upsert(["make": "Ford", "color": "red", "owner": docId] as [String: Any?])        try cars.upsert(["make": "Toyota", "color": "black", "owner": docId] as [String: Any?])    } catch (let err) {      print(err.localizedDescription)    }    people.findByID(docId).evict()}

Multi threading on updating list of contents

When writing many documents to a small peer, you can start the transaction asynchronously to avoid blocking the main thread. For example, in Swift you can use DispatchQueue.global. The syntax is different for other languages but the concept is the same.

DispatchQueue.global(qos: .default).async {
    ditto.store.write { transaction in
        let scope = transaction.scoped(toCollectionNamed: "passengers-\(thisFlight)")
    // Loop inside the transaction to avoid writing to database too frequently        self.passengers.forEach {            scope.upsert($0.dict)        }    }}

Blocked transactions

warning

You cannot start a write transaction within an update clause or a write transaction.

At any given time, there can be only one write transaction. Any subsequent attempts to open another write transaction will become blocked and not progress until the existing write transaction finishes. The process of unblocking is automatic and you don’t need to write any code to handle this.

The following code will create a deadlock which never resolves itself.

// Start a write transaction:ditto.store["people"].findByID(docID).update { mutableDoc in  // Start a write transaction _within_ a write transacton.  // !! Deadlocks !!  let docID = try! ditto.store["settings"].upsert([      "_id": "abc123",      "preference": 31,  ])   // ...}

This might manifest itself in the logs as:

LOG_LEVEL: Waiting for write transaction (elapsed: XXs), originator=User blocked_by=User

If the transaction never unblocks and the log messages at ERROR level continue forever - you have a strong indication that there’s a deadlock and should investigate the application code.