Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.ditto.live/llms.txt

Use this file to discover all available pages before exploring further.

Overview

This guide will help you successfully migrate your Ditto Kotlin Android application from the legacy query builder APIs to the modern DQL (Ditto Query Language). After reviewing this documentation, you’ll understand how to convert method chaining patterns to DQL syntax and systematically update your data operations.

AI Agent Prompt

Use this prompt when working with an AI coding assistant to migrate your Ditto Kotlin Android app from legacy query builder to DQL.
I need help migrating a Ditto Kotlin Android application from the legacy query builder APIs to modern DQL (Ditto Query Language). This migration involves converting method chaining patterns to SQL-like DQL syntax.

CRITICAL RULES:
1. All query builder method chains (.collection().find()) must be replaced with ditto.store.execute() using DQL
2. Use parameterized queries with :paramName syntax - NEVER string interpolation
3. Counter operations must use PN_INCREMENT BY in APPLY clause - do NOT initialize counter fields
4. Sync subscriptions must use ditto.sync.registerSubscription() instead of .find().subscribe()
5. observeLocal must be replaced with registerObserver + DittoDiffer

---

CORE MIGRATION AREAS:

1. QUERY SYNTAX MIGRATION

BEFORE (Legacy Query Builder):
```
ditto.store.collection("cars")
    .find("color == $0", "red")
    .exec()
```

AFTER (DQL):
```
ditto.store.execute(
    "SELECT * FROM cars WHERE color = :color",
    mapOf("color" to "red")
)
```

2. INSERT OPERATIONS

BEFORE (Legacy Query Builder):
```
ditto.store.collection("cars")
    .upsert(mapOf("_id" to id, "color" to "blue"))
```

AFTER (DQL):
```
ditto.store.execute(
    "INSERT INTO cars DOCUMENTS (:car)",
    mapOf("car" to carData)
)
```

3. UPDATE OPERATIONS

BEFORE (Legacy Query Builder):
```
ditto.store.collection("cars")
    .findById(id)
    .update { doc -> doc["color"].set("green") }
```

AFTER (DQL):
```
ditto.store.execute(
    "UPDATE cars SET color = :color WHERE _id = :id",
    mapOf("color" to "green", "id" to id)
)
```

4. DELETE OPERATIONS

BEFORE (Legacy Query Builder):
```
ditto.store.collection("cars").findById(id).remove()
```

AFTER (DQL):
```
ditto.store.execute(
    "DELETE FROM cars WHERE _id = :id",
    mapOf("id" to id)
)
```

5. EVICTION OPERATIONS

BEFORE (Legacy Query Builder):
```
ditto.store.collection("cars").findById(id).evict()
```

AFTER (DQL):
```
ditto.store.execute(
    "EVICT FROM cars WHERE _id = :id",
    mapOf("id" to id)
)
```

6. COUNTER OPERATIONS (PN_COUNTER)

BEFORE (Legacy Query Builder):
```
ditto.store.collection("cars")
    .findById(id)
    .update { doc ->
        doc["numUpdates"].counter?.increment(amount = 1.0)
    }
```

AFTER (DQL with PN_INCREMENT):
```
ditto.store.execute(
    "UPDATE cars APPLY numUpdates PN_INCREMENT BY :increment WHERE _id = :id",
    mapOf("increment" to 1, "id" to id)
)
```

IMPORTANT: Do NOT initialize counter fields in documents:
```
// WRONG - Creates a register, not a counter
mapOf("counter" to 0)

// CORRECT - Omit counter field, it's created on first PN_INCREMENT
mapOf("_id" to id, "color" to "blue")
```

7. DOCUMENT FIELD ACCESS MIGRATION

BEFORE (Legacy Query Builder):
```
val document: DittoDocument = collection.findById(res.id).exec()
val color = document.value["color"] as String
```

AFTER (DQL):
```
val result: DittoQueryResult = ditto.store.execute(dqlString)
val item: DittoQueryResultItem = result.items.first()
val color = item.value["color"] as String
```

8. LIVE QUERY MIGRATION (observeLocal → registerObserver)

BEFORE (Legacy observeLocal):
```
liveQuery = collection.find("_id.locationId == '${Constants.locationId}'")
    .observeLocal { docs, event ->
        when (event) {
            is DittoLiveQueryEvent.Update -> {
                adapter.delete(event.deletions)
                adapter.insert(event.insertions)
                adapter.update(event.updates)
            }
            is DittoLiveQueryEvent.Initial -> {
                adapter.setInitialCars(docs)
            }
        }
    }
```

AFTER (DQL with registerObserver + DittoDiffer):
```
import live.ditto.DittoDiffer

class MainFragment : Fragment() {
    private var differ = DittoDiffer()
    private var previousDocumentIds: MutableList<String> = mutableListOf()

    private fun startLiveQuery() {
        val observer = ditto.store.registerObserver(
            query = "SELECT * FROM cars WHERE _id.locationId = :locationId",
            parameters = mapOf("locationId" to Constants.locationId)
        ) { queryResult ->
            val diff = differ.diff(queryResult.items)

            // Extract current document IDs and dematerialize items
            val currentDocumentIds = queryResult.items.map { item ->
                val id = item.value["_id"] as? String ?: "unknown"
                item.dematerialize() // IMPORTANT: Free memory
                id
            }

            // Handle deletions using stored IDs from previous emission
            for (index in diff.deletions) {
                val deletedId = previousDocumentIds[index]
                // Handle deletion
            }

            // Handle insertions using current IDs
            for (index in diff.insertions) {
                val insertedId = currentDocumentIds[index]
                // Handle insertion
            }

            // Handle updates using current IDs
            for (index in diff.updates) {
                val updatedId = currentDocumentIds[index]
                // Handle update
            }

            previousDocumentIds = currentDocumentIds.toMutableList()
        }
    }
}
```

9. SYNC SUBSCRIPTIONS MIGRATION

BEFORE (Legacy Query Builder):
```
val subscription = ditto.store.collection("cars")
    .find("color == $0", "red")
    .subscribe()
```

AFTER (DQL):
```
val subscription = ditto.sync.registerSubscription(
    "SELECT * FROM cars WHERE color = :color",
    mapOf("color" to "red")
)
```

---

COMMON PITFALLS TO AVOID:

1. DQL Syntax Errors
   - Use :paramName for parameters, not $0 or string interpolation

2. Missing Parameter Binding
   - NEVER use string interpolation in queries
   - Always use parameterized queries with mapOf()

3. Counter Type Errors
   - Do NOT initialize counter fields with DittoCounter() or numbers
   - Use PN_INCREMENT BY in APPLY clause
   - Pass negative values for decrements

4. Memory Leaks in Observers
   - Always call item.dematerialize() after extracting data
   - Store only IDs, not full query items

5. Attachment Handling
   - Use ATTACHMENT annotation: "(image ATTACHMENT)"
   - Create attachments with ditto.store.newAttachment()

---

MIGRATION CHECKLIST:

Search for these legacy patterns and replace:
- [ ] .collection(" → ditto.store.execute("SELECT * FROM 
- [ ] .find( → Convert to DQL WHERE clause with parameters
- [ ] .findById( → Convert to DQL WHERE _id = :id
- [ ] .upsert( → Convert to DQL INSERT INTO 
- [ ] .update( → Convert to DQL UPDATE SET
- [ ] .remove( → Convert to DQL DELETE FROM
- [ ] .evict( → Convert to DQL EVICT FROM
- [ ] .counter?.increment( → Convert to PN_INCREMENT BY in APPLY clause
- [ ] DittoCounter() → Remove initialization, use PN_INCREMENT
- [ ] .observeLocal( → Convert to registerObserver + DittoDiffer
- [ ] .subscribe() → Convert to ditto.sync.registerSubscription()
- [ ] DittoDocument → DittoQueryResultItem
- [ ] DittoSubscription → DittoSyncSubscription

---

Please help me convert all legacy query builder patterns in my codebase to DQL syntax. Focus on:
1. Maintaining the same functionality
2. Using proper parameterized queries
3. Handling counter operations correctly with PN_INCREMENT
4. Implementing proper memory management in observers
5. Converting all sync subscriptions to DQL

Start by identifying all uses of .collection() in my codebase and systematically converting each one to the appropriate DQL pattern.

Key API Changes Reference

Query Syntax Migration

Legacy Query Builder → DQL Query Syntax
ditto.store.execute(
    "SELECT * FROM cars WHERE color = :color",
    mapOf("color" to "red")
)

Data Operations Migration

Legacy Query Builder → DQL Insert Operations
ditto.store.execute(
    "INSERT INTO cars DOCUMENTS (:car)",
    mapOf("car" to carData)
)
Legacy Query Builder → DQL Update Operations
ditto.store.execute(
    "UPDATE cars SET color = :color WHERE _id = :id",
    mapOf("color" to "green", "id" to id)
)
Legacy Query Builder → DQL Delete Operations
ditto.store.execute(
    "DELETE FROM cars WHERE _id = :id",
    mapOf("id" to id)
)
Legacy Query Builder → DQL Eviction Operations
// Evict by ID
ditto.store.execute(
    "EVICT FROM cars WHERE _id = :id",
    mapOf("id" to id)
)

// Evict all matching documents
ditto.store.execute(
    "EVICT FROM cars WHERE color = :color",
    mapOf("color" to "red")
)

Document Field Access Migration

Legacy Query Builder → Modern Field Access
val result: DittoQueryResult = ditto.store.execute(dqlString)
val item: DittoQueryResultItem = result.items.first()
val color = item.value["color"] as String
Legacy Query Builder → Modern Document Conversion
// Use an extension function — close() each item after extracting data
fun List<DittoQueryResultItem>.toCars(): List<Car> = map { item ->
    val car = Car(
        id = item.value["_id"] as String,
        color = item.value["color"] as String
    )
    item.close() // Release native memory
    car
}

// Usage:
val cars = result.items.toCars()

Observer Migration

Legacy Query Builder → DQL Store Observer Migration
import live.ditto.DittoDiffer

class MainFragment : Fragment() {
    private var differ = DittoDiffer()
    private var previousDocumentIds: MutableList<String> = mutableListOf()

    private fun startLiveQuery() {
        // Remove the old live query
        liveQuery?.close()

        // Register new observer with Differ
        val observer = ditto.store.registerObserver(
            query = "SELECT * FROM cars WHERE _id.locationId = :locationId",
            parameters = mapOf("locationId" to Constants.locationId)
        ) { queryResult ->
            // Use Differ to compute the diff
            val diff = differ.diff(queryResult.items)

            // Extract current document IDs and dematerialize items
            val currentDocumentIds = queryResult.items.map { item ->
                val id = item.value["_id"] as? String ?: "unknown"
                // Dematerialize to free memory - this is important!
                item.dematerialize()
                id
            }

            // Handle deletions using stored IDs from previous emission
            for (index in diff.deletions) {
                val deletedId = previousDocumentIds[index]
                // Handle deletion of document with ID: deletedId
                println("Deleted car with ID: $deletedId")
            }

            // Handle insertions using current IDs
            for (index in diff.insertions) {
                val insertedId = currentDocumentIds[index]
                // Handle insertion of document with ID: insertedId
                println("Inserted car with ID: $insertedId")
            }

            // Handle updates using current IDs
            for (index in diff.updates) {
                val updatedId = currentDocumentIds[index]
                // Handle update of document with ID: updatedId
                println("Updated car with ID: $updatedId")
            }

            // Store only the document IDs for next callback - no live references!
            previousDocumentIds = currentDocumentIds.toMutableList()

            // Update UI on main thread if needed
            requireActivity().runOnUiThread {
                // Update your adapter or UI components here
                // Note: You'll need to implement proper UI updates based on diff
            }
        }

        // Store the observer reference for cleanup later
        this.observer = observer
    }
}
Performance Consideration: DQL observers provide more advanced return results including aggregates and projections. This requires more database full scans to ensure consistent results compared to the legacy query builder.Use indexes on query fields to maintain and improve observer performance. Indexes ensure your observers remain functional with optimal query performance.
Best Practice: Create Indexes for Observer Queries
// Create index on frequently queried fields
ditto.store.execute("""
    CREATE INDEX idx_cars_locationId
    ON cars (_id.locationId)
""")

// Then register observer - queries will use the index
val observer = ditto.store.registerObserver(
    query = "SELECT * FROM cars WHERE _id.locationId = :locationId",
    parameters = mapOf("locationId" to Constants.locationId)
) { queryResult ->
    // Process results
}
For more information on creating and managing indexes, see the DQL Indexing documentation.

Sync Subscriptions Migration

Legacy Query Builder → DQL Sync Subscriptions Subscribe with Query
val subscription = ditto.sync.registerSubscription(
    "SELECT * FROM cars WHERE color = :color",
    mapOf("color" to "red")
)
Subscribe with Parameters
val subscription = ditto.sync.registerSubscription(
    "SELECT * FROM cars WHERE _id.locationId = :locationId",
    mapOf("locationId" to Constants.locationId)
)
Multiple Subscriptions
val subscriptions = mutableListOf<DittoSyncSubscription>()
subscriptions.add(
    ditto.sync.registerSubscription(
        "SELECT * FROM cars WHERE color = :color",
        mapOf("color" to "red")
    )
)
subscriptions.add(
    ditto.sync.registerSubscription(
        "SELECT * FROM cars WHERE year > :year",
        mapOf("year" to 2020)
    )
)
Cancel Subscription
subscription.cancel()
Subscribe to All Documents
val subscription = ditto.sync.registerSubscription(
    "SELECT * FROM cars"
)

Counter Type Migration

PN_COUNTER is the DQL equivalent of the legacy DittoCounter type. When migrating counter operations from the legacy query builder’s .counter?.increment() method, use PN_INCREMENT BY in the APPLY clause. This maintains full compatibility with existing counter data created by DittoCounter.
Counter Increment
ditto.store.execute(
    "UPDATE cars APPLY numUpdates PN_INCREMENT BY :increment WHERE _id = :id",
    mapOf("increment" to 1, "id" to id)
)
Counter Decrement
ditto.store.execute(
    "UPDATE cars APPLY viewCount PN_INCREMENT BY :decrement WHERE _id = :id",
    mapOf("decrement" to -1, "id" to id)
)
Initialize Counter in Document
// Counter fields are automatically created on first PN_INCREMENT use
ditto.store.execute(
    "INSERT INTO cars DOCUMENTS (:car)",
    mapOf("car" to mapOf(
        "_id" to id,
        "color" to "blue"
        // Do NOT initialize counter fields - they are created on first PN_INCREMENT
    ))
)

// Then use PN_INCREMENT with APPLY clause to create and increment the counter
ditto.store.execute(
    "UPDATE cars APPLY numUpdates PN_INCREMENT BY 1 WHERE _id = :id",
    mapOf("id" to id)
)
Multiple Counter Operations
ditto.store.execute(
    """UPDATE cars
       APPLY likes PN_INCREMENT BY :likeIncrement,
             dislikes PN_INCREMENT BY :dislikeDecrement,
             views PN_INCREMENT BY :viewIncrement
       WHERE _id = :id""",
    mapOf(
        "likeIncrement" to 1,
        "dislikeDecrement" to -1,
        "viewIncrement" to 1,
        "id" to id
    )
)

Attachment Operations with DQL

Attachment Creation and Storage
// Create attachment using store
val attachment = ditto.store.newAttachment(
    inputStream = fileInputStream,
    metadata = metadata
)

// Store attachment with DQL
ditto.store.execute(
    "INSERT INTO COLLECTION cars (image ATTACHMENT) DOCUMENTS (:doc)",
    mapOf("doc" to docWithAttachment)
)
Attachment Fetching
// Fetch attachment with progress callback
val fetchResult = ditto.store.fetchAttachment(
    token = attachmentToken,
    onFetchProgress = { progress ->
        updateProgress(progress.downloadedBytes, progress.totalBytes)
    }
)

Performance Enhancements

Indexes for Improved Query Performance

DQL observers and queries benefit significantly from proper indexing. When migrating from the legacy query builder to DQL, creating indexes on frequently queried fields is essential for maintaining optimal performance. Why Indexes Matter for DQL:
  • DQL observers support advanced features like aggregates and projections
  • These advanced features require full database scans to ensure consistent results
  • Indexes dramatically reduce query execution time by avoiding full scans
  • Combining indexes with observers provides better performance than legacy query builder
Creating Indexes:
// Create index on single field
ditto.store.execute("""
    CREATE INDEX idx_cars_color
    ON cars (color)
""")

// Create compound index on multiple fields
ditto.store.execute("""
    CREATE INDEX idx_cars_color_year
    ON cars (color, year)
""")

// Create index on nested field
ditto.store.execute("""
    CREATE INDEX idx_cars_location
    ON cars (_id.locationId)
""")
Best Practices:
  1. Create indexes on fields used in WHERE clauses
  2. Create indexes before registering observers for those queries
  3. Use compound indexes for queries with multiple filter conditions
  4. Monitor query performance and add indexes as needed
For comprehensive information on indexing strategies, syntax, and best practices, see the DQL Indexing documentation.

Common Pitfalls to Avoid

1. DQL Syntax Errors

Use :paramName for parameters, not $0 or string interpolation.
// ❌ Wrong: String interpolation
val color = "red"
ditto.store.execute("SELECT * FROM cars WHERE color = '$color'")

// ✅ Correct: Using :paramName with mapOf()
ditto.store.execute(
    "SELECT * FROM cars WHERE color = :color",
    mapOf("color" to "red")
)

2. Missing Parameter Binding

NEVER use string interpolation in queries. Always use parameterized queries with Map<String, Any>.
// ❌ Wrong: Direct string interpolation
val locationId = "loc_123"
ditto.store.execute(
    "SELECT * FROM cars WHERE _id.locationId = '$locationId'"
)

// ✅ Correct: Parameterized query
ditto.store.execute(
    "SELECT * FROM cars WHERE _id.locationId = :locationId",
    mapOf("locationId" to locationId)
)

3. Counter Type Errors

Use COUNTER annotation in collection definitions. Do NOT use SET with COUNTER fields. Use APPLY with PN_INCREMENT BY. Pass negative values for decrements.
// ❌ Wrong: Initializing counter with a number (creates REGISTER, not COUNTER)
ditto.store.execute(
    "INSERT INTO items DOCUMENTS (:doc)",
    mapOf("doc" to mapOf("counter" to 0, "_id" to id))
)

// ❌ Wrong: Using SET on counter field
ditto.store.execute(
    "UPDATE items SET counter = 5 WHERE _id = :id",
    mapOf("id" to id)
)

// ✅ Correct: Use PN_INCREMENT BY with APPLY clause (creates counter on first use)
ditto.store.execute(
    "UPDATE COLLECTION items (counter COUNTER) APPLY counter PN_INCREMENT BY :value WHERE _id = :id",
    mapOf("value" to 1, "id" to id)
)

// ✅ Correct: Decrement by passing negative value
ditto.store.execute(
    "UPDATE items APPLY counter PN_INCREMENT BY :value WHERE _id = :id",
    mapOf("value" to -1, "id" to id)
)

4. Memory Management with Observers

Always call item.dematerialize() after extracting data. Use DittoDiffer to compute diffs. Use indexes for improved memory and performance.
// ❌ Wrong: Storing QueryResultItems outside callback scope
var items: List<DittoQueryResultItem> = emptyList()
val observer = ditto.store.registerObserver(
    query = "SELECT * FROM cars"
) { result ->
    items = result.items  // Holds native memory
}

// ✅ Correct: Extract data and dematerialize immediately
class MainFragment : Fragment() {
    private var differ = DittoDiffer()
    private var previousDocumentIds: MutableList<String> = mutableListOf()
    private var observer: DittoStoreObserver? = null

    private fun startLiveQuery() {
        observer = ditto.store.registerObserver(
            query = "SELECT * FROM cars",
            parameters = mapOf()
        ) { queryResult ->
            val diff = differ.diff(queryResult.items)

            // Extract data and dematerialize immediately
            val currentDocumentIds = queryResult.items.map { item ->
                val id = item.value["_id"] as? String ?: "unknown"
                item.dematerialize()  // Free native memory
                id
            }

            // Handle diff using stored IDs
            previousDocumentIds = currentDocumentIds.toMutableList()
        }
    }

    override fun onDestroyView() {
        super.onDestroyView()
        observer?.close()  // Always close observer
    }
}

5. Attachment Handling

Use ATTACHMENT annotation in collection definitions. Create attachments with ditto.store.newAttachment().
// ❌ Wrong: Missing ATTACHMENT annotation
ditto.store.execute(
    "INSERT INTO cars DOCUMENTS (:doc)",
    mapOf("doc" to docWithAttachment)
)

// ✅ Correct: Use ATTACHMENT annotation in COLLECTION definition
val attachment = ditto.store.newAttachment(
    inputStream = fileInputStream,
    metadata = metadata
)

ditto.store.execute(
    "INSERT INTO COLLECTION cars (image ATTACHMENT) DOCUMENTS (:doc)",
    mapOf("doc" to mapOf(
        "_id" to id,
        "image" to attachment
    ))
)