Rust Legacy→DQL Migration Guide

Overview

This guide will help you successfully migrate your Ditto Rust application from the legacy query builder APIs to the modern DQL (Ditto Query Language). After reviewing this documentation, you’ll understand how to convert method chaining patterns to DQL syntax and systematically update your data operations.

AI Agent Prompt

Use this prompt when working with an AI coding assistant to migrate your Ditto Rust app from legacy query builder to DQL.

Copy AI Migration Prompt (Click to Expand)

I need help migrating a Ditto Rust application from the legacy query builder APIs to modern DQL (Ditto Query Language). This migration involves converting method chaining patterns to SQL-like DQL syntax.

CRITICAL RULES:
1. All query builder method chains (.collection().find()) must be replaced with ditto.store().execute() using DQL
2. Use parameterized queries with :paramName syntax - NEVER string formatting
3. Counter operations must use PN_INCREMENT BY in APPLY clause - do NOT initialize counter fields
4. Sync subscriptions must use ditto.sync().register_subscription() instead of .find().subscribe()
5. observe_local must be replaced with register_observer

---

CORE MIGRATION AREAS:

1. QUERY SYNTAX MIGRATION

BEFORE (Legacy Query Builder):
```rust
ditto.store.collection("cars")
    .find("color == $args.color")
    .args(HashMap::from([("color", "red".into())]))
    .exec()?;
```

AFTER (DQL):
```rust
ditto.store().execute(
    "SELECT * FROM cars WHERE color = :color",
    Some(HashMap::from([("color", "red".into())]))
)?;
```

2. INSERT OPERATIONS

BEFORE (Legacy Query Builder):
```rust
ditto.store.collection("cars")
    .upsert(HashMap::from([("_id", id.into()), ("color", "blue".into())]))?;
```

AFTER (DQL):
```rust
ditto.store().execute(
    "INSERT INTO cars DOCUMENTS (:car)",
    Some(HashMap::from([("car", car_data)]))
)?;
```

3. UPDATE OPERATIONS

BEFORE (Legacy Query Builder):
```rust
ditto.store.collection("cars")
    .find_by_id(id)
    .update(|doc| {
        doc.get_mut("color")?.set("green");
        Ok(())
    })?;
```

AFTER (DQL):
```rust
ditto.store().execute(
    "UPDATE cars SET color = :color WHERE _id = :id",
    Some(HashMap::from([("color", "green".into()), ("id", id.into())]))
)?;
```

4. DELETE OPERATIONS

BEFORE (Legacy Query Builder):
```rust
ditto.store.collection("cars").find_by_id(id).remove()?;
```

AFTER (DQL):
```rust
ditto.store().execute(
    "DELETE FROM cars WHERE _id = :id",
    Some(HashMap::from([("id", id.into())]))
)?;
```

5. EVICTION OPERATIONS

BEFORE (Legacy Query Builder):
```rust
ditto.store.collection("cars").find_by_id(id).evict()?;
```

AFTER (DQL):
```rust
ditto.store().execute(
    "EVICT FROM cars WHERE _id = :id",
    Some(HashMap::from([("id", id.into())]))
)?;
```

6. COUNTER OPERATIONS (PN_COUNTER)

BEFORE (Legacy Query Builder):
```rust
ditto.store.collection("cars")
    .find_by_id(id)
    .update(|doc| {
        doc.get_mut("numUpdates")?.counter()?.increment(1.0);
        Ok(())
    })?;
```

AFTER (DQL with PN_INCREMENT):
```rust
ditto.store().execute(
    "UPDATE cars APPLY numUpdates PN_INCREMENT BY :increment WHERE _id = :id",
    Some(HashMap::from([("increment", 1.into()), ("id", id.into())]))
)?;
```

IMPORTANT: Do NOT initialize counter fields in documents:
```rust
// WRONG - Creates a register, not a counter
HashMap::from([("counter", 0.into())])

// CORRECT - Omit counter field, it's created on first PN_INCREMENT
HashMap::from([("_id", id.into()), ("color", "blue".into())])
```

7. DOCUMENT FIELD ACCESS MIGRATION

BEFORE (Legacy Query Builder):
```rust
let document = ditto.store.collection("cars").find_by_id(id).exec()?;
let color = document.and_then(|d| d.value.get("color"));
```

AFTER (DQL):
```rust
let result = ditto.store().execute(dql_string, None)?;
let item = result.items.first();
let color = item.and_then(|i| i.value.get("color"));
```

8. OBSERVER MIGRATION (observe_local → register_observer)

BEFORE (Legacy observe_local):
```rust
let live_query = ditto.store.collection("cars")
    .find(&format!("_id.locationId == '{}'", Constants::LOCATION_ID))
    .observe_local(|docs, event| {
        match event {
            DittoLiveQueryEvent::Update(changes) => {
                // Handle changes
            }
            DittoLiveQueryEvent::Initial => {
                // Handle initial data
            }
        }
    })?;
```

AFTER (DQL with register_observer):
```rust
let observer = ditto.store().register_observer(
    "SELECT * FROM cars WHERE _id.locationId = :locationId",
    Some(HashMap::from([("locationId", Constants::LOCATION_ID.into())])),
    |result| {
        // Process result items
        let items = &result.items;

        // Update UI
        update_ui(items);
    }
)?;

// Don't forget to stop when done
observer.stop();
```

9. SYNC SUBSCRIPTIONS MIGRATION

BEFORE (Legacy Query Builder):
```rust
let subscription = ditto.store.collection("cars")
    .find("color == $args.color")
    .args(HashMap::from([("color", "red".into())]))
    .subscribe()?;
```

AFTER (DQL):
```rust
let subscription = ditto.sync().register_subscription(
    "SELECT * FROM cars WHERE color = :color",
    Some(HashMap::from([("color", "red".into())]))
)?;
```

---

COMMON PITFALLS TO AVOID:

1. DQL Syntax Errors
   - Use :paramName for parameters, not $args or string formatting

2. Missing Parameter Binding
   - NEVER use string formatting in queries
   - Always use parameterized queries with HashMap

3. Counter Type Errors
   - Do NOT initialize counter fields with DittoCounter::new() or numbers
   - Use PN_INCREMENT BY in APPLY clause
   - Pass negative values for decrements

4. Memory Management with Observers
   - Always store observer reference
   - Call observer.stop() when done

5. Attachment Handling
   - Use ATTACHMENT annotation: "(image ATTACHMENT)"
   - Create attachments with ditto.store().new_attachment()

---

MIGRATION CHECKLIST:

Search for these legacy patterns and replace:
- [ ] .collection( → ditto.store().execute("SELECT * FROM
- [ ] .find( → Convert to DQL WHERE clause with HashMap arguments
- [ ] .find_by_id( → Convert to DQL WHERE _id = :id
- [ ] .upsert( → Convert to DQL INSERT INTO
- [ ] .update( → Convert to DQL UPDATE SET
- [ ] .remove( → Convert to DQL DELETE FROM
- [ ] .evict( → Convert to DQL EVICT FROM
- [ ] .counter()?.increment( → Convert to PN_INCREMENT BY in APPLY clause
- [ ] DittoCounter::new() → Remove initialization, use PN_INCREMENT
- [ ] .observe_local( → Convert to register_observer
- [ ] .subscribe() → Convert to ditto.sync().register_subscription()
- [ ] ditto.store → ditto.store()
- [ ] ditto.sync → ditto.sync()

---

Please help me convert all legacy query builder patterns in my codebase to DQL syntax. Focus on:
1. Maintaining the same functionality
2. Using proper parameterized queries with HashMap
3. Handling counter operations correctly with PN_INCREMENT
4. Implementing proper observer cleanup with stop()
5. Converting all sync subscriptions to DQL

Start by identifying all uses of .collection() in my codebase and systematically converting each one to the appropriate DQL pattern.

Syntax Change Reference

Document Query Syntax

Document Query All

ditto.store().execute(
    "SELECT * FROM cars",
    None
)?;

Document Query by ID

ditto.store().execute(
    "SELECT * FROM cars WHERE _id = '123'",
    None
)?;

Document Query with Predicate

ditto.store().execute(
    "SELECT * FROM cars WHERE color = 'blue'",
    None
)?;

Document Query with Arguments

use std::collections::HashMap;

ditto.store().execute(
    "SELECT * FROM cars WHERE color = :color",
    Some(HashMap::from([("color", "red".into())]))
)?;

Document Insert

ditto.store().execute(
    "INSERT INTO cars DOCUMENTS (:car)",
    Some(HashMap::from([("car", car_data)]))
)?;

Document Update

ditto.store().execute(
    "UPDATE cars SET color = :color WHERE _id = :id",
    Some(HashMap::from([
        ("color", "green".into()),
        ("id", id.into())
    ]))
)?;

Document Delete

ditto.store().execute(
    "DELETE FROM cars WHERE _id = :id",
    Some(HashMap::from([("id", id.into())]))
)?;

Document Local Eviction

// Evict by ID
ditto.store().execute(
    "EVICT FROM cars WHERE _id = :id",
    Some(HashMap::from([("id", id.into())]))
)?;

// Evict all matching documents
ditto.store().execute(
    "EVICT FROM cars WHERE color = :color",
    Some(HashMap::from([("color", "red".into())]))
)?;

Query Response Handling

Legacy Query Builder → DQL Response

let result = ditto.store().execute(dql_string, None)?;
let item = result.items.first();
let color = item.and_then(|i| i.value.get("color"));

Legacy Query Builder → Modern Document Conversion

fn document_to_car(item: &DittoQueryResultItem) -> Car {
    Car {
        id: item.value["_id"].as_str().unwrap().to_string(),
        color: item.value["color"].as_str().unwrap().to_string(),
    }
}

Observer Migration

Legacy Query Builder → DQL Store Observer Migration

let observer = ditto.store().register_observer(
    "SELECT * FROM cars WHERE _id.locationId = :locationId",
    Some(HashMap::from([("locationId", Constants::LOCATION_ID.into())])),
    |result| {
        // Process result items
        let items = &result.items;

        // Update UI
        update_ui(items);
    }
)?;

Performance Consideration: DQL observers provide more advanced return results including aggregates and projections. This requires more database full scans to ensure consistent results compared to the legacy query builder.Use indexes on query fields to maintain and improve observer performance. Indexes ensure your observers remain functional with optimal query performance.

Best Practice: Create Indexes for Observer Queries

// Create index on frequently queried fields
ditto.store().execute(
    "CREATE INDEX idx_cars_locationId ON cars (_id.locationId)",
    None,
)?;

// Then register observer - queries will use the index
let observer = ditto.store().register_observer(
    "SELECT * FROM cars WHERE _id.locationId = :locationId",
    Some(HashMap::from([("locationId", Constants::LOCATION_ID.into())])),
    |result| {
        // Process results
    }
)?;

For more information on creating and managing indexes, see the DQL Indexing documentation.

Sync Subscriptions Migration

Legacy Query Builder → DQL Sync Subscriptions Subscribe with Query

let subscription = ditto.sync().register_subscription(
    "SELECT * FROM cars WHERE color = :color",
    Some(HashMap::from([("color", "red".into())]))
)?;

Subscribe with Parameters

let subscription = ditto.sync().register_subscription(
    "SELECT * FROM cars WHERE _id.locationId = :locationId",
    Some(HashMap::from([("locationId", Constants::LOCATION_ID.into())]))
)?;

Multiple Subscriptions

let mut subscriptions = Vec::new();
subscriptions.push(
    ditto.sync().register_subscription(
        "SELECT * FROM cars WHERE color = :color",
        Some(HashMap::from([("color", "red".into())]))
    )?
);
subscriptions.push(
    ditto.sync().register_subscription(
        "SELECT * FROM cars WHERE year > :year",
        Some(HashMap::from([("year", 2020.into())]))
    )?
);

Cancel Subscription

subscription.cancel();

Subscribe to All Documents

let subscription = ditto.sync().register_subscription(
    "SELECT * FROM cars",
    None
)?;

Counter Type Migration

PN_COUNTER is the DQL equivalent of the legacy DittoCounter type. When migrating counter operations from the legacy query builder’s counter methods, use PN_INCREMENT BY in the APPLY clause. This maintains full compatibility with existing counter data created by DittoCounter.

Counter Increment

ditto.store().execute(
    "UPDATE cars APPLY numUpdates PN_INCREMENT BY :increment WHERE _id = :id",
    Some(HashMap::from([
        ("increment", 1.into()),
        ("id", id.into())
    ]))
)?;

Counter Decrement

ditto.store().execute(
    "UPDATE cars APPLY viewCount PN_INCREMENT BY :decrement WHERE _id = :id",
    Some(HashMap::from([
        ("decrement", (-1).into()),
        ("id", id.into())
    ]))
)?;

Initialize Counter in Document

// Counter fields are automatically created on first PN_INCREMENT use
ditto.store().execute(
    "INSERT INTO cars DOCUMENTS (:car)",
    Some(HashMap::from([
        ("car", HashMap::from([
            ("_id", id.into()),
            ("color", "blue".into())
            // Do NOT initialize counter fields - they are created on first PN_INCREMENT
        ]).into())
    ]))
)?;

// Then use PN_INCREMENT with APPLY clause to create and increment the counter
ditto.store().execute(
    "UPDATE cars APPLY numUpdates PN_INCREMENT BY 1 WHERE _id = :id",
    Some(HashMap::from([("id", id.into())]))
)?;

Multiple Counter Operations

ditto.store().execute(
    r#"UPDATE cars
       APPLY likes PN_INCREMENT BY :likeIncrement,
             dislikes PN_INCREMENT BY :dislikeDecrement,
             views PN_INCREMENT BY :viewIncrement
       WHERE _id = :id"#,
    Some(HashMap::from([
        ("likeIncrement", 1.into()),
        ("dislikeDecrement", (-1).into()),
        ("viewIncrement", 1.into()),
        ("id", id.into())
    ]))
)?;

Attachment Operations with DQL

Attachment Creation and Storage

// Create attachment using store
let attachment = ditto.store().new_attachment(
    file_path,
    metadata
)?;

// Store attachment with DQL
ditto.store().execute(
    "INSERT INTO COLLECTION cars (image ATTACHMENT) DOCUMENTS (:doc)",
    Some(HashMap::from([("doc", doc_with_attachment)]))
)?;

Attachment Fetching

// Fetch attachment with progress callback
let fetch_result = ditto.store().fetch_attachment(
    attachment_token,
    |event| {
        if let DittoAttachmentFetchEvent::Progress { downloaded_bytes, total_bytes } = event {
            update_progress(downloaded_bytes, total_bytes);
        }
    }
)?;

Performance Enhancements

Indexes for Improved Query Performance

DQL observers and queries benefit significantly from proper indexing. When migrating from the legacy query builder to DQL, creating indexes on frequently queried fields is essential for maintaining optimal performance. Why Indexes Matter for DQL:

DQL observers support advanced features like aggregates and projections
These advanced features require full database scans to ensure consistent results
Indexes dramatically reduce query execution time by avoiding full scans
Combining indexes with observers provides better performance than legacy query builder

Creating Indexes:

// Create index on single field
ditto.store().execute(
    "CREATE INDEX idx_cars_color ON cars (color)",
    None,
)?;

// Create compound index on multiple fields
ditto.store().execute(
    "CREATE INDEX idx_cars_color_year ON cars (color, year)",
    None,
)?;

// Create index on nested field
ditto.store().execute(
    "CREATE INDEX idx_cars_location ON cars (_id.locationId)",
    None,
)?;

Best Practices:

Create indexes on fields used in WHERE clauses
Create indexes before registering observers for those queries
Use compound indexes for queries with multiple filter conditions
Monitor query performance and add indexes as needed

For comprehensive information on indexing strategies, syntax, and best practices, see the DQL Indexing documentation.

Common Pitfalls to Avoid

1. DQL Syntax Errors

Use :paramName for parameters, not $args.paramName or string formatting.

// ❌ Wrong: String formatting
let color = "red";
ditto.store().execute(
    &format!("SELECT * FROM cars WHERE color = '{}'", color),
    None
)?;

// ✅ Correct: Using :paramName with HashMap
ditto.store().execute(
    "SELECT * FROM cars WHERE color = :color",
    Some(HashMap::from([("color", "red".into())]))
)?;

2. Missing Parameter Binding

NEVER use string formatting in queries. Always use parameterized queries with HashMap<&str, Value>.

// ❌ Wrong: String formatting
let location_id = "loc_123";
ditto.store().execute(
    &format!("SELECT * FROM cars WHERE _id.locationId = '{}'", location_id),
    None
)?;

// ✅ Correct: Parameterized query
ditto.store().execute(
    "SELECT * FROM cars WHERE _id.locationId = :locationId",
    Some(HashMap::from([("locationId", location_id.into())]))
)?;

3. Counter Type Errors

Use COUNTER annotation in collection definitions. Do NOT use SET with COUNTER fields. Use APPLY with PN_INCREMENT BY. Pass negative values for decrements.

// ❌ Wrong: Initializing counter with a number (creates REGISTER, not COUNTER)
let doc = HashMap::from([
    ("_id", id.into()),
    ("counter", 0.into())
]);
ditto.store().execute(
    "INSERT INTO items DOCUMENTS (:doc)",
    Some(HashMap::from([("doc", doc.into())]))
)?;

// ❌ Wrong: Using SET on counter field
ditto.store().execute(
    "UPDATE items SET counter = 5 WHERE _id = :id",
    Some(HashMap::from([("id", id.into())]))
)?;

// ✅ Correct: Use PN_INCREMENT BY with APPLY clause (creates counter on first use)
ditto.store().execute(
    "UPDATE COLLECTION items (counter COUNTER) APPLY counter PN_INCREMENT BY :value WHERE _id = :id",
    Some(HashMap::from([("value", 1.into()), ("id", id.into())]))
)?;

// ✅ Correct: Decrement by passing negative value
ditto.store().execute(
    "UPDATE items APPLY counter PN_INCREMENT BY :value WHERE _id = :id",
    Some(HashMap::from([("value", (-1).into()), ("id", id.into())]))
)?;

4. Memory Management with Observers

Always call observer.stop() when done. Be mindful of ownership and lifetimes. Use indexes for improved memory and performance.

// ❌ Wrong: Not stopping observer
{
    let observer = ditto.store().register_observer(
        "SELECT * FROM cars",
        None,
        |result| {
            // Process results
        }
    )?;
    // Observer dropped without stopping
}

// ✅ Correct: Always stop observer when done
struct CarRepository {
    observer: Option<StoreObserver>,
}

impl CarRepository {
    fn start_observing(&mut self, ditto: &Ditto) -> Result<(), DittoError> {
        let observer = ditto.store().register_observer(
            "SELECT * FROM cars",
            None,
            |result| {
                // Extract data immediately
                let cars: Vec<String> = result.items()
                    .iter()
                    .filter_map(|item| item.value().get("_id")?.as_str())
                    .map(String::from)
                    .collect();

                // Update UI with extracted data
            }
        )?;

        self.observer = Some(observer);
        Ok(())
    }

    fn stop_observing(&mut self) {
        if let Some(observer) = self.observer.take() {
            observer.stop();  // Always stop observer
        }
    }
}

5. Attachment Handling

Use ATTACHMENT annotation in collection definitions. Create attachments with ditto.store().new_attachment().

// ❌ Wrong: Missing ATTACHMENT annotation
ditto.store().execute(
    "INSERT INTO cars DOCUMENTS (:doc)",
    Some(HashMap::from([("doc", doc_with_attachment)]))
)?;

// ✅ Correct: Use ATTACHMENT annotation in COLLECTION definition
let attachment = ditto.store().new_attachment(
    file_path,
    metadata
)?;

let doc = HashMap::from([
    ("_id", id.into()),
    ("image", attachment.into())
]);

ditto.store().execute(
    "INSERT INTO COLLECTION cars (image ATTACHMENT) DOCUMENTS (:doc)",
    Some(HashMap::from([("doc", doc.into())]))
)?;

Overview & Setup

SDKs

Accessing Data

Syncing Data

Authentication and Authorization

Managing the Mesh

Deploying your App

Rust Legacy→DQL Migration Guide

Overview

AI Agent Prompt

Syntax Change Reference

Document Query Syntax

Query Response Handling

Observer Migration

Sync Subscriptions Migration

Counter Type Migration

Attachment Operations with DQL

Performance Enhancements

Indexes for Improved Query Performance

Common Pitfalls to Avoid

1. DQL Syntax Errors

2. Missing Parameter Binding

3. Counter Type Errors

4. Memory Management with Observers

5. Attachment Handling

Overview & Setup

SDKs

Accessing Data

Syncing Data

Authentication and Authorization

Managing the Mesh

Deploying your App

Documentation Index

​Overview

​AI Agent Prompt

​Syntax Change Reference

​Document Query Syntax

​Query Response Handling

​Observer Migration

​Sync Subscriptions Migration

​Counter Type Migration

​Attachment Operations with DQL

​Performance Enhancements

​Indexes for Improved Query Performance

​Common Pitfalls to Avoid

​1. DQL Syntax Errors

​2. Missing Parameter Binding

​3. Counter Type Errors

​4. Memory Management with Observers

​5. Attachment Handling

Overview

AI Agent Prompt

Syntax Change Reference

Document Query Syntax

Query Response Handling

Observer Migration

Sync Subscriptions Migration

Counter Type Migration

Attachment Operations with DQL

Performance Enhancements

Indexes for Improved Query Performance

Common Pitfalls to Avoid

1. DQL Syntax Errors

2. Missing Parameter Binding

3. Counter Type Errors

4. Memory Management with Observers

5. Attachment Handling