​

4.12 - DQL Enhancements & Platform Expansion

​

DQL Projections & Aggregates

​

Projections

-- Select specific fields
SELECT make, model, year FROM cars

-- Create calculated fields with aliases
SELECT make, model, price * 0.9 AS discounted_price FROM cars

-- Remove duplicates
SELECT DISTINCT color FROM cars

​

Aggregate Functions

-- Count documents
SELECT COUNT(*) AS total_cars FROM cars

-- Calculate statistics
SELECT 
  AVG(price) AS avg_price,
  MIN(price) AS lowest_price,
  MAX(price) AS highest_price,
  SUM(price) AS total_value
FROM cars

-- Group and aggregate
SELECT make, COUNT(*) AS models, AVG(price) AS avg_price
FROM cars
GROUP BY make
HAVING COUNT(*) > 3

​

DQL Index Management

-- Create an index
CREATE INDEX idx_cars_make ON cars (make)

-- Drop an index
DROP INDEX idx_cars_make

-- View all indexes
SELECT * FROM system::dql_indexes

​

Flutter Platform Expansion

​

Linux Support

• Native Linux binary (libdittoffi.so) included
• Full support for development, testing, and production deployments
• Configurable binary path via $LIBDITTOFFI_PATH environment variable
• Compatible with Flutter’s standard Linux development workflow

​

Legacy Query Builder Deprecation

• SQL-like syntax familiar to developers
• Unified interface for all data operations
• Better performance and optimization
• More powerful query capabilities

​

Additional Enhancements

​

System Improvements

• Enhanced disk observer: Now enabled at startup with configurable batch settings
• Improved error handling: Better error propagation and syntax error identification in DQL
• Memory optimizations: Reduced memory usage for aggregate operations and projections

​

DQL Function Additions

• Duration scalar functions for date/time calculations
• Dynamic field references in queries
• Support for arrays as operation values
• Extended multiplication operations for better numeric handling

​

Security & Diagnostics

• Support for SSLKEYLOGFILE for debugging TLS connections
• New system::data_sync_info virtual collection for sync diagnostics
• Transport diagnostics APIs across multiple SDKs

​

Upgrading to 4.12

1. Update your SDK to version 4.12.0 or later
2. Begin migrating legacy query builder code to DQL
3. Explore projections and aggregates for query optimization
4. Consider Flutter Linux support for desktop deployments

​

4.12.0 Common Changelog

​

Added

• The transports_discovered_peers system parameter to dynamically update peers discovered out-of-band (#17009)
• The new system::indexes virtual collection for viewing DQL indexes on small peers (#17011)
• Support for SSLKEYLOGFILE for inspecting network traffic in tools like Wireshark (#17029)
• UPDATE_LOCAL_DIFF id conflict strategy for INSERT statements (#17310)
• A system:data_sync_info virtual collection that exposes per-remote data sync status via DQL (#17360)
• transports_tcp_so_nodelay system parameter to control TCP_NODELAY for TCP and TCP-based websockets connections (#17673)
• Additional metadata to keep track of the timestamp of the last update file received (#DS-290)
• Observers for DQL queries can now access system virtual collections (#DS-294)
• Query executor runner to observers - they now support all of the new DQL features like projections, aggregates, use ids… (#QE-143)
• SYSTEM:COLLECTIONS & SYSTEM:ALL_COLLECTIONS virtual collections to list accessible collections (#QE-146)
• Support for ARRAYs as the value options list in IN DQL expressions (#QE-167)
• DQL scalar functions for durations (#QE-170)
• DQL support for CREATE INDEX (#QE-182)
• DQL support for DROP INDEX (#QE-183)
• Support for dynamic field references in DQL statements (#QE-2)
• Query request history cache qualifiers for system datasource deletes and general mutations (#QE-207)
• Basic RBO for index scan selection (#QE-210)
• Query parser handles context dependent keyword (#QE-241)
• Virtual collections for active and historical DQL engine requests (#QE-82)

​

Changed

• device_disk_observer_is_enabled system parameter can now be changed at runtime (#CORE-612)
• After eviction, document sync now retains high-level session metadata for non-expired sessions, for sync status tracking purposes (#DS-348)
• Projection, distinct and aggregate features out of preview mode! (#QE-100)
• Wildcard expansion to top-level with left-to-right, last-wins projection aliasing (#QE-156)
• Improved ScalarExpr representation in EXPLAIN output (#QE-193)
• Improved syntax error identification in UPDATE statements (#QE-199)
• ALTER SYSTEM DQL statements now accept static expressions (#QE-204)
• USE IDS clause to accept expressions that can be statically evaluated (#QE-217)
• Permit use of unary plus in DQL expressions (#QE-220)
• Extended DQL multiplication to better handle multiplying two large negative integers (#QE-224)
• Query directives processing to accept USE INDEX as the equivalent of USE DIRECTIVES {"#index":...} (#QE-255)
• CREATE & DROP INDEX statements accept IF [NOT] EXISTS to permit repeated execution without error (#QE-259)
• Websocket connectivity errors are now logged with additional detail (#TRAN-511)
• Ditto no longer includes a peer’s device name and OS during P2P advertisement (BLE, mDNS, Wi-Fi Aware, AWDL) (#TRAN-652)

​

Fixed

• A rare race condition which caused connection handshakes to erroneously fail (#17251)
• Merge tombstone summaries when applying remote tombstone (#17654)
• Set default TCP_NODELAY socket option to true to improve latency, notably on Linux (#17673)
• DQL collation order is based on length of arrays / objects, not padding values with extra MISSING (#QE-171)
• DQL scalar function clock() and “local” timezone in WASM environments (#QE-181)
• Error propagation from collection scans (#QE-191)
• Collation in DQL aggregates (#QE-216)
• Panic when using COUNT(*) in a dynamic object (#QE-219)
• Panic when subtracting a large positive integer value from a large negative integer value (#QE-223)
• Aggregate output on empty result sets (#QE-242)

​

Performance

• Revised fetching documents by ID in Query to operate in batches (#QE-248)

​

Other

• Added a warning to the log when a subscription query uses a LIMIT or ORDER BY (#QE-169)

​

4.12.0 Flutter Specific Changes

​

4.11 - DQL & Legacy Compatibility

​

DQL Strict Mode

• Objects are treated as CRDT maps
• Scalars and arrays are treated as Registers
• Counters and attachments are inferred from operations
• Collection definitions are no longer required

ALTER SYSTEM SET DQL_STRICT_MODE = false

​

DQL Differ

​

DQL Transactions

​

Advanced DQL Features

​

Tombstone Reaper

• Automatically removes tombstones after a configurable time-to-live (TTL)
• Uses read transactions for efficiency
• Includes random scheduling variations to prevent performance problems
• Tracks the number of tombstones evicted for monitoring
• Remembers previous reap times across restarts

​

4.11.0 Common Changelog

​

Added

• Introduced DQL_STRICT_MODE as a system parameter to opt-out of requiring collection definitions (#16573)
• Added transports_ble_adapter_mac system parameter to allow configuring bluetooth adapter MAC address (#16159)
• Small peers now log the result of OnlinePlayground authentication, including if the supplied shared token was incorrect (#16301)
• A cumulative running total of the number of tombstones ever evicted by the reaper to Small Peer Info (#16424)
• Added system parameters to fine-tune Ditto behavior when it encounters a locked persistence directory (#CORE-479)
• New DQL CAST function to explicitly cast between types (#QE-108)
• Conditional functions for unknowns (#QE-111)
• DQL DATE processing functions (#QE-112)
• New DQL clause USE IDS for direct document retrieval by id (#QE-114)
• Additional string functions (#QE-124)
• SIMILAR TO filter expression (#QE-125)
• PN_COUNTER support (APPLY clause in UPDATE statement) (#QE-127)
• UNSET clause to the UPDATE statement (#QE-128)
• MERGE as an alias for UPDATE in the ON ID CONFLICT clause of an INSERT statement (#QE-132)
• The ability to use runtime-evaluated expressions in object constructs in DQL statements (#QE-133)
• The ability to use runtime-evaluated expressions in array constructs in DQL statements (#QE-134)
• Static expressions in mutators (#QE-139)
• Additional object manipulation DQL scalar functions (#QE-159)
• EXPLAIN for query plan (#QE-105)
• More detailed messages about network problems are logged when authentication fails (#SEC-124)

​

Changed

• Use the modern rustls-platform-verifier TLS crate on all platforms rather than rustls-native-certs (#15956)
• The tombstone reaper scheduling to remember the previous reap times across shutdown and restart of the small peer (#16237)
• The tombstone reaper scheduling code to add random variations, which helps prevent performance problems if/when all peers run the reaper simultaneously (#16286)
• The tombstone reaper to be enabled by default for all SDKs (#16350)
• INSERT to now accept either DOCUMENTS or VALUES as the keyword introducing the list of values to insert (#QE-140)
• Date functions accept “local” as a timezone & use the executing peer’s local timezone (#QE-168)
• Unified dedicated TCP servers across different transports (Multicast, mDNS, and WiFi Aware) by utilizing a single static TCP server implementation (#TRANS-131)

​

Fixed

• Fixed issue where user_collection_sync_scopes could not be updated after initial set (#16299)
• Fixed: A rare crash on Android devices when shutting down Bluetooth. (#17069)
• Fixed: Bluetooth connectivity on Apple devices works when background Bluetooth permissions are missing. (#17065)

​

Removed

• Old multihop networking stack and system parameter network_use_nextgen_multihop_stack (#16370)
• TLS certificate compression, which led to incompatibilities (#SDKS-1046)

​

Performance

• The tombstone reaper uses read transactions for some of its work instead of holding open a long-running write transaction, thus reducing contention on the small peer database (#16338)
• Improved speed and efficiency of the tombstone reaping process (#16535)

​

4.11.0 Flutter Specific Changes

• Added DittoDiff class to represent diffs produced by the DittoDiffer
• Added DittoDiffer class for calculating diffs between query results
• Added transaction() method on DittoStore for performing DQL transactions
• Added DittoTransaction class to represent an active DQL transaction
• Added DittoTransactionInfo class for transaction information
• Added DittoTransactionCompletionAction for commit/rollback actions

​

4.10.4 Security & Stability Improvements

• Improved reliability of secure connections by gracefully handling certificate validation challenges
• Enhanced network stability with smarter handling of security certificates
• Added flexibility in secure connection handling while maintaining robust security standards

​

4.10.3 Stability Improvements, DQL Enhancements

• Added powerful sorting and filtering capabilities to UPDATE operations, giving you more control over how you modify data on Small Peer devices
• Enhanced reliability and performance of data queries across your application

​

4.10.2

• No significant changes. Bumped version to 4.10.2 to align with other SDKs.

​

4.10.1

• Added support for the web platform
• Attachment fetchers now successfully complete the attachment fetch operation
• The isStopped property of AttachmentFetcher instances now becomes true after completing an attachment fetch
• Resolved a memory leak that affected QueryResult and QueryResultItem
• ditto.store.newAttachment now returns a Future<Attachment> rather than an Attachment
• ditto.close() now returns a Future<void> rather than void

​

4.10.0 New Capabilities

​

Now Supporting Flutter 3.19+

​

Experimental Support for Hot Restart in Flutter

• Logging must be disabled. To do this set DittoLogger.isEnabled = false immediately after calling await Ditto.init().
• Attempting to use attachments will cause a crash.
• Hot restarting will leak some resources, notably memory. In our testing, approximately 20-30MB is leaked per hot-restart.
• Data integrity cannot be guaranteed. You may experience data corruption when hot-restarting, which may be propagated to any peers that are being synced with.
• Sporadic crashes can occur.

​

4.10.0 Flutter Specific Changelog

• Add support for Flutter 3.19 (Flutter - support 3.19 #16080)
• Throw an error when trying to use Ditto APIs before awaiting Ditto.init() (#15880)
• persistenceDirectory now supports relative paths and no longer needs to exist, and also works on web (#15890)
• compiling with —wasm now works. Required for upcoming Flutter Web release. (#15991)
• LAN transport shuts down when stopSync() is called (#SDKS-707)

​

4.9.2

​

4.9.1

​

4.9.0 (GA Release)

​

API Support Guarantees

​

Changes from the Public Preview

• Enhanced developer ergonomics through synchronous APIs
• Improved performance for all operations across all platforms
• Reduced complexity improving reliability
• Reduced background operations for better battery life
• Lower memory consumption

​

Breaking Changes

​

Ditto Initialization Required using await Ditto.init()

Future<void> main() async {
    WidgetsFlutterBinding.ensureInitialized();
    await Ditto.init();
    runApp(MyCoolApp());
}

​

Select asnyc functions have become sync

• ditto.startSync();
• ditto.store.registerObserver(...);
• ditto.sync.registerSubscription(...);

// Previous syntax
final observer = await ditto.store.registerObserver("SELECT * FROM tasks");

// New syntax
final observer = ditto.store.registerObserver("SELECT * FROM tasks");

// Previous syntax
ditto.store.registerObserver("SELECT * FROM tasks").then((observer) {
  setState(() {
    _observer = observer;
  });
});

// New syntax
final observer = ditto.store.registerObserver("SELECT * FROM tasks");
setState(() {
    _observer = observer;
});

​

Methods to setter/getter variables

// Previous syntax
await DittoLogger.setMinimumLogLevel(LogLevel.debug);
await DittoLogger.getMinimumLogLevel();

// New syntax
DittoLogger.minimumLogLevel = LogLevel.debug;

// Previous syntax
await DittoLogger.setCustomLogCallback((logLevel, logString) {
    print("$logLevel: $logString");
});

// New syntax
DittoLogger.customLogCallback = (logLevel, logString) {
    print("$logLevel: $logString");
}

​

Boolean functions renamed from getFoo/setFoo to isFoo

// Previous syntax
await DittoLogger.setEnabled()
await DittoLogger.getEnabled()

// New syntax
DittoLogger.isEnabled

// Previous syntax
await ditto.setTransportConfig()
await ditto.getTransportConfig()

// New syntax
ditto.transportConfig

​

Select Async Initialization is now Sync

• All Identity subclass (E.g. OnlinePlaygroundIdentity)

// Previous syntax
final identity = await OnlinePlaygroundIdentity.create(
  appID: appID,
  token: token,
);

// New syntax
final identity = OnlinePlaygroundIdentity(
  appID: appID,
  token: token,
);

​

Restricted Class Modifiers

// Previous syntax
class MockDitto extends Mock implements Ditto {}

// New syntax
class DittoWrapper {
    final Ditto _ditto;
    // constructor

    Future<List<Task>> loadTasks() => ...;
}

class MockDitto extends Mock implements DittoWrapper {}

​

Removal of dart:io types in public API

// Previous syntax
final persistenceDirectory = Directory("/my/cool/directory");
final ditto = await Ditto.open(
  persistenceDirectory: persistenceDirectory,
  // ...
);

// New syntax
final persistenceDirectory = Directory("/my/cool/directory");
final ditto = await Ditto.open(
  persistenceDirectory: persistenceDirectory.path,
  // ...
);

​

Min Kotlin Gradle Plugin Version Upgraded to 1.8.0

1. Navigate to the root folder
2. Open ./android/settings.gradle
3. Update the org.jetbrains.kotlin.android plugin to 1.8.0
   Copy

   Ask AI

   plugins {
       // ...
       id "org.jetbrains.kotlin.android" version "1.8.0" apply false
   }