​

4.12.1 Rust Specific Changes

​

4.12.1 Common Changes

​

4.12 - DQL Enhancements & Platform Expansion

​

DQL Projections & Aggregates

​

Projections

-- Select specific fields
SELECT make, model, year FROM cars

-- Create calculated fields with aliases
SELECT make, model, price * 0.9 AS discounted_price FROM cars

-- Remove duplicates
SELECT DISTINCT color FROM cars

​

Aggregate Functions

-- Count documents
SELECT COUNT(*) AS total_cars FROM cars

-- Calculate statistics
SELECT 
  AVG(price) AS avg_price,
  MIN(price) AS lowest_price,
  MAX(price) AS highest_price,
  SUM(price) AS total_value
FROM cars

-- Group and aggregate
SELECT make, COUNT(*) AS models, AVG(price) AS avg_price
FROM cars
GROUP BY make
HAVING COUNT(*) > 3

​

DQL Index Management

-- Create an index
CREATE INDEX idx_cars_make ON cars (make)

-- Drop an index
DROP INDEX idx_cars_make

-- View all indexes
SELECT * FROM system::dql_indexes

​

Flutter Platform Expansion

​

Linux Support

• Native Linux binary (libdittoffi.so) included
• Full support for development, testing, and production deployments
• Configurable binary path via $LIBDITTOFFI_PATH environment variable
• Compatible with Flutter’s standard Linux development workflow

​

Legacy Query Builder Deprecation

• SQL-like syntax familiar to developers
• Unified interface for all data operations
• Better performance and optimization
• More powerful query capabilities

​

Additional Enhancements

​

System Improvements

• Enhanced disk observer: Now enabled at startup with configurable batch settings
• Improved error handling: Better error propagation and syntax error identification in DQL
• Memory optimizations: Reduced memory usage for aggregate operations and projections

​

DQL Function Additions

• Duration scalar functions for date/time calculations
• Dynamic field references in queries
• Support for arrays as operation values
• Extended multiplication operations for better numeric handling

​

Security & Diagnostics

• Support for SSLKEYLOGFILE for debugging TLS connections
• New system::data_sync_info virtual collection for sync diagnostics
• Transport diagnostics APIs across multiple SDKs

​

Upgrading to 4.12

1. Update your SDK to version 4.12.0 or later
2. Begin migrating legacy query builder code to DQL
3. Explore projections and aggregates for query optimization
4. Consider Flutter Linux support for desktop deployments

​

4.12.0 Common Changelog

​

Added

• The transports_discovered_peers system parameter to dynamically update peers discovered out-of-band (#17009)
• The new system::indexes virtual collection for viewing DQL indexes on small peers (#17011)
• Support for SSLKEYLOGFILE for inspecting network traffic in tools like Wireshark (#17029)
• UPDATE_LOCAL_DIFF id conflict strategy for INSERT statements (#17310)
• A system:data_sync_info virtual collection that exposes per-remote data sync status via DQL (#17360)
• transports_tcp_so_nodelay system parameter to control TCP_NODELAY for TCP and TCP-based websockets connections (#17673)
• Additional metadata to keep track of the timestamp of the last update file received (#DS-290)
• Observers for DQL queries can now access system virtual collections (#DS-294)
• Query executor runner to observers - they now support all of the new DQL features like projections, aggregates, use ids… (#QE-143)
• SYSTEM:COLLECTIONS & SYSTEM:ALL_COLLECTIONS virtual collections to list accessible collections (#QE-146)
• Support for ARRAYs as the value options list in IN DQL expressions (#QE-167)
• DQL scalar functions for durations (#QE-170)
• DQL support for CREATE INDEX (#QE-182)
• DQL support for DROP INDEX (#QE-183)
• Support for dynamic field references in DQL statements (#QE-2)
• Query request history cache qualifiers for system datasource deletes and general mutations (#QE-207)
• Basic RBO for index scan selection (#QE-210)
• Query parser handles context dependent keyword (#QE-241)
• Virtual collections for active and historical DQL engine requests (#QE-82)

​

Changed

• device_disk_observer_is_enabled system parameter can now be changed at runtime (#CORE-612)
• After eviction, document sync now retains high-level session metadata for non-expired sessions, for sync status tracking purposes (#DS-348)
• Projection, distinct and aggregate features out of preview mode! (#QE-100)
• Wildcard expansion to top-level with left-to-right, last-wins projection aliasing (#QE-156)
• Improved ScalarExpr representation in EXPLAIN output (#QE-193)
• Improved syntax error identification in UPDATE statements (#QE-199)
• ALTER SYSTEM DQL statements now accept static expressions (#QE-204)
• USE IDS clause to accept expressions that can be statically evaluated (#QE-217)
• Permit use of unary plus in DQL expressions (#QE-220)
• Extended DQL multiplication to better handle multiplying two large negative integers (#QE-224)
• Query directives processing to accept USE INDEX as the equivalent of USE DIRECTIVES {"#index":...} (#QE-255)
• CREATE & DROP INDEX statements accept IF [NOT] EXISTS to permit repeated execution without error (#QE-259)
• Websocket connectivity errors are now logged with additional detail (#TRAN-511)
• Ditto no longer includes a peer’s device name and OS during P2P advertisement (BLE, mDNS, Wi-Fi Aware, AWDL) (#TRAN-652)

​

Fixed

• A rare race condition which caused connection handshakes to erroneously fail (#17251)
• Merge tombstone summaries when applying remote tombstone (#17654)
• Set default TCP_NODELAY socket option to true to improve latency, notably on Linux (#17673)
• DQL collation order is based on length of arrays / objects, not padding values with extra MISSING (#QE-171)
• DQL scalar function clock() and “local” timezone in WASM environments (#QE-181)
• Error propagation from collection scans (#QE-191)
• Collation in DQL aggregates (#QE-216)
• Panic when using COUNT(*) in a dynamic object (#QE-219)
• Panic when subtracting a large positive integer value from a large negative integer value (#QE-223)
• Aggregate output on empty result sets (#QE-242)

​

Performance

• Revised fetching documents by ID in Query to operate in batches (#QE-248)

​

Other

• Added a warning to the log when a subscription query uses a LIMIT or ORDER BY (#QE-169)

​

4.12.0 Rust Specific Changes

​

4.11.4 Common Changes

​

4.11.3 Common Changes

​

4.11.2 Common Changes

​

4.11.1 Common Changes

​

4.11 - DQL & Legacy Compatibility

​

DQL Strict Mode

• Objects are treated as CRDT maps
• Scalars and arrays are treated as Registers
• Counters and attachments are inferred from operations
• Collection definitions are no longer required

ALTER SYSTEM SET DQL_STRICT_MODE = false

​

DQL Differ

​

DQL Transactions

​

Advanced DQL Features

​

Tombstone Reaper

• Automatically removes tombstones after a configurable time-to-live (TTL)
• Uses read transactions for efficiency
• Includes random scheduling variations to prevent performance problems
• Tracks the number of tombstones evicted for monitoring
• Remembers previous reap times across restarts

​

4.11.0 Common Changelog

​

Added

• Introduced DQL_STRICT_MODE as a system parameter to opt-out of requiring collection definitions (#16573)
• Added transports_ble_adapter_mac system parameter to allow configuring bluetooth adapter MAC address (#16159)
• Small peers now log the result of OnlinePlayground authentication, including if the supplied shared token was incorrect (#16301)
• A cumulative running total of the number of tombstones ever evicted by the reaper to Small Peer Info (#16424)
• Added system parameters to fine-tune Ditto behavior when it encounters a locked persistence directory (#CORE-479)
• New DQL CAST function to explicitly cast between types (#QE-108)
• Conditional functions for unknowns (#QE-111)
• DQL DATE processing functions (#QE-112)
• New DQL clause USE IDS for direct document retrieval by id (#QE-114)
• Additional string functions (#QE-124)
• SIMILAR TO filter expression (#QE-125)
• PN_COUNTER support (APPLY clause in UPDATE statement) (#QE-127)
• UNSET clause to the UPDATE statement (#QE-128)
• MERGE as an alias for UPDATE in the ON ID CONFLICT clause of an INSERT statement (#QE-132)
• The ability to use runtime-evaluated expressions in object constructs in DQL statements (#QE-133)
• The ability to use runtime-evaluated expressions in array constructs in DQL statements (#QE-134)
• Static expressions in mutators (#QE-139)
• Additional object manipulation DQL scalar functions (#QE-159)
• EXPLAIN for query plan (#QE-105)
• More detailed messages about network problems are logged when authentication fails (#SEC-124)

​

Changed

• Use the modern rustls-platform-verifier TLS crate on all platforms rather than rustls-native-certs (#15956)
• The tombstone reaper scheduling to remember the previous reap times across shutdown and restart of the small peer (#16237)
• The tombstone reaper scheduling code to add random variations, which helps prevent performance problems if/when all peers run the reaper simultaneously (#16286)
• The tombstone reaper to be enabled by default for all SDKs (#16350)
• INSERT to now accept either DOCUMENTS or VALUES as the keyword introducing the list of values to insert (#QE-140)
• Date functions accept “local” as a timezone & use the executing peer’s local timezone (#QE-168)
• Unified dedicated TCP servers across different transports (Multicast, mDNS, and WiFi Aware) by utilizing a single static TCP server implementation (#TRANS-131)

​

Fixed

• Fixed issue where user_collection_sync_scopes could not be updated after initial set (#16299)
• Fixed: A rare crash on Android devices when shutting down Bluetooth. (#17069)
• Fixed: Bluetooth connectivity on Apple devices works when background Bluetooth permissions are missing. (#17065)

​

Removed

• Old multihop networking stack and system parameter network_use_nextgen_multihop_stack (#16370)
• TLS certificate compression, which led to incompatibilities (#SDKS-1046)

​

Performance

• The tombstone reaper uses read transactions for some of its work instead of holding open a long-running write transaction, thus reducing contention on the small peer database (#16338)
• Improved speed and efficiency of the tombstone reaping process (#16535)

​

4.11.0 Rust Specific Changes

​

4.10.5 Rust Specific Changes

​

4.10.5 Common Changes

​

4.10.2 Common Changes

​

4.10.1 Common Changes

​

4.10.0 New Capabilities

​

4.10.0 Rust Specific Changelog

• BatchIdOperation::exec to allow getting a specific document within the scope of a write transaction block. (#SDKS-425)
• Repeated reads of a document within an ID-specific transaction scope of the query builder API now reflect changes made in the same transaction. (SDKS-245)
• (Experimental) Bus API correctly falls back to multihop connectivity if a direct connection is lost. (#15670)
• Deprecated arm-unknown-linux-gnueabihf target from support due to missing internal coverage and usage. (#15795)

​

4.9.4 Common Changes

​

4.9.3 Common Changes

​

4.9.1 Common Changes

​

4.9.0 Rust Specific Changes

​

General Changes

​

API Renaming

• Useditto.auth()

• Use ditto.transport_config()

• Use ditto.app_id()

​

Module Reorganization

• use dittolive_ditto::presence::{Peer, Presence, PresenceGraph, PresenceObserver, PresenceOs, Connection, ConnectionType}

• use store::attachment::{FetcherVersion, DittoAttachmentFetcher, DittoAttachmentToken, DittoAttachmentFetchEvent, DittoAttachmentTokenLike}

• use store::collections::{CollectionsEvent, PendingCollectionsOperation, CollectionsEventHandler}

• use store::collection::{DocumentId, PendingCursorOperation, PendingIdSpecificOperation, DittoDocument, DittoMutDocument, MutableValue}

• Moved ditto module contents to crate-level (#13645)

• Changed: Updated sync module docs (#13702)
• Deprecated: dittolive_ditto::store::dql::SyncSubscription
• Added: dittolive_ditto::sync::SyncSubscription

• Deprecated: crate-level types module, contents moved into query_builder
• Deprecated: store::{batch,collection,collections,live_query,update} modules, contents moved into query_builder
• Deprecated: crate-level subscription module, Subscription moved into query_builder module

​

Depreciated unused and unintentially exposed items

• use Ditto::builder() instead.

​

4.9.0 Logging Clarity Improvements

• More information is included when physical connections start and end (such as the type of connection and the remote peer’s key)
• Devices that try to connect to themselves now say “device has connected to itself” instead of an error about the remote site ID being the same as the local peer
• Authentication client logs have been made more human readable. In particular:
• Recoverable errors are logged together with information about the recovery actions that are being undertaken
• Several abbreviations have been replaced with human-readable descriptions
• Errors are printed using their descriptions rather than by dumping debug representations
• Abbreviated jargon has been replaced with clearer wording in various log lines
• “phy ended” has become “physical connection ended”
• “error getting char” has become “required Ditto BLE characteristic not found”
• “error getting radio state” has become “error getting Windows Bluetooth radio state”
• Peer events that represent a transport starting or stopping are now logged with the message “transport started”/“transport ended” rather than as a raw peer_event=… log
• Reduced the noisiness of “starting” log lines around Ditto initialization time
• Noise from long peer keys in logs has been reduced in some places by replacing them with their truncated form
• Noise from duplicated peer ID/peer key information in replication logs has been reduced by consolidating the information
• “starting compression and export” is now followed by a “completed export” log at the same level
• Successful SQLite WAL recovery logs were further lowered from INFO to DEBUG

​

4.9.0 Common Changes