Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.ditto.live/llms.txt

Use this file to discover all available pages before exploring further.

​
5.0.0
Release Date: May 5, 2026

​
5.0 - Built for Speed and Developer Experience

Ditto 5.0 brings significant performance improvements and a developer experience redesigned for usability. Your applications run faster with optimized queries and sync, while modern APIs and simplified patterns eliminate complexity at every turn.

Faster local queries

Automatic optimizations make your apps more responsive with zero code changes

No more schemas

Start building immediately without upfront type definitions

Simpler initialization

Clear, modern patterns replace confusing legacy APIs

One query language

DQL handles all data operations consistently across platforms

Built-in observability

Query system metrics and configuration directly with DQL

25% smaller footprint

Faster builds, lower memory usage, cleaner codebase

Plus much more

Advanced query features, networking improvements, and enhanced reliability

​
Our Most Rigorously Tested Release

Ditto 5.0 has undergone extensive validation in our internal mesh lab, ensuring reliability across real-world scenarios before reaching production.Testing scope:
  • Multi-device mesh scenarios - Validated with up to 40 concurrent devices across iOS, Android, and budget hardware
  • Cross-platform compatibility - Tested heterogeneous meshes mixing iOS, Android, and version combinations
  • Extended reliability testing - 24-hour continuous operation tests validating memory stability and connection resilience
  • Real-world datasets - From small 1MB datasets to large 50MB+ product catalogs
  • Network conditions - LAN, BLE, WiFi Aware, AWDL, and mixed transport scenarios
  • Lifecycle edge cases - Background/foreground transitions, network partitions, and recovery scenarios
Every test validates performance, memory management, data integrity, and sync convergence across the scenarios that matter most to production deployments.

​
In This Release

This is a major version with breaking changes, but maintains backwards compatibility with v4 deployments (4.11+). Migrate at your own pace, and upgrade to v4.14 first for the smoothest transition.DQL for All Data Operations:Core Capabilities:Performance & Platform:Upgrading to v5:C++ Specific ChangesFull Changelog

​
DQL for All Data Operations

​
DQL for All Data Operations

Ditto 5.0 completes the transition to DQL (Ditto Query Language) as the single API for all data operations. The legacy query builder has been removed.

​
What Changed

  • Legacy query builder removed: All store.collection() methods and fluent query APIs are no longer available
  • Full feature parity: Every legacy operation has a DQL equivalent
  • Single query language: DQL handles reads, writes, subscriptions, and observers
  • Works everywhere: Same syntax across mobile, server, and web
  • SQL-familiar: Standard SQL patterns for immediate productivity

​
Why One Query Language

A unified query language means one implementation to maintain, faster feature delivery, and consistent behavior across all platforms. New capabilities and optimizations benefit every SDK simultaneously.

​
Migration Path

All legacy query operations have direct DQL equivalents:Update Operations:
store.execute("UPDATE cars SET miles = 50000 WHERE _id = 'abc123'");
Observers:
store.register_observer(
    "SELECT * FROM cars WHERE miles > 100000",
    [](ditto::QueryResult result) {
        // handle changes
    }
);
Complete migration examples for all legacy query patterns are available in the Legacy to DQL Migration Guide.

​
Core Improvements

​
Simplified Initialization & Configuration

Ditto 5.0 introduces a completely redesigned initialization flow built around the new DittoConfig pattern. This replaces the previous Identity-based approach with a clearer, more predictable setup that aligns with modern best practices.

​
What’s New

The new configuration system provides:
  • Unified configuration object: All initialization parameters are set through DittoConfig factory methods
  • Fallible initialization: Explicit error handling during setup catches configuration issues early
  • Asynchronous patterns: Native async/await support where appropriate for each platform
  • Simplified authentication: Clearer authorization client that’s easier to understand and implement

​
What This Solves

The previous initialization flow required multiple unintuitive settings due to legacy compatibility concerns. For example, developers had to set disableCloudSync = true to connect to a Big Peer, which was confusing and non-obvious.The new pattern consolidates all configuration into a single, coherent flow that makes the relationship between settings explicit and easier to reason about.

​
Accessing Configuration at Runtime

After initializing Ditto, you can access the configuration object to retrieve settings like the database ID. This replaces methods like getAppID() from v4:
// Example: Access database ID
const config = ditto.config;  // or ditto.getConfig() depending on SDK
const databaseID = config.databaseID;
Each SDK provides access to the config object with language-appropriate naming conventions. See the SDK-specific migration guides for exact syntax.

​
Availability

The DittoConfig pattern was introduced as an option in v4.12 and becomes the only initialization method in v5.0.
Migration guides for each SDK are available in the v5 documentation. If you’re currently on v4.11 or later, the migration path is straightforward.

​
Migration Example

The initialization flow has changed from Identity-based to DittoConfig-based patterns. Here’s how to migrate:
auto config = ditto::DittoConfig::online_playground(
    "your-database-id",  // was: app_id
    "your-token"
);
auto ditto = ditto::Ditto::open(config).get();

// Start sync
ditto.get_sync().start();  // was: ditto.start_sync()

​
Accessing Configuration at Runtime

After initialization, you can access your Ditto configuration to retrieve settings like the database ID:
auto config = ditto.get_config();
auto database_id = config.database_id;  // Replaces ditto.get_app_id() from v4

​
Schema-Free Data Modeling

Ditto 5.0 transforms the developer experience by making DQL strict mode disabled by default. This eliminates the need to define collection schemas or CRDT types upfront, allowing you to insert nested objects and data structures without pre-defining types.

​
What’s New

With strict mode disabled by default, Ditto changes how objects are stored and synchronized:Objects default to MAP type instead of REGISTER typeThis fundamental change means:
  • Field-level sync: Ditto syncs individual field changes instead of replacing entire objects
  • Automatic type inference: No need to pre-define collection schemas or CRDT types
  • Nested structures: Insert complex JSON-like documents without type definitions
  • Concurrent updates merge: When peers update different fields simultaneously, both changes are preserved
For example, if two peers update different fields in the same object concurrently, both changes sync successfully rather than one overwriting the other.

​
What This Solves

This change provides a more simplified data management structure with two key benefits:
  • Add-wins behavior on objects: When peers create or modify objects, additions are preserved rather than overwritten
  • Field-level delta sync on all nodes: Every field in your document syncs independently, reducing bandwidth and enabling fine-grained conflict resolution throughout the entire document structure

​
New Customers

Schema-free data modeling is enabled by default in Ditto 5.0. No action is needed—simply start building with automatic type inference and field-level sync.

​
Migrating Customers

If you’re migrating from Ditto 4.X, set DQL_STRICT_MODE = true to ensure your application behavior remains the same:
ALTER SYSTEM SET DQL_STRICT_MODE = true
This maintains the same data modeling semantics you’re currently using. Once your application is stable on v5, follow the strict mode migration guide and work with the Ditto CX team to migrate to schema-free data modeling and take advantage of the improved developer experience.

​
Migration Considerations

Strict mode is a local configuration setting on each device that controls how DQL interprets and writes data structures.How it works across peers:
  • Data syncs successfully between peers regardless of different strict mode settings
  • Each peer interprets data based on its own strict mode setting when reading/writing
  • With DQL_STRICT_MODE=false: Objects are inferred as MAPs (field-level merging)
  • With DQL_STRICT_MODE=true: Objects without explicit type definitions are treated as REGISTERs (whole object replacement)
Impact on data:
  • Collections/documents created, modified, or read while strict mode is disabled will use inferred types (objects → MAPs by default)
  • Collections/documents created, modified, or read while strict mode is enabled use default REGISTER type and require explicit definitions for other types
Best practices for mixed deployments:
  • If mixing settings, explicitly define MAP types in collection definitions on peers with strict mode enabled
Learn more about strict mode, cross-peer synchronization, and troubleshooting in the DQL Strict Mode documentation.

​
How It Works

The key difference is whether you need to explicitly define MAP types for objects:
// Objects are automatically inferred as MAPs - no definition needed
store.execute(R"(
    INSERT INTO products
    DOCUMENTS ({
        _id: '123',
        name: 'Widget',
        metadata: {
            manufacturer: 'Acme Corp',
            warehouse: 'East'
        }
    })
)");

// Query without type definitions
store.execute("SELECT * FROM products WHERE _id = '123'");

​
New DQL Query Features

Ditto 5.0 introduces several new DQL syntax features that expand query capabilities and make complex queries easier to write.

​
CASE Statements

Add conditional logic to queries with CASE expressions:
-- Match a field against multiple values
SELECT CASE a
  WHEN 1 THEN 'one'
  WHEN 2 THEN 'two'
  ELSE 'neither'
END
FROM test

​
BETWEEN Expressions

The BETWEEN operator provides a shorthand for defining an inclusive numeric range for an expression result.Syntax:
<expr> BETWEEN <low-expr> AND <high-expr>
Example:
SELECT * FROM test WHERE a BETWEEN 1 AND 10
This is equivalent to (a >= 1 AND a <= 10).
Order matters: The order of expressions is important. Reversing the terms implies an inverse range - BETWEEN 1 AND 10 and BETWEEN 10 AND 1 are NOT equivalent.

​
Array & Object Search Syntax

Test elements within arrays or objects using ANY, EVERY, or ANY AND EVERY operators:Syntax:
(ANY|EVERY|ANY AND EVERY) [name:] value (IN|WITHIN) expression SATISFIES condition END
  • IN - Searches the array/object directly
  • WITHIN - Searches recursively through nested structures
  • ANY - Returns true if at least one element matches
  • EVERY - Returns true if all elements match (empty arrays/objects pass)
  • ANY AND EVERY - Like EVERY, but empty arrays/objects fail
-- Check if any element in an array equals 2
SELECT ANY x IN [1,2,3] SATISFIES x = 2 END
FROM system:dual

-- Check if any nested element equals 2
SELECT ANY x WITHIN [1,[2],3] SATISFIES x = 2 END
FROM system:dual

-- Check if all elements equal 1 (or are arrays)
SELECT EVERY x WITHIN [1,[1],1]
SATISFIES x = 1 OR type(x) = 'array' END
FROM system:dual

​
Array & Object Transformation Syntax

Create new arrays and objects by transforming existing data structures:Array Transformation Syntax:
ARRAY valueExpr FOR [name:] value IN source [WHEN condition] END
Object Transformation Syntax:
OBJECT nameExpr:valueExpr FOR name:value IN source [WHEN condition] END

-- Transform array elements to objects with index
ARRAY {"index":i,"val":v} FOR i:v IN [1,2,3] WHEN v%2 = 0 END
-- Result: [{"index":1,"val":2}]

-- Convert numbers to strings, excluding value 1
ARRAY cast(v,'string') FOR v IN [1,2,3] WHEN v != 1 END
-- Result: ["2","3"]

-- Transform string characters with conditional logic
ARRAY CASE WHEN i%2 = 0 THEN "foo" ELSE "bar" END
FOR i:v IN split("hello","") END
-- Result: ["foo","bar","foo","bar","foo","bar"]

-- Extract values from object into array
ARRAY v FOR n:v IN {"a":"one","b":"two"} END
-- Result: ["one","two"]
Key behaviors:
  • If source evaluates to MISSING, the result is MISSING
  • For arrays: if source is not an array/object, the result is NULL
  • For objects: duplicate field names will overwrite previous values
  • Elements/fields where valueExpr evaluates to MISSING are excluded from the result

​
Extended String Literals

Support for escape sequences in strings:
-- Use escape sequences in string literals
SELECT * FROM messages
WHERE content = 'Line 1\nLine 2\tTabbed'

-- [PLACEHOLDER: Add more escape sequence examples if needed]

​
Hexadecimal Numeric Constants

Additional numeric literal formats:
-- Use hexadecimal literals
SELECT * FROM config
WHERE flags = 0xFF

-- [PLACEHOLDER: Add more hex literal examples if needed]

​
Performance & Platform

​
DQL Query Performance Improvements

Your applications will feel noticeably faster and more responsive. Data queries complete faster, delivering a snappier user experience whether users are searching, filtering, or loading content. These performance gains happen automatically—no code changes required.

​
Query Planner Enhancements

The DQL query planner has been enhanced to recognize more optimization opportunities:
  • Automatic ID scan conversion: Equality filters on _id fields automatically convert to ID scans, bypassing index lookups when exact document IDs are known
  • Deferred document fetching: Query planner can defer fetching full documents until after sorting and applying offset/limit when index access supports it
  • Improved covering index support: More scenarios where the query planner can satisfy queries entirely from index data without retrieving documents
  • Index-only queries: Additional cases where queries can be answered using only index scans

​
Streaming Query Execution

The query engine now uses streaming interfaces internally:
  • Reduced memory overhead: Results are streamed rather than fully materialized where possible
  • DISTINCT operator streaming: DISTINCT queries now stream results, reducing memory usage for large result sets
  • Improved operator inlining: Query operators can be inlined into producers for better performance

​
Shared Statement Cache

Ditto 5.0 introduces a shared statement cache that stores and reuses compiled query plans:
  • Plan reuse: Compiled query plans are cached and reused for identical or similar statements, eliminating redundant parsing and planning overhead
  • Automatic validation: Cached plans are automatically verified and invalidated when collection schemas or system directives change
  • Dynamic sizing: The cache automatically resizes based on workload patterns
  • Improved large statement performance: Particularly benefits complex queries and larger statements by avoiding expensive re-compilation
This optimization is especially impactful for applications that execute the same queries repeatedly, such as real-time dashboards or frequently-accessed data views.
These improvements are automatic - no code changes required. Your existing DQL queries will benefit from the enhanced query planner.

​
Data Sync Performance Improvements

Building on the performance improvements delivered in v4.13 & v4.14, Ditto 5.0 further optimizes data synchronization through tiered blob storage and protocol enhancements, making sync operations faster and more efficient.

​
What’s New

Version 5.0 introduces:
  • Tiered blob storage: Smaller sync updates avoid unnecessary disk I/O by using optimized storage tiers
  • Improved session handling: Better avoidance of session resets on reconnection when in-flight updates were lost
  • Optimized fsync policy: Document sync avoids forcing files to disk by default, decreasing I/O and improving latency

​
Impact

Applications upgrading to v5.0 will experience:
  • Faster sync operations: Reduced disk I/O overhead leads to quicker data synchronization
  • Lower latency: Optimized file handling decreases sync latency across the board
  • Better reconnection handling: Fewer redundant updates after temporary disconnections
  • Improved efficiency: Reduced disk operations lower memory and CPU pressure during sync

​
Local System Observability

Gain unprecedented visibility into how Ditto operates on your devices. Query real-time metrics, inspect runtime configuration, and monitor system health directly using DQL—giving you deeper insights than ever before to debug issues, optimize performance, and understand your application’s behavior.

​
New Virtual Collections

system:metrics
  • Query performance metrics and diagnostics in real-time
  • Access counters, timers, and other operational metrics via DQL
  • Monitor system health and performance without external tools
system:system_info
  • Query peer_key, database_id, and configuration settings
  • Inspect runtime configuration and system parameters
  • Useful for debugging and operational awareness
system:shared_statements
  • Inspect the query plan cache
  • View cached statements and their execution plans
  • Supports DELETE operations to clear specific cached statements
  • Helps optimize query performance and troubleshoot query planning
Local-Only Collections: System collections are local to each peer and are not replicated across the mesh. To query these collections on remote peers, use Remote Query from the Ditto Portal.

​
Usage Example

-- Query system information
SELECT * FROM system:system_info

-- View cached query statements
SELECT * FROM system:shared_statements

-- Access metrics
SELECT * FROM system:metrics
WHERE metric_name = 'sync.documents_synced'

-- Clear a specific cached statement
DELETE FROM system:shared_statements
WHERE statement_id = 'abc123'
These collections provide unprecedented visibility into Ditto’s internal state, making it easier to monitor, debug, and optimize your applications.

​
Additional Improvements

​
Reliability & Error Handling

  • Enhanced diagnostics: Improved logging when peers receive data that cannot be deserialized
  • Recovery mechanisms: Additional recovery paths for document deserialization errors
  • Smart log levels: Connection failures start at warning level, escalate to error only after repeated failures
  • Panic messages: Filtered to remove internal Rust machinery frames for improved readability

​
Networking Improvements

  • Graceful shutdown: Network connections close cleanly when Ditto is stopped
  • Faster disconnection detection: When a peer crashes, Ditto stops attempting to connect within 15 seconds (previously up to 75 minutes)
  • mDNS improvements: More reliable mDNS discovery, configurable service names, better address filtering
  • BLE improvements: Fixed connection issues on Android 9 and earlier devices
  • WebSocket BYOD support: Bring Your Own Discovery now supports WebSocket connections
  • Connection cleanup: Fixed deadlock where devices could fail to establish new P2P connections until restarted

​
DQL Engine Improvements

Beyond the query performance improvements detailed above, v5 includes:
  • Better error messages: Improved parser error messages for invalid DQL syntax
  • Transaction safety: Fixed deadlock scenarios in concurrent transactions
  • Index correctness: Fixed issues where index scans could yield incorrect results on document deletion

​
Logging & Diagnostics

  • Better disk utilization: On-disk logs resume writing to incomplete files, making better use of available space
  • Compressed size limits: Log file limits now apply to compressed size, significantly increasing retention
  • Explicit flushing: Logs explicitly flushed before aborting due to panic
  • Virtual collections: New system:metrics and system:system_info collections for DQL access to metrics and system information

​
Platform Support

  • Linux aarch64: Kotlin SDK now supports ARM64 Linux (Raspberry Pi, AWS Graviton, etc.)
  • Swift 6: Full Swift 6 support with Sendable conformance
  • 16KB alignment: React Native Android meets Google Play’s November 2025 requirement

​
Upgrading to v5

​
C++ Migration Guide

Upgrading to Ditto 5.0 requires updating your initialization code and migrating from legacy query APIs to DQL. The migration process involves:
  • Updating from DittoIdentity to DittoConfig-based initialization
  • Replacing legacy query builder operations with DQL statements
  • Migrating collection observers to DQL observers
  • Updating authentication patterns
For comprehensive migration instructions, code examples, and best practices, see the C++ v4 to v5 Migration Guide.

​
Terminology Updates

Ditto 5.0 updates terminology across the platform to align with industry standards and reduce confusion.

​
Database ID (formerly App ID)

  • appID → databaseID in all configuration methods
  • getAppId() → getConfig().databaseId in SDK APIs
  • Portal and documentation updated to use “Database ID” terminology
Why this matters: The term “App ID” caused confusion, particularly for mobile developers who associate “app” with the mobile application itself rather than the Ditto database instance. “Database ID” more accurately describes what the identifier represents: a unique identifier for your Ditto database that persists across all clients.

​
Ditto Server (formerly Ditto Cloud)

  • isConnectedToDittoCloud → isConnectedToDittoServer in presence APIs
  • Documentation updated to use “Ditto Server” terminology
Why this matters: This clarifies that the property indicates connection to any Big Peer (Ditto Server), not just those running in Ditto’s cloud service. This is more accurate for deployments using self-hosted Big Peers.

​
Migration

Update your code to use the new terminology:
// Database ID
auto config = ditto::DittoConfig::online_playground(
    "your-database-id",  // was: app_id
    "your-token"
);

// Presence API
if (peer.is_connected_to_ditto_server()) {
    // handle connection
}
The actual ID values and functionality remain unchanged - only the parameter and property names have been updated.

​
Breaking Changes

Ditto 5.0 is a major version release that removes deprecated APIs and legacy features. For migration guidance, see Migration Guidance.

​
Removed APIs

​
Legacy Query Builder (All SDKs)

All legacy query builder APIs have been removed:
  • store.collection() → Use DQL INSERT, UPDATE, EVICT statements
  • collection.find() → Use DQL SELECT queries
  • collection.findById() → Use DQL with _id filter
  • Live queries → Use DQL observers with store.registerObserver()
  • Write transactions → Use DQL store.write() with transactions

​
Legacy Initialization (All SDKs)

  • Identity classes and all subclasses removed
  • Ditto(identity:, persistenceDirectory:) constructors removed
  • Use DittoConfig factory methods and Ditto.open() instead

​
Sync Methods Moved

  • ditto.startSync() → ditto.sync.start()
  • ditto.stopSync() → ditto.sync.stop()
  • ditto.isSyncActive → ditto.sync.isActive

​
Other Removals

  • disableSyncWithV3() - no longer needed, v3 sync removed entirely
  • AttachmentToken - use dictionary variant
  • Transport diagnostics APIs - obsolete, removed
  • Various deprecated presence properties (queryOverlapGroup, meshRole, etc.)
  • Emoji log level headings - setting had no effect, removed

​
Behavioral Changes

Several default behaviors have changed in v5:
  • DQL strict mode: Now defaults to false - no schema definitions required, automatic CRDT type inference
  • String literals in DQL: Double quotes now delimit strings (not identifiers) for JSON compatibility
  • Subscription queries: Reject LIMIT and ORDER BY unless DQL_RESTRICT_SUBSCRIPTION=false
  • Observer ordering: Observers require explicit ORDER BY clause for stable ordering
  • WebSocket sync: Disabled by default in new TransportConfig instances - must explicitly enable
  • Document IDs: null is no longer allowed as a document ID
These behavioral changes may affect existing code. Review your DQL queries and subscription logic when migrating to v5.

​
SDK Size Reduction

The removal of legacy APIs has reduced SDK footprint by approximately 25%, resulting in:
  • Smaller application binary sizes
  • Reduced memory usage
  • Faster SDK initialization
  • Simpler maintenance and debugging

​
C++ Specific Changes

​
C++-Specific Changes

The C++ SDK has additional platform-specific changes in v5.0 beyond the common breaking changes.

​
API Changes

Query Arguments:
  • SyncSubscription::get_query_args() removed - Use get_query_arguments(), get_query_arguments_cbor_data(), or get_query_arguments_json_string() instead
  • get_query_arguments() no longer guarantees strict equality to original arguments due to serialization roundtrip
  • Added get_query_arguments_cbor_data() and get_query_arguments_json_string() for custom deserialization
  • SyncSubscription no longer inherits from Observer
Store Observer:
  • Added get_query() method to fetch the query associated with a store observer
  • Added get_query_arguments(), get_query_arguments_cbor_data(), and get_query_arguments_json_string() methods
  • get_observers() now returns unordered_set instead of set
  • Removed observers property - Use get_observers() method instead
  • StoreObserver no longer inherits from Observer
  • StoreObservationHandlerWithNextSignal renamed to StoreObservationHandlerWithSignalNext for consistency
Peer & Connection Properties:
  • Renamed Peer::peer_key_string to Peer::peer_key
  • Renamed Connection::peer_key_string1 and Connection::peer_key_string2 to Connection::peer1 and Connection::peer2
  • Renamed ConnectionRequest::peer_key_string() to ConnectionRequest::peer_key()
  • Renamed Peer::is_connected_to_ditto_cloud to Peer::is_connected_to_ditto_server
Other Changes:
  • Removed Ditto::close() - Rely on Ditto destructor for cleanup
  • Removed nlohmann::json from ditto namespace (inclusion was unintentional)
  • Removed Store() constructor - Use Ditto::get_store() to obtain a reference
  • PresenceCallback function argument changed to const PresenceGraph& instead of PresenceGraph to avoid unnecessary copying
  • QueryResult::mutated_document_ids() now returns std::vector<std::string> instead of std::vector<DocumentId>
  • Store::observers is now private - Use Store::get_observers() instead
  • Added virtual destructor for AuthenticationCallback class for proper cleanup

​
Internal API Reorganization

The following types have been moved to ditto::internal namespace and are not intended for public API use:
  • PresenceGraphInternal
  • Arc<> and ArcPointee<>
  • AsyncContextBase<> and AsyncContext<>
  • AttachmentFetcherCtx
The following methods are now private:
  • Attachment::get_type()
  • Attachment::get_id_internal()

​
Logging Changes

  • Log::set_log_file() removed - Use Log::export_to_file() instead
  • Log::disable_log_file() removed - No replacement needed
  • Log::Callback type changed from raw function pointer to std::function<void(LogLevel, const std::string&)> for lambda support

​
Disk Usage API

  • DiskObserverContext renamed to DiskUsageObserver
  • DiskUsage::exec() renamed to DiskUsage::item()
  • DiskUsageChild renamed to DiskUsageItem

​
Full Changelog

​
C++ Specific Changes

Changed:
  • get_observers method on StoreObserver instances now returns an unordered_set instead of a set (#CORE-298)
  • StoreObserver no longer inherits from Observer (#CORE-298)
  • StoreObservationHandlerWithNextSignal renamed to StoreObservationHandlerWithSignalNext for consistency with other SDKs (#SDKS-1109)
  • PresenceCallback function argument is now const PresenceGraph& instead of PresenceGraph, to avoid unnecessary copying of the graph in memory (#SDKS-1164)
  • PresenceGraphInternal has been moved to the ditto::internal namespace. It is not intended for use as part of the public API (#SDKS-1164)
  • Renamed Peer::peer_key_string to Peer::peer_key (#SDKS-1181)
  • Renamed Connection::peer_key_string1 and Connection::peer_key_string2 to Connection::peer1 and Connection::peer2 (#SDKS-1181)
  • Renamed ConnectionRequest::peer_key_string() to ConnectionRequest::peer_key() (#SDKS-1181)
  • Renamed Peer::is_connected_to_ditto_cloud to Peer::is_connected_to_ditto_server to reflect new product terminology (#SDKS-1235)
  • Arc<> and ArcPointee<> have been moved to the ditto::internal namespace. They are not intended for use as part of the public API (#SDKS-1470)
  • AsyncContextBase<> and AsyncContext<> have been moved to the ditto::internal namespace. They are not intended for use as part of the public API (#SDKS-1470)
  • Attachment::get_type() and Attachment::get_id_internal() are now private members of the Attachment class. They are not intended for use as part of the public API (#SDKS-1470)
  • AttachmentFetcherCtx has been moved to the ditto::internal namespace. It is not intended for use as part of the public API (#SDKS-1470)
  • QueryResult::mutated_document_ids() now returns std::vector<std::string> instead of std::vector<DocumentId> (#SDKS-1470)
  • Store::observers is now private. Use Store::get_observers() to access the set of observers (#SDKS-1470)
  • SyncSubscription no longer inherits from Observer (#17003)
  • The ditto::Log class is now named ditto::Logger for consistency with other Ditto SDKs (#SDKS-2379)
  • DiskObserverContext renamed to DiskUsageObserver, DiskUsage::exec() renamed to DiskUsage::item(), and DiskUsageChild renamed to DiskUsageItem, for consistency with names in other Ditto SDKs and documentation (#SDKS-595)
  • Log::Callback type changed from raw function pointer to std::function<void(LogLevel, const std::string&)>, enabling support for lambda functions and function objects with captures (#SDKS-747)
  • Moved ditto_semver_version() to the ditto::internal namespace. This function is not part of the public API. Use Ditto::get_version() instead (#SDKS-2097)
Added:
  • Virtual destructor for the abstract AuthenticationCallback class, to support proper cleanup of instances of derived classes (#SDKS-1470)
  • get_query_arguments_cbor_data and get_query_arguments_json_string methods to SyncSubscription instances. If you want to deserialize query arguments into a specific type, then use these and deserialize things as required (#17003)
  • get_query method to StoreObserver to allow fetching the query associated with a store observer (#CORE-298)
  • get_query_arguments, get_query_arguments_cbor_data and get_query_arguments_json_string methods to StoreObserver to allow fetching the query arguments associated with a store observer (#CORE-298)
  • Ditto::get_version() which returns semantic version of the Ditto SDK. Example: “5.1.2” (#SDKS-2097)
Removed:
  • get_query_args method from SyncSubscription. Use one of get_query_arguments, get_query_arguments_cbor_data, or get_query_arguments_json_string instead. Note that get_query_arguments (the replacement for get_query_args) no longer guarantees that the returned value will be strictly equal to the original arguments passed when registering the sync subscription. This is due to a serialization roundtrip, which may affect equality checks, particularly for non-primitive values (#17003)
  • observers property from StoreObserver instances. Use the get_observers method instead (#CORE-298)
  • Ditto::close(). Rely on the Ditto destructor to clean up (#SDKS-1110)
  • nlohmann::json from the ditto namespace. The inclusion was unintentional (#SDKS-1124)
  • Store() constructor. Use Ditto::get_store() to obtain a reference to a Store instance (#SDKS-1163)
  • AbstractDocumentPath class. Use DQL path expressions instead (#SDKS-1470)
  • Arc type alias. Internal type, no replacement needed (#SDKS-1470)
  • AsyncContext type alias. Internal type, no replacement needed (#SDKS-1470)
  • Authenticator::login_with_token(). Use Authenticator::login() instead (#SDKS-1470)
  • CborValue type alias. Use nlohmann::json instead (#SDKS-1470)
  • Collection class. Use DQL INSERT, UPDATE, EVICT statements instead (#SDKS-1470)
  • CollectionsEvent class. Use DQL observers instead (#SDKS-1470)
  • Ditto::Ditto(jobject, std::shared_ptr<Identity>, std::string) constructor. Use Ditto::open(DittoConfig) instead (#SDKS-1470)
  • Ditto::Ditto(std::shared_ptr<Identity>, std::string) constructor. Use Ditto::open(DittoConfig) instead (#SDKS-1470)
  • Ditto::get_is_sync_active() method. Use .get_sync().is_active() instead (#SDKS-1470)
  • Ditto::get_small_peer_info() method. Use presence APIs instead (#SDKS-1470)
  • Ditto::set_android_context() method. Pass context to DittoConfig::online_playground() or DittoConfig::online_with_authentication() instead (#SDKS-1470)
  • Ditto::start_sync() method. Use .get_sync().start() instead (#SDKS-1470)
  • Ditto::stop_sync() method. Use .get_sync().stop() instead (#SDKS-1470)
  • Ditto::sync() method. Use Ditto::get_sync() instead (#SDKS-1470)
  • Document class. Use DQL query results instead (#SDKS-1470)
  • DocumentId class. Use document _id field directly (#SDKS-1470)
  • DocumentIdPath class. Use DQL path expressions instead (#SDKS-1470)
  • DocumentPath class. Use DQL path expressions instead (#SDKS-1470)
  • EvictResult type alias. Use DQL EVICT statement results instead (#SDKS-1470)
  • Identity class and all subclasses (OfflinePlayground, OnlinePlayground, OnlineWithAuthentication, SharedKey). Use DittoConfig factory methods instead (#SDKS-1470)
  • InsertResult type alias. Use DQL INSERT statement results instead (#SDKS-1470)
  • LiveQuery class. Use DQL observers instead (#SDKS-1470)
  • LiveQueryEvent class. Use DQL observers instead (#SDKS-1470)
  • LiveQueryMove class. Use DQL observers instead (#SDKS-1470)
  • MutableDocument class. Use DQL mutation statements instead (#SDKS-1470)
  • MutableDocumentPath class. Use DQL path expressions instead (#SDKS-1470)
  • PendingCollectionsOperation class. Use DQL instead (#SDKS-1470)
  • PendingCursorOperation class. Use DQL instead (#SDKS-1470)
  • PendingIDSpecificOperation class. Use DQL instead (#SDKS-1470)
  • Presence::exec() method. Use Presence::graph() instead (#SDKS-1470)
  • RemoveResult type alias. Use DQL EVICT statement results instead (#SDKS-1470)
  • ScopedWriteTransaction class. Use DQL transactions instead (#SDKS-1470)
  • SingleDocumentLiveQueryEvent class. Use DQL observers instead (#SDKS-1470)
  • SmallPeerInfo::set_sync_scope() method. Use DQL “ALTER SYSTEM SET USER_COLLECTION_SYNC_SCOPES” instead (#SDKS-1470)
  • SmallPeerInfo::sync_scope() method. Use DQL “SHOW USER_COLLECTION_SYNC_SCOPES” instead (#SDKS-1470)
  • SortDirection class. Use DQL ORDER BY ASC/DESC instead (#SDKS-1470)
  • Store::collection() method. Use DQL statements instead (#SDKS-1470)
  • Store::collections() method. Use SELECT * FROM system:collections instead (#SDKS-1470)
  • Store::queries_hash_mnemonic() method. No replacement needed (#SDKS-1470)
  • Store::queries_hash() method. No replacement needed (#SDKS-1470)
  • Store::write() method. Use DQL transactions instead (#SDKS-1470)
  • Subscription class. Use DQL observers instead (#SDKS-1470)
  • SyncScope enum (#SDKS-1470)
  • UpdateResult class. Use DQL mutation results instead (#SDKS-1470)
  • UpdateResultItem class. Use DQL mutation results instead (#SDKS-1470)
  • UpsertResult type alias. Use DQL UPSERT statement results instead (#SDKS-1470)
  • WriteStrategy class. Use DQL INSERT vs UPSERT statements instead (#SDKS-1470)
  • WriteTransaction class. Use DQL transactions instead (#SDKS-1470)
  • WriteTransactionPendingCursorOperation class. Use DQL instead (#SDKS-1470)
  • WriteTransactionPendingIDSpecificOperation class. Use DQL instead (#SDKS-1470)
  • WriteTransactionResult class. Use DQL transaction results instead (#SDKS-1470)
  • Log::set_log_file() method. Use Log::export_to_file() instead (#SDKS-1727)
  • Log::disable_log_file() method. No replacement needed (#SDKS-1727)
  • Store::collection_names() method. Use DQL query SELECT name FROM system:collections instead (#SDKS-1743)
  • Ditto::disable_sync_with_v3(). This function is no longer needed in v5 (#SDKS-2069)
  • Ditto::get_sdk_version(), which returned an encoded platform/language/version string. Instead, use Ditto::get_version(), which returns a semantic version identifier (#SDKS-2097)
  • Ditto::get_app_id(). Use Ditto::get_config().database_id instead (#SDKS-2330)
  • Log::get_emoji_log_level_headings_enabled() and Log::set_emoji_log_level_headings_enabled() methods (#SDKS-2603)
  • Ditto::get_emoji_log_level_headings_enabled() and Ditto::set_emoji_log_level_headings_enabled() convenience methods (#SDKS-2603)

​
5.0.0 Common Changelog

Performance:
  • Improved the underlying representation of Ditto documents for better performance (#17509)
  • Document synchronization between peers now batches updates more efficiently, reducing processing time for large document sets (#19273)
  • Document sync avoids sending large, redundant updates after reconnecting a long-dormant session between two peers that are otherwise well-synced with the mesh (#DS-433)
  • Improved avoidance of large redundant doc sync updates after session reconnect, based on the number of diffs sent in the initial post-reconnect update (#DS-475)
  • Faster initial sync when processing rkyv-encoded document diffs (#DS-773)
  • Faster eviction of indexed documents when using rkyv (#DS-774)
  • Improved performance of initial index generation when using rkyv as a document format (#DS-774)
  • Reduced redundant replication GC work during rapid eviction bursts by debouncing and coalescing compatible GC tasks (#CORE-1466)
  • Synchronization protocol enhanced to better avoid resetting sessions on reconnect if in-flight updates were lost (#DS-820)
  • Document Sync now uses tiered blob store for update file storage by default, allowing smaller updates to avoid unnecessary disk I/O (#DS-836)
  • Document sync implementation avoids forcing outbound update files to disk by default, decreasing disk I/O and improving sync latency (#DS-921)
  • Observers to avoid sort by id on non order by query / add limit to sort operator (#QE-261)
  • Improved out-the-box performance for larger statements (#QE-377 & QE-378)
  • Improved the performance of IN-list evaluation for large lists of static values (#QE-386)
Fixed:
  • An issue with BLE on some Android 9 and earlier devices that prevented connection establishment (#17760)
  • Multiple concurrent DQL transactions can no longer possibly lead to a deadlock (#17816)
  • A bug where x509 refresh could speed up uncontrollably (#17947)
  • A bug where a peer could get stuck with incorrect connection information (#17967)
  • Network connections close gracefully when Ditto is stopped (#18053)
  • The system:data_sync_info collection may briefly report sync immediately after connecting (#18137)
  • An issue where peers could fail to connect other local peers via mDNS on macOS (#18488)
  • A bug that meant that document id indexes were not created (#18512)
  • A bug where forced TCP connections would be retried more frequently than expected (#19727)
  • A bug where mDNS registration would fail with NamingConflict (#20411)
  • mDNS discovery should support TCP and UDP independently (#20490)
  • Connection manager now correctly cleans up orphaned connecting state when tasks are cancelled during shutdown (e.g., during auth refresh), preventing “Already connecting” errors on reconnection (#20377)
  • A deadlock could occur where a device would fail to establish new P2P connections until restarted (#20810)
  • A race condition where fetchAttachment could permanently fail to find locally-created attachments due to stale in-memory cache entries (#21146)
  • A bug that caused small peers to re-upload remotely requested files (e.g. logs) on every startup (#CORE-810)
  • High latency when write transactions trigger evictions (#CORE-1453)
  • Live queries being unable to use indices (#DS-447)
  • A rare scenario where an attachment fetch could be delayed by up to 60 seconds (#DS-461)
  • Deadlock in sync session metadata cleanup caused cascading lockups for doc sync and/or local doc store access (#DS-485)
  • Post eviction cleanup of disconnected document sync sessions now retains metadata for non-evicted documents (#DS-487)
  • Document sync session metadata became unlinked after session reset (#DS-509)
  • DQL SELECT queries on indexed collections could deadlock if executed in an explicit transaction (#DS-648)
  • DQL queries using indexes could yield wrong results upon document deletion (#DS-851)
  • Inconsistent internal hashing of Document IDs could cause failure to sync documents (#DS-944)
  • Premature connection cleanup causing duplicate connections (#NETW-1021)
  • When another peer crashes, Ditto will stop attempting to connect to it in under 15 seconds. Previously, connection attempts would occur for up to 75 minutes at 5 second intervals (#NETW-856)
  • Changed live queries to use the new streaming interface for query execution (#QE-261)
  • Use of DISTINCT with ORDER BY & OFFSET/LIMIT (#QE-281)
  • In BP profile directive does not produce path in collection scan operator profile (#QE-294)
  • Handling of references to group keys that include array element selection (#QE-306)
  • Use deterministic summaries for documents created as part of observer evaluation (#QE-310)
  • Removed fabricated descriptor information from collection scans in profile/explain information (#QE-315)
  • Performance of the DQL parser for large statements (#QE-321)
  • DQL duration function floating-point rounding errors (#QE-402)
  • Incorrect association of terms following an IN expr clause (#QE-418)
  • Executing index scans in observers preserves document ids to provide consistent ordering (#QE-429)
  • The query planner misses using a Filter operator if collection scan filter pushdown is enabled (#QE-437)
  • Small peer collection scan handles offset and limit incorrectly (#QE-438)
  • Changed DQL planner index selection process to correctly handle predicate paths that haven’t been formalised avoiding incorrect index selection (#QE-456)
  • The DQL query planner will no longer generate index scans on fields which have been specified in a COLLECTIONS clause as the variant specified may differ from the indexed variant. This prevents incorrect query results arising from the mismatch (#QE-475)
  • The small peer DQL query planner to not produce index scans when DQL_STRICT_MODE is set to true. This avoids incorrect results from filters on fields where the latest variant is not REGISTER (#QE-478)
  • The store SQLITE logging callback has been changed to not report schema change errors avoiding flooding the logs with spurious warnings from SQLITE internal operations (#QE-479)
  • A panic in the DQL parser when an invalid JSON directive is used prior to the PROFILE keyword (#QE-490)
  • -9223372036854775808 is now parsed and handled as an integer value by DQL (#QE-499)
  • Queries in the legacy language might fail to surface documents with settable counters (#QE-512)
  • Documentation for on-disk logger incorrectly stated logs are retained for 3 days; actual retention is 15 days (#SDKS-1726)
  • FFI resource cleanup now safely handles null context pointers in release callbacks, preventing potential null pointer dereferences during Ditto shutdown (#SDKS-2744)
  • Attachment callback dispatch now properly handles thread pool dispatch failures by retaining context to prevent use-after-free errors when the callback pool is unavailable (#SDKS-2744)
  • Transport watchdog no longer logs spurious “Address already in use” errors when restarting TCP/HTTP servers. The watchdog now waits for previous server tasks to fully exit and release their sockets before attempting to rebind ports, preventing EADDRINUSE races when servers are invalidated during e.g. app backgrounding (#SPO-127)
  • Attachment garbage collection now removes empty shard directories, preventing inode leaks (#SPO-158)
  • Bug which meant a Ditto starting log failed to be logged (#SPO-626)
  • Android devices that fail to acquire an IP address will now continue to sync over LAN (#TRAN-725)
  • A long-running on connecting callback no longer causes ReceiveTimeout failures in connectivity (#TRAN-729)
Changed:
  • network_enable_multihop system parameter renamed to network_enable_ngn, gating NGN features which include multihop (#17882)
  • Enabled opentelemetry tracing in core (#18478)
  • Connection failures now use smart log levels, starting at warning and escalating to error only after repeated failures (#18779)
  • mDNS now only advertises connectable addresses based on TCP server binding address. TCP Client now attempts to connect to all addresses in parallel (#18930)
  • The default value of ENABLE_ATTACHMENT_PERMISSION_CHECKS store configuration parameter to false (#18931)
  • mDNS service name is now configurable via the transports_mdns_service_name SystemParameter (default: _http-alt), ensuring both TCP and UDP transports use the same service name (#20289)
  • Panic messages now filter out internal Rust machinery frames (backtrace capture, panic handling, runtime startup) for improved readability, with an environment variable DITTO_PANIC_WITH_FULL_STACK_TRACE=1 available to show complete stack traces when needed (#20401)
  • On-disk logs now make better use of available disk space by resuming writing to incomplete files (#CORE-561)
  • On-disk log file limits now apply to the compressed size, significantly increasing log retention (#CORE-729)
  • Write transaction diagnostic logs now include the blocking transaction’s current operation, elapsed time, and queue depth (#CORE-1449)
  • More information is logged at debug level around peer GC and eviction progress including what remote peers are being deleted (#CORE-1455)
  • Significantly lowered the maximum values of some system parameters governing the on-disk rotating file logger’s behavior (#CORE-848)
  • More information is logged when a Ditto peer receives data that it cannot deserialize due to a hash mismatch (#CORE-897)
  • We now make an additional attempt to recover from some kinds of document deserialization errors, which may reduce errors or crashes due to deserialization (#CORE-916)
  • Document sync protocol now supports rkyv-encoded diffs, for more efficient initial synchronization (#DS-531)
  • TCP server is no longer auto-enabled when TCP is disabled in config, NGN and UDP is enabled with LAN discovery (#NETW-1039)
  • Mutators are preserved for Big Peer V4 (#QE-126)
  • Disallow null as an id (#QE-202)
  • DQL_STRICT_MODE default to false (#QE-267)
  • Planning for statements using DISTINCT projections, in some circumstances (#QE-282)
  • system:vitals outputs timers as sub-document (#QE-312)
  • DQL queries run via observers now require that the user provides stable ordering themselves via a suitable ORDER BY clause (#QE-427)
  • Double quotes delimit strings, not identifiers (JSON compatibility) (#QE-44)
  • DQL query planner now recognises additional index access plans that eliminate the need for document retrieval (#QE-449)
  • Added the key sorting direction to the system:indexes virtual collection output (#QE-467)
  • The DQL planner to automatically convert equality filters on the _id field to ID scans, improving performance by skipping index use when exact document IDs are known up front (#QE-469)
  • The DQL query planner can now create a query plan that defers fetching full documents until after sorting and application of offset and limit, if the index access portion of the plan can support it. This means performance improvements for queries with affected plans (#QE-473)
  • Revised the Query Engine DISTINCT operator to stream results reducing memory overhead (#QE-474)
  • The DQL planner to consider additional cases where an index may cover a query leading to improved performance in those scenarios (#QE-491)
  • The DQL Query Engine planner can now produce specialised higher performance plans for Ditto Server COUNT(*) queries that include simple filters (#QE-510)
  • The Query Engine DQL planner can now generate index-access plans against Ditto Server improving performance queries with filters that can be applied when scanning an index and where combining multiple index scans is beneficial (#QE-511)
  • The Query Engine intersect scan operator can now stop before all inputs have completed, once it has been established that no further complete intersections can be produced, improving the performance where the number of values produced by each input differs greatly (#QE-530)
  • Added specialisation of simple queries to take advantage of small peer indexing (#QE-266)
  • Added support for microseconds to duration scalar functions (#QE-396)
  • Removed the Parseable query trait to minimise the impact of the query parser maintenance on the monorepo (#QE-421)
  • System collection system:ditto_metrics renamed to system:metrics for consistency with naming conventions. Existing queries using system:ditto_metrics will need to be updated to use system:metrics (#SDKS-2653)
  • Ditto shutdown logging promoted to info level (#SPO-626)
Added:
  • ENABLE_ATTACHMENT_PERMISSION_CHECKS ALTER SYSTEM parameter to be set to false to avoid certain rare cases of attachment fetcher hanging (#18016)
  • Bring Your Own Discovery now supports WebSocket connections (#18965)
  • system:metrics virtual collection for DQL access to metrics (#MESHCON-53)
  • Introduced new entries to the system:system_info collection for peer_key and database_id (#19435)
  • DATA_SYNC_ENABLED system parameter which, while set to false, halts the data sync machinery, but without loss of network connectivity (allowing for operations such as remote query) (#19643)
  • DITTO_USE_TIERED_BLOB_STORE_DOC_SYNC system parameter env var to improve the performance of doc sync in heavy mesh scenarios, at the expense of sync chatter overhead upon restart (#19847)
  • The new transport_tcp_connect_timeout system parameter which shortcuts long platform-based TCP connection timeouts (#20782)
  • Explicit log flushing before aborting due to panic (#CORE-723)
  • system:system_info virtual collection for DQL access to system info (#CORE-751)
  • small_peer_info subscription queries now include arguments (#DS-1027)
  • system:system_info subscription queries now include arguments; query location moved from key to value.query (#DS-1027)
  • Improved logging for document deserialization errors (#DS-568)
  • System parameters that can be used to configure sqlite pragmas (#DS-682)
  • System parameter doc_sync_outbound_update_fsync_policy to govern the use of fsyncs when creating doc sync update files (#DS-817)
  • Channel open retries and watchdog to tear down VirtualConnections that never open a channel, preventing resource exhaustion from stalled peers (#NETW-1098)
  • A system parameter transports_dns_sd_backend for choosing the mDNS backend implementation (#NETW-940)
  • Ability to specify documents to insert in arrays in DQL (#QE-118)
  • USE IDS LIST syntax (#QE-119)
  • Support for settable counters to DQL (#QE-130, QE-393, QE-394, QE-395)
  • Specialisation of COUNT(*) DQL queries on the big peer (#QE-138)
  • Subscription queries reject LIMIT and ORDER by unless DQL_RESTRICT_SUBSCRIPTION is set to false (#QE-218)
  • INTERSECT and UNION (index) scans (#QE-230_and_240)
  • Covering index scans (#QE-239)
  • Support for the BETWEEN DQL expression (#QE-26)
  • Streaming interfaces for query execution (#QE-262)
  • Array & object search syntax to DQL (#QE-265)
  • Ability to inline consumers into producer operators (#QE-272)
  • Consolidated results across subservers for queries against ACTIVE_REQUESTS, REQUEST_HISTORY and VITALS (#QE-273)
  • Array transformation DQL syntax (#QE-274)
  • Object transformation syntax to DQL (#QE-276)
  • Promethus metrics to the DQL Query engine (#QE-296)
  • Automatic generation of USE IDS clause from _id equality predicates when possible (#QE-313)
  • A shared statement cache and virtual collection (#QE-316)
  • Configurable concurrent request limit to the Query engine (#QE-317)
  • Periodic dumping of request history cache entries to the log (SP only) (#QE-322)
  • The ability for the shared statement to store, verify, amend and use existing plans for qualifying statements (#QE-324)
  • PROFILE keyword as the equivalent to the #profile directive (#QE-336)
  • CASE statement (#QE-41)
  • Support for settable counters in INSERT DQL statements (#QE-406)
  • The statement caches invalidates existing statements if the default directives change (#QE-407)
  • Support for extended string literals that can contain escape sequences in DQL (#QE-410)
  • Support for hexadecimal numeric constants in DQL statements (#QE-413)
  • Statement cache resizing (#QE-414)
  • The ability for Remote Query to handle DELETE, EVICT, TOMBSTONE DQL statement (#QE-441)
  • Enable sending all statements via the SYNC CONTEXT statement (#QE-447)
  • Support for DQL DELETE against the system:shared_statements virtual collection (#QE-450)
  • Publish sync scopes in small peer info document (#SPO-276)
Removed:
  • The static path option has been removed from TransportConfig. Please use attachments to serve content instead (#TRAN-256)
  • Deprecated fields in presence have been removed across all SDKs (queryOverlapGroup, meshRole, approximateDistanceInMeters, rssi) (#TRAN-680)