Kotlin
ADDED: Support for specifying the mDNS connection ports for Small Peer SDK instances
Allows you to specify a specifc port that can will be used for mDNS announcements and connections on a given device. This can be used for connect with peers across a VLAN while scanning for open ports.
Not setting the config will result in the current default behavior of a random port being assigned.
If running multiple application on the same device, for example a debug and/or test flight application along side a production application, each application needs to have a unique port.
Failure to do this will result in only one application being able to establish connections at a given time.
Setting the config
The config can be set using DQL ALTER SYSTEM and the tcp_server_bind_mdns_server_port config.
This command should be run using the ditto.store.execute(...) command within your application.
tcp_server_bind_mdns_server_port needs to be set ***before ***startSync() is called to be applied.
FIXED: A finalizer issue that was causing crashes when closing sync subscriptions (#13849, #13732, #13738)
Resolves a race condition that could be hit when closing a sync subscription or stopping sync that caused Ditto to crash.
IMPROVED: (Linux/Android) Increased mDNS multicast TTL from 1 to 255 to allow for cross-VLAN announcements (#13590)
To support scenarios with cross-VLAN device discovery we increated the TTL to ensure message have enough time to reach their destination and get acknowledged.
Fixed: Intermittent crash when finalizing a legacy DittoSubscription that contained a sort() clause. (#13394)
Resolves and issue where using sort() in sync subsriptions with the legacy query builder would result in a crash.
Fixed: Included necessary symbols so that Ditto Core crashes can be properly symbolicated with crash reporters. (#13696)
Previously not all symbols were being included in the package bundle and this led to crashes not being able to be symbolicated and ultimately made readable.
Now the Kotlin bundle includes the correct packages
The extra symbols slightly increase the bundle size. If this additional size is an issue for your application, we recommend stripping these symbols when compiling your production build.
However, please note that stripping symbols will make crash reports in production unreadable, which could lead to longer resolution times for live site incidents.
FIXED: An issue causing a race condition with a Live Query callback. (#13180)
This fix addresses the issue causing a Live Query callback to receive a delta update, known as a diff, before the initial document.
FIXED: A compatibility issue between documents created using different CRDT versions. (#13236)
Previously, there was an issue in which two documents on different internal versions could fail to integrate changes created using an older CRDT version.
IMPROVED: Logging for batch write transactions. (#13339, #13339)
Due to a race condition caused by the logging system mistakenly identifying the previous transaction as blocking the current one, the logger would inaccurately indicate the current transaction as being blocked by the previous one from an unknown originator.
FIXED: An issue in the Small Peer datastore causing slow performance or process termination. (#13395)
This issue was due to an aging dependency with a race condition, which caused thread tasks to get stuck and transactions to be blocked for extended periods.
IMPROVED: The Ditto SDKs compilation performance. (#13174)
We’ve optimized compilation by reducing the size of each binary slice of the Ditto SDKs by 19% and fixing incompatibility with PLCrashReporter.
FIXED: Kotlin and Java Android devices now appear as JVMBased in the Device Dashboard instead of Unknown. (#13251)
FIXED: An issue causing a crash with the message gatt must not be null. (#13411)
ADDED: mesh_chooser_avoid_redundant_bluetooth system parameter. (#12984)
This new system parameter allows greater control over the use of Bluetooth Low Energy (LE) connections in Ditto’s peer-to-peer mesh network. When set to false, Ditto creates Bluetooth LE connections alongside other peer-to-peer connections, serving as a fallback if the primary transport, such as Apple Wireless Direct Link (AWDL) or LAN, fails.
However, before you enable the mesh_chooser_avoid_redundant_bluetooth parameter, consider the following technical tradeoffs:
- Benefits:
- Provides additional fallback to maintain connection stability when primary connections cease to function.
- Reduces time to establish new connections since, by having multiple fallback options, Ditto can more quickly re-establish connections.
- Drawbacks:
- Increases network traffic since using multiple transports results in additional network overhead
- Using Bluetooth LE alongside other transports increases resource consumption. In the worst case, enabling this setting can result in slower sync performance and battery drain.
To enable the mesh_chooser_avoid_redundant_bluetooth parameter, use the following DQL query:
ALTER SYSTEM SET mesh_chooser_avoid_redundant_bluetooth = false
CHANGED: The AuthClient by improving the log message for certificate issues returned from the Big Peer Authentication Service. (#13138)
FIXED: A performance-related issue affecting bulk eviction operations for documents with attachments. (#12331)
ADDED: Improved data compression in the handshake protocol when establishing connections to other peers. (#11264)
This allows peers with large permission sets to establish connections on low bandwidth transports like Bluetooth LE more quickly.
FIXED: The HTTP protocol by adding content-length: 0 header compliance in empty POST requests. (#12346)
Before, if you invoked an empty POST request, the cloud load balancer would reject your request with a 411 - Length Required error message.
Now you must explicitly state that the body of your POST request is empty by setting content-length: 0 in the header of your HTTP API call.
FIXED: Handling of malformed subscription queries in the legacy query builder API by adding validation logic early to ensure invalid queries are identified early and logged. (#12463)
There was an issue where invoking subscribe() on an invalid query caused sync to be silently disabled.
Invalid queries in the subscribe() method are now identified, disabled, and logged as errors.
FIXED: Subscription assumption logic so all acknowledgments are processed, regardless of subscription changes. (#12541)
In scenarios where subscriptions rapidly changed back and forth while documents were being concurrently altered, documents would diverge at merge.
We’ve removed subscription logic ignoring acknowledgements based on assumed subscription mismatch and introduced salting functionality to ensure each document is unique for each occurrence of a subscription. This helps differentiate between new and reverted subscription states.
FIXED: Sync inefficiency where changing a subscription resulted in unnecessary syncing of redundant update files. (#12967)
To eliminate redundant data from being synced unnecessarily, regardless of whether the remote peer previously acknowledged them, we’ve added validation checks ensuring that only unacknowledged data is synced.
This fix reduces sync update file sizes, leading to more performant session sync updates, particularly in apps consisting of large, stable subscriptions and infrequent document changes.
CHANGED: Reduced Bluetooth LE warn log noise. (#12674)
Changed: handle reverse DNS lookup queries in Android mdns to avoid mdns failing to connect in specific network configs
CHANGED: Android mdns to handle reverse DNS lookup queries. (#13000)
By adding reverse DNS lookup queries, Android mdns no longer fails to connect in specific network configurations.
FIXED: Wi-fi Aware so it is available on API level 32. (#12631)
FIXED: Missing version metadata. (#12843)
Now version metadata is correctly included in the string returned from Ditto.sdkVersion.
FIXED: Misleading errors related to attachment fetch operations. Now, associated error logs are more accurate and clear. (#12409)
FIXED: A specific issue where incorrect sync metadata rollback caused problems with data consistency at merge. (#12422)
FIXED: The documentation for the DittoStore.fetchAttachment API method. (#11783)
Before this patch release, the documentation incorrectly stated that the return value had to be kept in scope for the fetch to complete. As of this release, the documentation specifies that the return value does not need to be kept in scope.
ADDED: DittoExperimental.constructDittoFailable() API method. (#11801)
Previously, your app crashed if the persistence directory was already in use by another Ditto instance. Now, rather than crashing in such scenarios, Ditto throws a DittoError.LockedWorkingDirectoryError error.
ADDED: DittoExperimental.constructDittoFailable() API method.
Now, you can use this experimental alternative to the original Ditto constructor to prevent crashes when the persistence directory is still in use by another Ditto instance at the time of calling the constructor. Now, rather than crashing in such scenarios, Ditto throws a DittoError.LockedWorkingDirectoryError error. (#11801)
FIXED: Exceptions that occur while creating or fetching attachments now include more details about the underlying cause in their error message for easier debugging. (#12235)
Fixed: An issue that prevented some attachment fetcher events from being delivered to event handlers. (#12331)
FIXED: Closing an attachment fetcher that failed to start downloading no longer results in an exception. (#12490)
FIXED: Handling of document-paths containing non-alphanumeric characters. Previous versions may have removed these characters from document-paths. (#12556)
FIXED: The legacy query language error-throwing mechanism to accept invalid field names properly. (#11888)
After upgrading to version 4.7 of the SDK, you may notice unexpected errors while using the legacy query builder language. These errors are likely due to invalid field names or path expressions previously undetected.
In addition, make sure to start unquoted_string with an underscore (_), a letter, or an alphanumeric character. For example, the following equivalent regular expression: [A-Za-z_][A-Za-z0-9_]*.
In the legacy query builder language, invalid field names and path expressions now throw explicit errors. Previously these invalid names did not throw and would cause undefined behavior.
Note: This new error-throwing behavior applies only to the legacy query builder language; DQL is unaffected by this improvement.
REMOVED: The is_connected_to_big_peer field from the Small Peer Info document. (#11883)
CHANGED: The small peer info JSON metadata by increasing the depth limit to 64 levels. (#11891)
IMPROVED: Transaction contention time for the ATTACHMENT data type, reducing potential timeouts. (#11893)
Before this improvement, concurrent write transactions were long-lived, potentially causing deadlocks and request timeouts. Now you’ll experience more effective handling of multiple attachments being processed at the same time.
FIXED: The issue causing sync crashes due to missing update files. (#11952)
With the latest release, Ditto handles unexpected situations or errors in a way that allows it to recover and continue functioning without crashing or causing disruptions.
ADDED: Parameters for adjusting certain settings during runtime. (#11968)
This rollout includes new system parameters for the MeshChooser feature for configuring a limited set of variables, allowing you to adjust certain settings during runtime.
RESOLVED: The warning message when peers connect is no more. (#11977)
The warning message previously logged when peers connected within the mesh is no longer recorded or printed.
IMPROVED: Sync performance by sending the entire state of the data. (#12029)
Previously, when syncing in mixed mesh environments from Ditto SDK version 4.0.0 of the SDK to version 3.0.0, only partial updates were transmitted. With this release, all updates are transmitted, resulting in greater data consistency and completeness when syncing across different versions of the SDK.
FIXED: The issue causing the Small Peer Info collector to become stuck. (#12051)
Previously, the Small Peer Info collector would become stuck when attempting to upload logs in certain situations. Since resolving this issue, the Small Peer Info collector functions properly in those conditions.
FIXED: The issue causing fetch operations to appear unresponsive. (#9501, #12067)
With this fix, operations related to fetching attachments function as expected.
FIXED: The issue causing type check failures when setting duration parameters through ALTER SYSTEM SET. (#12070)
When using SystemParameter to configure timeouts before this fix, if using different types to encode the values representing the timeout, for instance, Milliseconds and Seconds, the values failed to downcast, an error log printed, and the new value ignored.
Now, when using mismatched wrapper types, Ditto performs a compatibility check, validates, and, if needed, converts them as appropriate.
FIXED: P2P LAN now advertises on newly added interfaces, such as enabling WiFi at runtime.
FIXED: The deviceName property on Ditto instances now reflects the truncation of the value to 24 bytes when starting sync.
ADDED: Property connectionRequestHandler to ditto.presence, including related types, allowing filtering of incoming connections from other peers.
Added: Properties peerMetadata and peerMetadataJsonString to ditto.presence.
Added: Property identityServiceMetadata to DittoPeer.
Added: Property peerKeyString to DittoPeer as a replacement for the now deprecated peerKey.
DEPRECATED: Property peerKey on DittoPeer, please use the newly added peerKeyString instead.
ADDED: Runtime checks and locks to guard against dangling raw Ditto pointer usage in extremely unlucky race condition scenarios.
FIXED: Possible crash while downloading an attachment due to improper decoding of the attachment token.
Kotlin
ADDED: Support for specifying the mDNS connection ports for Small Peer SDK instances
Allows you to specify a specifc port that can will be used for mDNS announcements and connections on a given device. This can be used for connect with peers across a VLAN while scanning for open ports.
Not setting the config will result in the current default behavior of a random port being assigned.
If running multiple application on the same device, for example a debug and/or test flight application along side a production application, each application needs to have a unique port.
Failure to do this will result in only one application being able to establish connections at a given time.
Setting the config
The config can be set using DQL ALTER SYSTEM and the tcp_server_bind_mdns_server_port config.
This command should be run using the ditto.store.execute(...) command within your application.
tcp_server_bind_mdns_server_port needs to be set ***before ***startSync() is called to be applied.
FIXED: A finalizer issue that was causing crashes when closing sync subscriptions (#13849, #13732, #13738)
Resolves a race condition that could be hit when closing a sync subscription or stopping sync that caused Ditto to crash.
IMPROVED: (Linux/Android) Increased mDNS multicast TTL from 1 to 255 to allow for cross-VLAN announcements (#13590)
To support scenarios with cross-VLAN device discovery we increated the TTL to ensure message have enough time to reach their destination and get acknowledged.
Fixed: Intermittent crash when finalizing a legacy DittoSubscription that contained a sort() clause. (#13394)
Resolves and issue where using sort() in sync subsriptions with the legacy query builder would result in a crash.
Fixed: Included necessary symbols so that Ditto Core crashes can be properly symbolicated with crash reporters. (#13696)
Previously not all symbols were being included in the package bundle and this led to crashes not being able to be symbolicated and ultimately made readable.
Now the Kotlin bundle includes the correct packages
The extra symbols slightly increase the bundle size. If this additional size is an issue for your application, we recommend stripping these symbols when compiling your production build.
However, please note that stripping symbols will make crash reports in production unreadable, which could lead to longer resolution times for live site incidents.
FIXED: An issue causing a race condition with a Live Query callback. (#13180)
This fix addresses the issue causing a Live Query callback to receive a delta update, known as a diff, before the initial document.
FIXED: A compatibility issue between documents created using different CRDT versions. (#13236)
Previously, there was an issue in which two documents on different internal versions could fail to integrate changes created using an older CRDT version.
IMPROVED: Logging for batch write transactions. (#13339, #13339)
Due to a race condition caused by the logging system mistakenly identifying the previous transaction as blocking the current one, the logger would inaccurately indicate the current transaction as being blocked by the previous one from an unknown originator.
FIXED: An issue in the Small Peer datastore causing slow performance or process termination. (#13395)
This issue was due to an aging dependency with a race condition, which caused thread tasks to get stuck and transactions to be blocked for extended periods.
IMPROVED: The Ditto SDKs compilation performance. (#13174)
We’ve optimized compilation by reducing the size of each binary slice of the Ditto SDKs by 19% and fixing incompatibility with PLCrashReporter.
FIXED: Kotlin and Java Android devices now appear as JVMBased in the Device Dashboard instead of Unknown. (#13251)
FIXED: An issue causing a crash with the message gatt must not be null. (#13411)
ADDED: mesh_chooser_avoid_redundant_bluetooth system parameter. (#12984)
This new system parameter allows greater control over the use of Bluetooth Low Energy (LE) connections in Ditto’s peer-to-peer mesh network. When set to false, Ditto creates Bluetooth LE connections alongside other peer-to-peer connections, serving as a fallback if the primary transport, such as Apple Wireless Direct Link (AWDL) or LAN, fails.
However, before you enable the mesh_chooser_avoid_redundant_bluetooth parameter, consider the following technical tradeoffs:
- Benefits:
- Provides additional fallback to maintain connection stability when primary connections cease to function.
- Reduces time to establish new connections since, by having multiple fallback options, Ditto can more quickly re-establish connections.
- Drawbacks:
- Increases network traffic since using multiple transports results in additional network overhead
- Using Bluetooth LE alongside other transports increases resource consumption. In the worst case, enabling this setting can result in slower sync performance and battery drain.
To enable the mesh_chooser_avoid_redundant_bluetooth parameter, use the following DQL query:
ALTER SYSTEM SET mesh_chooser_avoid_redundant_bluetooth = false
CHANGED: The AuthClient by improving the log message for certificate issues returned from the Big Peer Authentication Service. (#13138)
FIXED: A performance-related issue affecting bulk eviction operations for documents with attachments. (#12331)
ADDED: Improved data compression in the handshake protocol when establishing connections to other peers. (#11264)
This allows peers with large permission sets to establish connections on low bandwidth transports like Bluetooth LE more quickly.
FIXED: The HTTP protocol by adding content-length: 0 header compliance in empty POST requests. (#12346)
Before, if you invoked an empty POST request, the cloud load balancer would reject your request with a 411 - Length Required error message.
Now you must explicitly state that the body of your POST request is empty by setting content-length: 0 in the header of your HTTP API call.
FIXED: Handling of malformed subscription queries in the legacy query builder API by adding validation logic early to ensure invalid queries are identified early and logged. (#12463)
There was an issue where invoking subscribe() on an invalid query caused sync to be silently disabled.
Invalid queries in the subscribe() method are now identified, disabled, and logged as errors.
FIXED: Subscription assumption logic so all acknowledgments are processed, regardless of subscription changes. (#12541)
In scenarios where subscriptions rapidly changed back and forth while documents were being concurrently altered, documents would diverge at merge.
We’ve removed subscription logic ignoring acknowledgements based on assumed subscription mismatch and introduced salting functionality to ensure each document is unique for each occurrence of a subscription. This helps differentiate between new and reverted subscription states.
FIXED: Sync inefficiency where changing a subscription resulted in unnecessary syncing of redundant update files. (#12967)
To eliminate redundant data from being synced unnecessarily, regardless of whether the remote peer previously acknowledged them, we’ve added validation checks ensuring that only unacknowledged data is synced.
This fix reduces sync update file sizes, leading to more performant session sync updates, particularly in apps consisting of large, stable subscriptions and infrequent document changes.
CHANGED: Reduced Bluetooth LE warn log noise. (#12674)
Changed: handle reverse DNS lookup queries in Android mdns to avoid mdns failing to connect in specific network configs
CHANGED: Android mdns to handle reverse DNS lookup queries. (#13000)
By adding reverse DNS lookup queries, Android mdns no longer fails to connect in specific network configurations.
FIXED: Wi-fi Aware so it is available on API level 32. (#12631)
FIXED: Missing version metadata. (#12843)
Now version metadata is correctly included in the string returned from Ditto.sdkVersion.
FIXED: Misleading errors related to attachment fetch operations. Now, associated error logs are more accurate and clear. (#12409)
FIXED: A specific issue where incorrect sync metadata rollback caused problems with data consistency at merge. (#12422)
FIXED: The documentation for the DittoStore.fetchAttachment API method. (#11783)
Before this patch release, the documentation incorrectly stated that the return value had to be kept in scope for the fetch to complete. As of this release, the documentation specifies that the return value does not need to be kept in scope.
ADDED: DittoExperimental.constructDittoFailable() API method. (#11801)
Previously, your app crashed if the persistence directory was already in use by another Ditto instance. Now, rather than crashing in such scenarios, Ditto throws a DittoError.LockedWorkingDirectoryError error.
ADDED: DittoExperimental.constructDittoFailable() API method.
Now, you can use this experimental alternative to the original Ditto constructor to prevent crashes when the persistence directory is still in use by another Ditto instance at the time of calling the constructor. Now, rather than crashing in such scenarios, Ditto throws a DittoError.LockedWorkingDirectoryError error. (#11801)
FIXED: Exceptions that occur while creating or fetching attachments now include more details about the underlying cause in their error message for easier debugging. (#12235)
Fixed: An issue that prevented some attachment fetcher events from being delivered to event handlers. (#12331)
FIXED: Closing an attachment fetcher that failed to start downloading no longer results in an exception. (#12490)
FIXED: Handling of document-paths containing non-alphanumeric characters. Previous versions may have removed these characters from document-paths. (#12556)
FIXED: The legacy query language error-throwing mechanism to accept invalid field names properly. (#11888)
After upgrading to version 4.7 of the SDK, you may notice unexpected errors while using the legacy query builder language. These errors are likely due to invalid field names or path expressions previously undetected.
In addition, make sure to start unquoted_string with an underscore (_), a letter, or an alphanumeric character. For example, the following equivalent regular expression: [A-Za-z_][A-Za-z0-9_]*.
In the legacy query builder language, invalid field names and path expressions now throw explicit errors. Previously these invalid names did not throw and would cause undefined behavior.
Note: This new error-throwing behavior applies only to the legacy query builder language; DQL is unaffected by this improvement.
REMOVED: The is_connected_to_big_peer field from the Small Peer Info document. (#11883)
CHANGED: The small peer info JSON metadata by increasing the depth limit to 64 levels. (#11891)
IMPROVED: Transaction contention time for the ATTACHMENT data type, reducing potential timeouts. (#11893)
Before this improvement, concurrent write transactions were long-lived, potentially causing deadlocks and request timeouts. Now you’ll experience more effective handling of multiple attachments being processed at the same time.
FIXED: The issue causing sync crashes due to missing update files. (#11952)
With the latest release, Ditto handles unexpected situations or errors in a way that allows it to recover and continue functioning without crashing or causing disruptions.
ADDED: Parameters for adjusting certain settings during runtime. (#11968)
This rollout includes new system parameters for the MeshChooser feature for configuring a limited set of variables, allowing you to adjust certain settings during runtime.
RESOLVED: The warning message when peers connect is no more. (#11977)
The warning message previously logged when peers connected within the mesh is no longer recorded or printed.
IMPROVED: Sync performance by sending the entire state of the data. (#12029)
Previously, when syncing in mixed mesh environments from Ditto SDK version 4.0.0 of the SDK to version 3.0.0, only partial updates were transmitted. With this release, all updates are transmitted, resulting in greater data consistency and completeness when syncing across different versions of the SDK.
FIXED: The issue causing the Small Peer Info collector to become stuck. (#12051)
Previously, the Small Peer Info collector would become stuck when attempting to upload logs in certain situations. Since resolving this issue, the Small Peer Info collector functions properly in those conditions.
FIXED: The issue causing fetch operations to appear unresponsive. (#9501, #12067)
With this fix, operations related to fetching attachments function as expected.
FIXED: The issue causing type check failures when setting duration parameters through ALTER SYSTEM SET. (#12070)
When using SystemParameter to configure timeouts before this fix, if using different types to encode the values representing the timeout, for instance, Milliseconds and Seconds, the values failed to downcast, an error log printed, and the new value ignored.
Now, when using mismatched wrapper types, Ditto performs a compatibility check, validates, and, if needed, converts them as appropriate.