EJSON support is available when using the MongoDB Connector to synchronize data between Ditto and MongoDB databases.
Read more about EJSON in the MongoDB Connector EJSON Mode section.
What is EJSON?
Extended JSON (EJSON) is MongoDB’s canonical representation of BSON (Binary JSON) data in JSON format. When EJSON mode is enabled in the MongoDB Connector, all BSON data types from MongoDB are converted to their canonical JSON representation before being stored in Ditto, and vice versa. This ensures complete fidelity of MongoDB’s rich data types when synchronizing with Ditto, but requires special handling in your application code and queries.EJSON Data Type Mappings
When EJSON mode is enabled, MongoDB BSON types are converted to their canonical representations:Common Type Conversions
MongoDB BSON | EJSON Canonical Format | Example |
---|---|---|
Date | { "$date": { "$numberLong": "<millis>" } } | { "createdAt": { "$date": { "$numberLong": "1704067200000" } } } |
ObjectId | { "$oid": "<hex string>" } | { "_id": { "$oid": "507f1f77bcf86cd799439011" } } |
NumberLong | { "$numberLong": "<string>" } | { "count": { "$numberLong": "9223372036854775807" } } |
NumberInt | { "$numberInt": "<string>" } | { "age": { "$numberInt": "42" } } |
Double | { "$numberDouble": "<string>" } | { "price": { "$numberDouble": "19.99" } } |
Decimal128 | { "$numberDecimal": "<string>" } | { "total": { "$numberDecimal": "123.456789" } } |
Binary | { "$binary": { "base64": "<base64>", "subType": "<hex>" } } | { "data": { "$binary": { "base64": "YmluYXJ5IGRhdGE=", "subType": "00" } } } |
Timestamp | { "$timestamp": { "t": <uint32>, "i": <uint32> } } | { "ts": { "$timestamp": { "t": 1704067200, "i": 1 } } } |
For a complete list of EJSON type representations, refer to the MongoDB Extended JSON documentation.
Querying EJSON Data in DQL
When EJSON mode is enabled, you need to account for the nested structure of EJSON types in your DQL queries.Working with Numbers
Numbers in EJSON are stored as strings, which affects comparison operations:Accessing Date Fields
Instead of directly accessing a date field, you need to traverse the EJSON structure:09/09/2001, 1:46:40 am
and 05/18/2033, 3:33:20 am
, as they fit in the range of 1000000000 to 2000000000.
If you are using dates outside of this range, you will need to use the CAST
function to convert the date to a number.
Converting EJSON numeric strings (using
CAST
or other conversion functions) currently makes queries ineligible for index optimization in DQL. This can significantly impact query performance on large datasets.Updating EJSON Data
When updating EJSON data, you need to ensure that the data is updated in the correct format. Typically this means ensuring that the nested fields are updated correctly using strings to represent numeric values.Changing Datatypes
We recommend avoiding changing the datatype of an EJSON field, aside from moving from a null type to a non-null type. For example, if you want to change the order date from a null type to a date type, you can do the following:Best Practices for EJSON Queries
1. Minimize Type Conversions
Avoid converting EJSON numeric types unless absolutely necessary, as this prevents index usage:2. Use String Comparisons When Possible
For some use cases, string comparison of EJSON numeric values works correctly:3. Create Helper Fields
If you frequently query EJSON fields, consider creating flattened helper fields that are not stored as BSON:Converting EJSON in Your Application
When working with EJSON data in your application (e.g. in your UIs or business logic), you’ll need to convert between EJSON and native types using platform-specific BSON libraries. You should pass the EJSON data contained in your DittoQueryResultItem to the BSON library to convert it to the native type.Platform-Specific BSON Libraries
BSON libraries are available for most platforms, you can find a suggested (though not exhaustive) list of libraries below:Platform | Library |
---|---|
Swift | swift-bson |
Kotlin | bson |
JavaScript/Node.js | bson |
Flutter | bson |
Java | MongoDB Java Driver |
C# | MongoDB.Bson |
Rust | bson |
Performance Considerations
Index Eligibility
Queries that use type conversion functions (like
CAST
) on EJSON fields cannot use indexes. This can severely impact query performance for larger datasets.- Avoid conversions in WHERE clauses: Design queries that work with EJSON string representations
- Pre-compute values: Store additional fields with converted values for frequently queried data
- Use compound strategies: Combine indexed fields with post-filtering for complex queries
Query Optimization Examples
Troubleshooting Common Issues
Query Performance Degradation
Query Performance Degradation
Symptom: Queries that were fast are now slow after enabling EJSON mode.Solution: Review queries for type conversions and create indexed helper fields for frequently queried EJSON data.
Numeric Comparison Failures
Numeric Comparison Failures
Symptom: Numeric comparisons return unexpected results.Solution: Remember that EJSON numbers are strings. Use
CAST
for accurate comparisons (accepting the performance trade-off) or implement application-level filtering.Date Queries Not Working
Date Queries Not Working
Symptom: Date range queries return no results.Solution: Access the nested
$date.$numberLong
field and compare as milliseconds since epoch: