Time-Based Replication

One common design pattern is to replicate only the data written to the datastore within the past 24 hours. At the end of that 24 hours, documents older than 24 hours are evicted and a new query is created.

Let's look through a typical example for syncing data: four flight attendants walk the aisle of an aircraft and record passenger meal orders on their tablets.

The database has a flights collection, with each flight represented as a document. Each document has a createdAt timestamp:

We want to only store data on the device that was created in the past 24 hours. Anything older than 24 hours should be removed locally from the device. However, we don't want to remove this data from the entire mesh network, because we may want to do analytics on it later using the Big Peer. So, we do not use remove() because that deletes data from the entire network, including the Cloud. We use evict() to remove data from the local device because the data is not relevant anymore.

One way to implement eviction would be to create a global interval. For example, every 24 hours check to see if there is any data on the local machine that is irrelevant. If so, evict it.

Whenever you have an eviction query that changes over time it is important that you also properly stop your subscription and update it with a new query each time your eviction query changes. This is because if you try to evict data that you are also subscribed to, then after you evict the data you will immediately sync it back to the device since the subscription is still syncing that data.

Using the example of evicting data every 24 hours we will see how to stop the subscription, evict the data, and then start the new subscription.

Notice that the query for eviction is exactly the opposite as the query for subscription. These queries should not overlap, so you must stop your subscription before you evict. Your subscription variable should remain in scope throughout the runtime time of the app, so that it will continue to sync data and not be garbage collected.