This article provides conceptual information for the ATTACHMENT data type. For instructions, see CRUD Operations > CREATE, READ, UPDATE, and DELETE.

The ATTACHMENT data type lets you link extensive amounts of binary data — such as images, videos, deeply embedded documents of 50 megapixels or more — as well as any other type of file that changes rarely to a document. This linking allows for on-demand sync, both online or offline, without any conflicts.

You can create and transmit attachments across peers, even with intermittent connectivity; however, unlike documents, which are always accessible and automatically sync when relevant per sync subscriptions, peers do not automatically fetch the attachment data associated with a subscription — so attachments must first be explicitly fetched before you can access them.

When incorporating an attachment into a document, rather than inserting the entire resource-heavy ATTACHMENT object into the document, internally Ditto inserts a pointer object, known as the attachment token. (See ATTACHMENT Object and Attachment Token)

The following structure and corresponding table provide an overview of the schema the attachment token uses to model the data it contains:

Here are the attachment token properties you can query to increase the efficiency of fetching operations in your app:

A JSON blob stores binary data representing any file type. For example, it can store image files like JPEG, PNG, and TIFF; video files such as MP4; audio files, including MP3 and WAV; document files like PDF; and more.

The metadata is stored in the document, while the blob is stored in a dedicated location independent of documents. The following table provides an overview of blob storage locations:

Ditto organizes and stores blob data using the content-addressable network (CAN) paradigm. CAN is a distributed network architecture widely implemented in decentralized systems like Ditto's where scalability, efficiency, and fault tolerance are essential requirements. (See the official Content-Addressable Network Wikipedia article.)

In more straightforward terms, Ditto organizes and accesses attachment data by the cryptographic hash table of its contents — for instance, attachment ID — rather than its physical location. A hash table is a data structure that stores key-value pairs.

By adhering to the CAN framework for attachment management, Ditto stores and retrieves data efficiently. For instance:

Blob storage is managed internally using a reference-counting process known as garbage collection.

The garbage collection process running periodically in the background of Small Peers frees up space by removing blobs that no longer have any references — once an attachment has no remaining references, its blob is considered safe to remove, and the garbage collection process clears it from the device.

The two components comprising an ATTACHMENT object — metadata stored internally within a document and blob data storing the actual binary data of your attachment — sync in different ways:

Attachment blob sync being asynchronous provides three key advantages:

Blob sync remains performant — Ditto uses data structures like Bloom filters to scale efficiently and remain light on system resources.

Bloom filters are randomized data structures designed to represent sets in a compressed form, making them highly effective in peer-to-peer environments where concerns such as memory usage and accuracy are prevalent. (See the official Bloom filter Wikipedia article.)

When an attachment token stored on a Small Peer end-user device is unused, meaning there are no remaining references to it, a background process that runs periodically, every 10 minutes, automatically deletes it to free up space.

This reference-counting process, known as garbage collection, only runs on Small Peer devices (not on the Big Peer). It involves scanning for objects, including blobs, that are no longer needed or relevant and are therefore safe to delete from its device.

The Big Peer stores its data in Ditto's third-party cloud object storage service provider, Amazon WebServices (AWS) Simple Storage Service (S3). For more information, see the official Amazon Simple Storage Service Documentation.

Since the blob is stored outside the local Ditto store, fetch it only when needed. Referred to as lazy loading, this "on-demand" retrieval pattern reduces resource usage on end-user devices.

There are two separate steps in the process of creating an attachment:

To access your attachment data on demand or perform other operations, such as updating or deleting the attachment, first fetch its associated attachment token. (See Fetching Attachment Tokens)