Skip to main content

​
Introduction

The Ditto Operator allows you to deploy Big Peer (aka Ditto Server), Ditto’s cloud sync engine and data integration platform, to your own self-hosted Kubernetes environment. Ditto Operator simplifies the configuration of the various components that form Big Peer, allowing you to focus on developing applications that get you connected to the cloud.

​
Prerequisites

A few prerequisites need to be running before you install the Operator and deploy Big Peer.

​
Kubernetes Environment

The following steps assume you have a Kubernetes cluster, on at least version 1.31. If you’re looking to try the Operator locally, we recommend using kind, a tool for running Kubernetes locally using Docker.
Here’s an example configuration you can use, which will expose the Big Peer Ingress on your local device, allowing you to follow the steps later in this guide:
cat <<'EOF' | kind create cluster --config -
---
apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
kubeadmConfigPatches:
  - |
    kind: ClusterConfiguration
    metadata:
      name: config
    etcd:
      local:
        extraArgs:
          unsafe-no-fsync: "true"
    networking:
      serviceSubnet: 10.0.0.0/16
nodes:
  - role: control-plane
    kubeadmConfigPatches:
      - |
        kind: InitConfiguration
        nodeRegistration:
          kubeletExtraArgs:
            node-labels: "ingress-ready=true"
    extraPortMappings:
      - containerPort: 80
        hostPort: 80
        protocol: TCP
      - containerPort: 443
        hostPort: 443
        protocol: TCP
      - containerPort: 4040
        hostPort: 4040
        protocol: TCP
EOF

​
Tools

You’ll also need to have kubectl and helm installed, with the appropriate KUBECONFIG current-context set.

​
Installing Certificate Manager

The Ditto Operator uses cert-manager to issue various auth-related certificates. You can install cert-manager into the target cluster using: 
helm install cert-manager cert-manager    \
    --repo https://charts.jetstack.io     \
    --namespace cert-manager              \
    --create-namespace                    \
    --set 'crds.enabled=true'             \
    --set 'startupapicheck.enabled=false' \
    --set 'prometheus.enabled=false'
Verify the installation was successful by checking the status of the cert-manager pods: 
kubectl get pods -n cert-manager

NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-58554cc75d-qcwxc              1/1     Running   0          10s
cert-manager-cainjector-6fbf54b8fb-2wvwq   1/1     Running   0          10s
cert-manager-webhook-6bccd45c78-w8rxm      1/1     Running   0          10s

​
Installing an Ingress

The Ditto Operator manages ingresses to route traffic into the various Big Peer services. We recommend using an ingress-nginx, as specific nginx specific annotations are used for optimal hashing. This guide will assume that nginx is being used.
The Ditto Operator can support other ingress controllers, but these may require different annotations. Please get in touch with Ditto support for guidance.
To install nginx:
  • Kind cluster
  • Standard Nginx
If you opted to use Kind config provided to run the cluster locally, run the following:
kubectl apply -f https://kind.sigs.k8s.io/examples/ingress/deploy-ingress-nginx.yaml
Verify the installation was successful by checking the status of the nginx pods: 
kubectl get pods -n nginx

NAME                                       READY   STATUS      RESTARTS     AGE
ingress-nginx-admission-create-4pbsb       0/1     Completed   0            10s
ingress-nginx-admission-patch-hgnfp        0/1     Completed   0            10s
ingress-nginx-controller-76666fb69-76fz5   1/1     Running     0            10s

​
Installing Strimzi

The Ditto Operator uses strimzi to create a Kafka cluster and topic for the transaction log. You must install strimzi, a Kafka Operator, into the target cluster using: 
helm install strimzi strimzi-kafka-operator \
    --repo https://strimzi.io/charts        \
    --namespace kafka                       \
    --create-namespace                      \
    --set watchAnyNamespace=true
Verify the installation was successful by checking the status of the strimzi pods: 
kubectl get pods -n kafka

NAME                                        READY   STATUS    RESTARTS   AGE
strimzi-cluster-operator-588bc8c768-dpsdw   1/1     Running   0          10s
For convenience, this instructs Strimzi to watch all namespaces for Kafka resources.If you wish to constrain this behaviour (for example you deploy Kafka for your other applications in another namespace) you can use --set watchNamespaces to specify the namespace you’re using for Ditto. Eg: --set watchNamespaces=ditto,another-ditto-namespace

​
Installing the Ditto Operator

With all prerequisites taken care of, the latest stable version of the Ditto Operator can be installed using the helm chart and image located at oci://quay.io/ditto-external/ditto-operator: 
helm install ditto-operator                   \
  oci://quay.io/ditto-external/ditto-operator \
  --namespace ditto                           \
  --create-namespace
This installs the Operator into a newly created ditto namespace. All further steps will refer to this namespace.
Various configuration options are available in the chart’s values.yaml. You can view them with helm show values oci://quay.io/ditto-external/ditto-operator.These values can be overriden using helm’s --set syntax. Further documentation for these values, and what they do, is coming soon.
You can now verify the Ditto Operator is installed and running with: 
kubectl logs -f deployment/ditto-operator --namespace ditto
The various Ditto-related CRDs (Custom Resource Definitions) that were installed by the chart can be viewed using: 
kubectl api-resources | grep ditto.live
Which will return output similar to: 
NAME                       APIVERSION            NAMESPACED   KIND
bigpeers                   ditto.live/v1alpha1   true         BigPeer
bigpeerapps                ditto.live/v1alpha1   true         BigPeerApp
bigpeerstores              ditto.live/v1alpha1   true         BigPeerStore
bigpeersubscriptions       ditto.live/v1alpha1   true         BigPeerSubscription

​
Deploying a Big Peer

Here’s a minimal example to deploy Big Peer: 
cat <<'EOF' | kubectl apply -f -
---
# Create the Big Peer deployment.
apiVersion: ditto.live/v1alpha1
kind: BigPeer
metadata:
  name: example
  namespace: ditto
spec:
  version: 1.43.0
  network:
    ingress:
      host: localhost
  auth:
    providers:
      __playgroundProvider:
        anonymous:
          permission:
            read:
              everything: true
              queriesByCollection: {}
            write:
              everything: true
              queriesByCollection: {}
          sessionLength: 630000
          sharedToken: abc123
EOF
This will deploy a Big Peer which is:
  • Running on version 1.43.0
  • Defines an ingress on localhost
  • Enables Ditto SDK device auth via the provider named __playgroundProvider:
    • Devices have read and write access to all collections
    • A Small Peer can authenticate using an “Online Playground” identity with a shared token of abc123
    • This is a global authentication configuration which can be used to authenticate to any app on the Big Peer
OnlinePlayground is ideal for development and testing, as it allows you to quickly define a shared token that can be shared with devices for auth.OnlineWithAuthentication is more appropriate for production use.For more on connecting with the SDK, see: SDK -> Authentication & Authorization
Small Peers using an OnlinePlayground or OnlineWithAuthentication identity try to connect to Ditto’s cloud service by default. Extra configuration is required to point it to your Big Peer.
  • In the parameters to OnlinePlayground, provide a custom authentication URL pointing to the ingress, e.g. “https://my-host”
  • In the parameters to OnlinePlayground, set “enable cloud sync” to false
  • Set a TransportConfig with a websocket URL pointing to your ingress. In the Rust SDK, for example: transport_config.connect.websocket_urls.insert("wss://my-host".to_owned())
If the BigPeer was applied successfully, the various components of the Big Peer can be observed by running: 
kubectl get pods -l ditto.live/big-peer=example --namespace ditto
Which will match all running pods with the above BigPeer’s metadata.name, which is "example" in this case: 
NAMESPACE  NAME                                                   READY  STATUS   RESTARTS  AGE
ditto      ditto-example-api-0                                    1/1    Running  0         1m20s
ditto      ditto-example-store-14d2fe44-0                         1/1    Running  0         1m19s
ditto      ditto-example-store-6a5d568b-0                         1/1    Running  0         1m19s
ditto      ditto-example-subscription-a475c814-7b459bfddb-brpr4   1/1    Running  0         1m20s*

​
Configuring Big Peer

Once Big Peer is deployed, you’ll need to configure it for first time usage.

​
The Operator API

The Ditto Operator provides a RESTful API for creating Apps and managing their configuration. At this time, the Operator API does not have an authentication mechanism, and should not be exposed to the publicly. It is intended for use within the K8s cluster only. To access the API, we recommend port-forwarding to ensure it remains secure and only accessible from your local machine. For example: 
kubectl -n ditto port-forward deployments/ditto-operator 8080:8080
The following steps will assume you’ve portforwarded in this way.

​
Creating an App

In Ditto, an App is a logical container for your data. An App needs to be registered with Big Peer before you can start creating documents. Each app has a unique ID that’s used to identify it across different Big Peers and devices. You’ll use this ID for all interactions with your App, including:
  • Connecting to the App from the Ditto SDK
  • Making HTTP API calls to the App
  • Setting up replication between Big Peers
  • Using the Operator API
  • Using a BigPeerApp Resource
Using the Operator API is the simplest way to create a new App.Assuming you’ve port-forwarded to the Operator as described in the previous section, create an App using a POST request as follows:
curl -H 'Content-Type: application/json' \
     -X POST http://localhost:8080/namespace/ditto/bigPeer/example/app \
     -d '{"name":"example-app"}'
This will create an App called example-app, running on the example Big Peer, in the ditto namespace.If successfil, you should receive a response like this:
{
  "name": "example-app",
  "appId": "2164bef3-37c0-489c-9ac6-c94b034525d7"
}
Take note of the appId from the response, you’ll need this later to use the HTTP API.

​
App-scoped Authentication Providers

The auth providers listed in the BigPeer resource are available to all apps. You can also configure providers for a specific BigPeerApp by including an auth configuration. The following example defines an external webhook with the provider name my-provider. 
cat <<'EOF' | kubectl apply -f -
---
apiVersion: ditto.live/v1alpha1
kind: BigPeerApp
metadata:
  name: example-app
  namespace: ditto
  labels:
    ditto.live/big-peer: example
spec:
  appId: 2164bef3-37c0-489c-9ac6-c94b034525d7
  auth:
    my-provider:
      tokenWebhook:
        url: https://webhook.example.com/validate_token
EOF
An anonymous provider may also be specified at the app level. Make sure to give it the name __playgroundProvider so that it can be used by Ditto SDKs using an OnlinePlayground identity.

​
Using the Big Peer HTTP API

Using the HTTP API is the quickest way to verify your installation is running.
If you’ve deployed locally using the kind example config provided, these queries should work out of the box.Otherwise, you may need to update the localhost according to the ingress you’ve configured on your K8s deployment.

​
Generate an API Key

To use the Big Peer HTTP API, you’ll need to generate an API key. This can only be done using the Operator API. For this guide, we’ll create an API key that can has read and write access to all collections, with a very long expiry. For production scenarios, you may want to customise the scope of the API Keys access, and its expiry. First, ensure you have port-forwarded to the Operator as described in the previous section. Then, create an API key for your app using a POST request: 
curl -H 'Content-Type: application/json' \
     -X POST http://localhost:8080/namespace/ditto/bigPeer/example/app/example-app/apiKey \
     -d '{
       "name": "my-key",
       "expiresAt": "2100-04-13T12:00:00Z",
       "permissions": {
         "remoteQuery": true,
         "read": {
           "everything": true,
           "queriesByCollection": {}
         },
         "write": {
           "everything": true,
           "queriesByCollection": {}
         }
       }
     }'
You’ll receive the raw API key as the response: 
YlYr32ZAXotlf9zS1wh8JanJ2kRI2L38KnFHEh6nvaGiqKEqmAf8EsnZ9e0r
The raw API key is only shown once and is not stored anywhere. Make sure to save it securely, as you’ll need it for all HTTP API requests.
You can verify the API key was created by listing all keys for your app: 
curl http://localhost:8080/namespace/ditto/bigPeer/example/app/example-app/apiKey

{
  "meta": {
    "page": 1,
    "size": 1,
    "totalPages": 1
  },
  "data": [
    {
      "name": "my-key",
      "expiresAt": "2100-04-13T12:00:00Z",
      "permissions": {
        "remoteQuery": true,
        "read": {
          "everything": true,
          "queriesByCollection": {}
        },
        "write": {
          "everything": true,
          "queriesByCollection": {}
        }
      },
      "description": null
    }
  ]
}
You can also verify this using kubectl as the Operator will make a BigPeerApiKey object to manage this key: 
kubectl get bigpeerapikey my-key -n ditto -o yaml

kubectl get bigpeerapikey my-key -n ditto -o yaml
apiVersion: ditto.live/v1alpha1
kind: BigPeerApiKey
metadata:
  creationTimestamp: "2025-04-25T14:05:54Z"
  generation: 1
  labels:
    ditto.live/app: example-app
  name: my-key
  namespace: ditto
  resourceVersion: "10664"
  uid: 128351cf-4c44-4056-a96d-d0aa6a0b17b4
spec:
  expiresAt: "2100-04-13T12:00:00Z"
  hash: $scrypt$ln=15,r=8,p=1$DNpYWZdaH+yAgJbnIYu00Q$jshJ3PmHN6qjNwTd+3nlzrtKAcASBAGHazDFUM0agOE
  permissions:
    read:
      everything: true
      queriesByCollection: {}
    remoteQuery: true
    write:
      everything: true
      queriesByCollection: {}
  prefix: 9sNPfXzuBi
  version: 0

​
Insert an item into the cars collection

These examples assume you’ve deployed locally using the kind example config provided. If not, you may need to update localhost to the hostname you’ve configured as for your ingress.Remember to replace YOUR_API_KEY with the API key generated earlier.
curl -X POST http://localhost/2164bef3-37c0-489c-9ac6-c94b034525d7/api/v4/store/execute \
    --header 'Authorization: bearer YOUR_API_KEY' \
    --header 'Content-Type: application/json' \
    --data-raw '{"statement":"INSERT INTO cars DOCUMENTS (:doc1)","args":{"doc1":{"_id":{"id":"777","locationId":"123456"},"color":"blue","timestamp":"1732192529"}}}'
This:
  • Uses the API key generated earlier
  • Inserts a new document into the cars collection, which will implicitly be created

​
List all items in the cars collection

We can then verify that the document was created with: 
curl -X POST http://localhost/2164bef3-37c0-489c-9ac6-c94b034525d7/api/v4/store/execute \
    --header 'Authorization: bearer YOUR_API_KEY' \
    --header 'Content-Type: application/json' \
    --data-raw '{"statement":"SELECT * FROM cars"}'

​
Connecting with the Ditto SDK

The Ditto Quickstart apps are the quickest way to get started using the Ditto SDK with an Operator managed Big Peer. Quickstart apps are available for various platforms including Swift, Java, React Native, Flutter, and more. You can find them in our GitHub repository. To get connected with minimal configuration for development and testing, we recommend using OnlinePlayground authentication. The Big Peer we deployed earlier has an OnlinePlayground auth provider configured with a shared token of abc123.
1

Clone the Quickstart repository

git clone https://github.com/getditto/quickstart.git
2

Configure .env file

cp .env.example .env
Edit the .env file to set the following variables:
VariableDescription
DITTO_APP_IDThe App ID of your BigPeerApp you wish to connect to
DITTO_AUTH_URLThe auth URL of your Big Peer, as specified in the BigPeer network.ingress
DITTO_WEBSOCKET_URLThe websocket URL of your Big Peer, as specified in the BigPeer network.ingress
DITTO_PLAYGROUND_TOKENThe playground token configured in your Big Peer’s auth provider
If you’ve deployed using our suggested kind example config, and deployed a Big Peer and App per this guide, these values will be:
DITTO_APP_ID=2164bef3-37c0-489c-9ac6-c94b034525d7
DITTO_AUTH_URL=http://localhost
DITTO_WEBSOCKET_URL=ws://localhost
DITTO_PLAYGROUND_TOKEN=abc123
3

Run the app

Follow the steps README in the Quickstart repository for the SDK you’re using to run the app.For example, for the JavaScript TUI Quickstart app, you’ll need to run:
cd javascript-tui
npm install
npm start 2>/dev/null
4

Verify the connection

The Quickstart app should launch and display “Sync Active”. You can then create some tasks.Quickstart DemoTo verify the tasks are being synced to your Big Peer, you can start another instance of the Quickstart app and confirm the tasks you’ve created are visible.You can also verify this by querying the tasks collection using the HTTP API:
curl -X POST http://localhost/2164bef3-37c0-489c-9ac6-c94b034525d7/api/v4/store/execute \
  --header "Authorization: bearer YOUR_API_KEY" \
  --header "Content-Type: application/json" \
  --data-raw '{"statement":"SELECT * FROM tasks"}'

​
Uninstalling using Helm

Before continuing, ensure any Big Peer resources that were created have been deleted. You can check for existing resources using the following command: 
kubectl get bigpeers,bigpeerapps --namespace ditto
It is recommended you delete these resources before uninstalling the Ditto Operator. Otherwise, if you plan to reinstall the Ditto Operator again later the new installation will immediately discover the previous resources and proceed to create them anew. You can now delete the helm release that was created in Installing the Ditto Operator: 
helm delete ditto-operator
⌘I