Syncing Qdrant with another Qdrant

[cemremengu]

I want to run 2 standalone qdrant instance that can sync each other (i.e. cross cluster replication). For example when I delete entity from qdrant instance1 that instance will delete from instance2 (update, insert, delete). Is there any way that I can do that ?

[timvisee]

Yes! Please see our documentation on this here: https://qdrant.tech/documentation/guides/distributed_deployment/

More specifically, see this section describing: 1. enabling cluster mode. 2. starting the first node. 3. bootstrapping other nodes.

Once you've set up a cluster you can send operations to any node. They'll be synced across the cluster.

Note that if you want to prevent downtime if one of the nodes is down, you should ideally have at least 3 nodes. That is because there should be a majority if nodes, which would not be possible with 1 alive + 1 dead node. In that case, also make sure you have at least a replication factor of 2.

[timvisee]

Thanks for elaborating. Since you were talking about instances, I was thinking about nodes in a cluster.

At this point we don't have any knobs for setting up multi-region deployments, pairs of clusters or availability zones.

Alternatively you could set up a separate second cluster and continuously sync data to it by using snapshots. It will not be real time though and will require significant manual effort. It is also possible to form a cluster with nodes across different zones though that'll likely introduce noticeable latency. Depending on your needs either of those may be sufficient (for now).

I would like to note that it is on our roadmap.

It's likely best to just rely on a single cluster now, and deploy multiple zones later once we support it.

[cemremengu]

Great! Thanks again for taking your time to reply and your work on Qdrant.

[Prakash2403]

It is also possible to form a cluster with nodes across different zones though that'll likely introduce noticeable latency.

Hi @timvisee, how do I redirect a request to a specific node in a cluster? If I am using docker, can I do IP-Address-of-Node:6333 (Assuming I have done relevant port mappings)?

My problem is that I have users worldwide but my server is in the USA. Due to this, there's a noticeable latency for users who are located far away from my servers (say India, Australia, etc). Hence I want to setup one server in each region, and redirect the incoming requests to the nearest server.

[timvisee]

If I am using docker, can I do IP-Address-of-Node:6333 (Assuming I have done relevant port mappings)?

Yes. You're responsible yourself for spreading requests to nodes. You can simply spread to all nodes, as all necessary internal routings are made internally.

Note that we do all this for you in Qdrant Cloud.

Hence I want to setup one server in each region, and redirect the incoming requests to the nearest server.

This currently is not possible. Unless you set up isolated clusters. That may very well be good enough for your current use case.

[Prakash2403]

Thanks for the prompt response! I might go ahead with isolated clusters. I am happy to switch to Qdrant cloud once cross-cluster replication is possible. Do keep us updated!