Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 44 additions & 15 deletions pages/clustering.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,30 +3,59 @@ title: Clustering
description: Learn all about replication and high availability features in Memgraph.
---

import {CommunityLinks} from '/components/social-card/CommunityLinks'
import { Callout } from 'nextra/components'
import { CommunityLinks } from '/components/social-card/CommunityLinks'

# Clustering

To create a cluster, you can [replicate data](/clustering/replication) across
several instances. One instance is the MAIN instance and others are either SYNC
or ASYNC replicas. With Memgraph Community, to achieve high availability, you
need to manage automatic failover. On the other hand, Memgraph Enterprise has
[high availability](/clustering/high-availability) features included in the
To ensure redundancy and increase uptime, you can set up a cluster of Memgraph instances which can
guarantee you 24/7 uptime and availability of your graph dependent services.

With Memgraph Community, you gain [replication](/clustering/replication) capabilities out of the box.
You can set up a MAIN instance (writes and reads) with as many REPLICA instances (reads) as you want.
However, to achieve high availability, you need to manage automatic failover.

On the other hand, Memgraph Enterprise has [high availability](/clustering/high-availability) features included in the
offering to ease the management of Memgraph clusters. In such case, the cluster
consists of MAIN instance, REPLICA instances and COORDINATOR instances which,
backed up by Raft protocol, manage the cluster state.
consists of:
- MAIN instance
- REPLICA instances
- COORDINATOR instances (backed up by Raft protocol, manage the cluster state and perform leader election)

<Callout type="info">

**We strongly suggest that user reads the guide on [how replication works](/clustering/concepts/how-replication-works)
in Memgraph on a logical level, before moving to the part of setting the cluster up.**
Choosing the appropriate number of Memgraph instances, as well as the replication mode on each
of them is crucial to understand, as that impacts performance and availability of the cluster based on your needs.

</Callout>

<Callout>

Replication and high availability currently **work only in the [in-memory
transactional storage mode](/fundamentals/storage-memory-usage#in-memory-transactional-storage-mode-default)**.

</Callout>

## [How replication works](/clustering/concepts/how-replication-works)

Learn about the underlying implementation and theoretical concepts behind Memgraph replication, including CAP theorem, replication modes, and synchronization mechanisms.

Replication and high availability currently **work only in the in-memory
transactional [storage mode](/fundamentals/storage-memory-usage)**.
## [Replication guide (Community)](/clustering/replication)

Learn how to set up a replication cluster with Memgraph.
**Replication is included in Memgraph Community**, making it accessible to all users who want to create data replicas across multiple instances.
Memgraph Community however does not ensure high availability itself, as **automatic failover is not included**. Community users are encouraged to
perform the necessary steps themselves for keeping the replication cluster up and running.

## [High availability](/clustering/high-availability)
## [High availability guide (Enterprise)](/clustering/high-availability)

Learn how to utilize high availability features and all the important under the
hood information.
Learn how to setup and manage a high availability cluster with Memgraph.
This guide is for users of **Memgraph Enterprise** who want to achieve clustering and 24/7 uptime.

## [Replication](/clustering/replication)
## [FAQ](/clustering/faq)

Learn how replication is achieved in Memgraph and how to set it up.
Frequently asked questions about clustering, replication, and high availability in Memgraph.

<CommunityLinks/>
5 changes: 3 additions & 2 deletions pages/clustering/_meta.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
export default {
"concepts": "Concepts",
"high-availability": "High availability",
"replication": "Replication"
"replication": "Replication",
"faq": "FAQ"
}

4 changes: 4 additions & 0 deletions pages/clustering/concepts/_meta.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
export default {
"how-replication-works": "How replication works",
"how-high-availability-works": "How high availability works",
}
779 changes: 779 additions & 0 deletions pages/clustering/concepts/how-high-availability-works.mdx

Large diffs are not rendered by default.

352 changes: 352 additions & 0 deletions pages/clustering/concepts/how-replication-works.mdx

Large diffs are not rendered by default.

49 changes: 49 additions & 0 deletions pages/clustering/faq.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
title: Frequently asked questions about clustering
description: Explore the documentation page for Memgraph and access the FAQ section to find solutions to common queries and concerns. Discover essential information and insights now.
---
import { CommunityLinks } from '/components/social-card/CommunityLinks'


# Frequently asked questions

## High availability (general)

#### Does Memgraph support chaining REPLICA instances?
Memgraph at the moment doesn't support chaining REPLICA instances, that is, a REPLICA
instance cannot be replicated on another REPLICA instance.

#### Can a REPLICA listen to multiple MAIN instances?
Memgraph enforces the behaviour that REPLICA can only listen to exactly one MAIN instance.
When starting any Memgraph instance, it is assigned a unique UUID of the instance. This is communicated
when a replica is registered, to ensure REPLICA does not receive replication data from another MAIN instance.
A REPLICA stores the UUID of the MAIN instance it listens to.
The instance UUID of each Memgraph is persisted on disk across restarts, so this behaviour is enforced throughout the
cluster lifecycle.

#### Can a REPLICA create snapshots by itself?
No. REPLICA can only receive snapshots during the recovery phase.

#### Can a REPLICA create WALs by itself?
Actually, this is being done in the system. When a MAIN is committing, it is sending the Delta objects to the REPLICA.
Here the replica is doing two things:
- it is applying the Delta objects from MAIN to catch up
- it is writing the Delta objects to its own WAL files

Why is this important?
Picture the following scenario. REPLICA is up to date with the MAIN. MAIN is constantly sending delta objects to the REPLICA.
After a while, the MAIN goes down and REPLICA is promoted to be the "new MAIN". The new MAIN would not have any durability
files, if it didn't write WALs during its period of being a REPLICA. If the old MAIN rises, the new MAIN would perhaps have
insufficient information to be sent to the new REPLICA. That's why REPLICA always needs to write down in WALs what it's
being received.


### High availability with Docker

### High availability with Docker Compose

### High availability with K8s



<CommunityLinks/>
Loading