Help Center/ MapReduce Service/ Best Practices/ MRS Cluster Management/ Backing Up and Restoring MRS Cluster Data
Updated on 2024-11-30 GMT+08:00
View PDF

Backing Up and Restoring MRS Cluster Data

MRS clusters can back up and restore user and system data. Each component has its own backup function.

MRS 3.x can back up data to LocalDir, LocalHDFS, RemoteHDFS, NFS/CIFS, SFTP servers, and OBS. For a component that supports multiple services, you can back up and restore multiple instances of a service. The backup and restoration process is the same as for a single service instance.

Backup and restoration are used in the following scenarios:

  • Performing routine backup to ensure the data security of the system and components;
  • Recovery of the failed system using the backup data;
  • Primary cluster restoration by creating a mirrored cluster that is identical to the primary one using the backup data.
Table 1 Metadata (MRS 2.x and earlier versions)

Backup Type

Backup Content

OMS

Database data (excluding alarm data) and configuration data in the cluster management system by default

LdapServer

User information, including the username, password, key, password policy, and group information

DBService

Metadata of the components (Hive) managed by DBService

NameNode

HDFS metadata
Table 2 Manager configuration data (MRS 3.x)

Backup Type

Backup Content

Backup Directory Type

OMS

Database data (excluding alarm data) and configuration data in the cluster management system by default
  • LocalDir
  • LocalHDFS
  • RemoteHDFS
  • NFS
  • CIFS
  • SFTP
  • OBS
Table 3 Component metadata or other data (MRS 3.x)

Backup Type

Backup Content

Backup Directory Type

DBService

Metadata of the components (including Loader, Hive, Spark, Oozie, CDL, and Hue) managed by DBService. For a cluster with multiple services installed, back up the metadata of multiple Hive and Spark service instances.
  • LocalDir
  • LocalHDFS
  • RemoteHDFS
  • NFS
  • CIFS
  • SFTP
  • OBS

Flink

(Applicable to MRS 3.2.0 and later versions)

Flink metadata.
  • LocalDir
  • LocalHDFS
  • RemoteHDFS

Kafka

Kafka metadata.
  • LocalDir
  • LocalHDFS
  • RemoteHDFS
  • NFS
  • CIFS
  • OBS

NameNode

HDFS metadata. After multiple NameServices are added, backup and restoration are supported for all of them and the operations are consistent with those of the default hacluster instance.
  • LocalDir
  • RemoteHDFS
  • NFS
  • CIFS
  • SFTP
  • OBS

Yarn

Information about the Yarn service resource pool.

HBase

tableinfo files and data files of HBase system tables.

IoTDB

IoTDB metadata.
  • LocalDir
  • NFS
  • RemoteHDFS
  • CIFS
  • SFTP

ClickHouse

ClickHouse metadata.
  • LocalDir
  • RemoteHDFS
Table 4 Service data of specific components (MRS 3.x)

Backup Type

Backup Content

Backup Directory Type

HBase

Table-level user data. For a cluster with multiple services installed, backup and restoration are supported for multiple HBase service instances and the backup and restoration operations are consistent with those of a single HBase service instance.
  • RemoteHDFS
  • NFS
  • CIFS
  • SFTP

HDFS

Directories or files of user services.

NOTE:

Encrypted directories cannot be backed up or restored.

Hive

Table-level user data. For a cluster with multiple services installed, backup and restoration are supported for multiple Hive service instances and the backup and restoration operations are consistent with those of a single Hive service instance.

IoTDB

IoTDB service data.
  • RemoteHDFS

ClickHouse

Table-level user data.
  • RemoteHDFS

Note that some components do not provide data backup or restoration:

  • Kafka supports replicas and allows multiple replicas to be specified when a topic is created.
  • CDL data is stored in DBService and Kafka. A system administrator can create DBService and Kafka backup tasks to back up data.
  • MapReduce and Yarn data is stored in HDFS. Therefore, they rely on the backup and restoration provided by HDFS.
  • Backup and restoration of service data in ZooKeeper are performed by their own upper-layer components.
Parent topic: MRS Cluster Management