Nebula Graph 提供快照(snapshot)功能,用于保存集群当前时间点的数据状态,当出现数据丢失或误操作时,可以通过快照恢复数据。
那是否可以通过 snapshot 进行数据迁移呢?本文就为大家带来利用 snapshot 进行集群迁移的实践。
本文在1.x和2.x版本下进行过测试,3.x版本暂未进行过测试。
前置条件
源集群与新集群同构(节点数量,磁盘数量,数据目录一致)
新集群 Nebula 已部署(无需启动服务)
本次实操以三节点集群为例:源集群(192.168.8.191,192.168.8.192,192.168.8.193)、新集群(192.168.8.91,192.168.8.92,192.168.8.93)。
操作步骤
在源集群创建 snapshot
(root@nebula) [(none)]> CREATE SNAPSHOT;
Execution succeeded (Time spent: 5.1133/5.11391 s)
Tue Nov 30 04:05:08 2021
(root@nebula) [(none)]> SHOW SNAPSHOTS;
=========================================================================================================
| Name | Status | Hosts |
=========================================================================================================
| SNAPSHOT_2021_11_30_04_05_03 | VALID | 192.168.8.191:44500, 192.168.8.192:44500, 192.168.8.193:44500 |
---------------------------------------------------------------------------------------------------------
Got 1 rows (Time spent: 1.24/1.875 ms)
Tue Nov 30 04:05:34 2021
拷贝文件
# 拷贝 meta 文件
-- ⚠️snapshot 只会在 meta leader 节点生成 checkpoints 目录,这里是 192.168.8.191,需要拷贝至新集群所有 meta 节点
-- 在 192.168.8.91,92,93 执行以下命令
$ cd /usr/local/nebula/data/meta/nebula/0/
$ sudo scp -r vesoft@192.168.8.191:/usr/local/nebula/data/meta/nebula/0/checkpoints/SNAPSHOT_2021_11_30_04_05_03/* ./
# 拷贝 storage 文件
-- ⚠️目录中 14 为对应 space 的 ID,192.168.8.191,192,193 -> 192.168.8.91,92,93 一对一拷贝
-- 192.168.8.91
$ cd /usr/local/nebula/data/storage/nebula/
$ sudo mkdir 14
$ cd 14
$ sudo scp -r vesoft@192.168.8.191:/usr/local/nebula/data/storage/nebula/14/checkpoints/SNAPSHOT_2021_11_30_04_05_03/* ./
-- 192.168.8.92
$ cd /usr/local/nebula/data/storage/nebula/
$ sudo mkdir 14
$ cd 14
$ sudo scp -r vesoft@192.168.8.192:/usr/local/nebula/data/storage/nebula/14/checkpoints/SNAPSHOT_2021_11_30_04_05_03/* ./
-- 192.168.8.93
$ cd /usr/local/nebula/data/storage/nebula/
$ sudo mkdir 14
$ cd 14
$ sudo scp -r vesoft@192.168.8.193:/usr/local/nebula/data/storage/nebula/14/checkpoints/SNAPSHOT_2021_11_30_04_05_03/* ./
新集群启动 meta 和 graph 服务
$ sudo /usr/local/nebula/scripts/nebula.service start metad
$ sudo /usr/local/nebula/scripts/nebula.service start graphd
查看新集群 meta leader 节点
# 查看 meta 日志,如果看到“The partition is elected as the leader”说明改节点为 leader
$ grep "leader" /usr/local/nebula/logs/nebula-metad.INFO
I1130 08:45:48.413197 26299 RaftPart.cpp:1179] [Port: 45501, Space: 0, Part: 0] The partition is elected as the leader
修改 meta 信息
# 192.168.8.92 为上一步查到的 meta leader 节点,from 和 to 参数分别对应源 IP 和目标 IP。如果返回“Replace Host successfully”即为成功。
-- ⚠️这里使用的是 1.2 版本,http 端口为 11000,如果是 2.x 版本端口为 19559
curl -G "http://192.168.8.92:11000/replace?from=192.168.8.191&to=192.168.8.91"
curl -G "http://192.168.8.92:11000/replace?from=192.168.8.192&to=192.168.8.92"
curl -G "http://192.168.8.92:11000/replace?from=192.168.8.193&to=192.168.8.93"
新集群启动 storage 服务
$ sudo /usr/local/nebula/scripts/nebula.service start storaged
检查服务是否正常
# show hosts 查看服务和 partition 分布
(root@nebula) [basketballplayer]> show hosts;
==================================================================================================
| Ip | Port | Status | Leader count | Leader distribution | Partition distribution |
==================================================================================================
| 192.168.8.91 | 44500 | online | 30 | basketballplayer: 30 | basketballplayer: 30 |
--------------------------------------------------------------------------------------------------
| 192.168.8.92 | 44500 | online | 0 | | basketballplayer: 30 |
--------------------------------------------------------------------------------------------------
| 192.168.8.93 | 44500 | online | 0 | | basketballplayer: 30 |
--------------------------------------------------------------------------------------------------
| Total | | | 30 | basketballplayer: 30 | basketballplayer: 90 |
--------------------------------------------------------------------------------------------------
Got 8 rows (Time spent: 1.298/2.199 ms)
Tue Nov 30 09:09:58 2021
# 如果 leader 分布不均匀,手工 balance
(root@nebula) [basketballplayer]> balance leader;
Execution succeeded (Time spent: 5.02013/5.0213 s)
Tue Nov 30 09:13:31 2021
# 检查业务数据
(root@nebula) [basketballplayer]> fetch prop on player 100;
=======================================
| VertexID | player.name | player.age |
=======================================
| 100 | Tim Duncan | 42 |
---------------------------------------
Got 1 rows (Time spent: 9.689/10.488 ms)
Tue Nov 30 09:09:55 2021
以上便是实操的全部过程。最后提醒下各位,Nebula Graph 的身份认证功能默认是关闭的,此时任何用户都能使用快照功能。如果身份认证开启,仅 God 角色用户可以使用快照功能。
交流图数据库技术?加入 Nebula 交流群请先填写下你的 Nebula 名片,Nebula 小助手会拉你进群~~
这是一个从 https://nebula-graph.com.cn/posts/use-snapshot-to-migrate-cluster/ 下的原始话题分离的讨论话题