2.6升级到3.3通过快照迁移数据

新集群3.3是和旧集群2.6配置一样,节点数保持一直

通过快照备份迁移恢复的方式,迁移数据

快照数据拷贝到新集群 一启动就给删了

目前只有工具一种方式。。。请严格按照文档操作,不要跳过任何步骤。

1 个赞

不同的版本通过快照的方式不一定靠谱。数据格式有可能都变了。

是说2.6 升级到 3.3.0 然后迁移的?还是 2.6 直接迁移? 后者肯定不行

我的旧集群是2.6,新集群是3.3,现在是用快照的方式直接迁移到新集群,这种方式不行吗?这种方式一启动就给删了

是需要旧集群2.6–快照迁移–新集群2.6–新集群2.6升级3.3–迁移数据 这种方案才行吗?

@wey

不行的, 因为大版本之间(2.6到3.0)之间,底层的存储结构变化了(因为有一些优化是要趁着大版本机会做掉的,那时候数据可以不兼容)

那目前有什么方案支持我这种场景吗?
数据不兼容,就只能数据导出,然后导入到新版本?

可以用升级的程序去升级数据文件哈,文档里有的哈~~

这种就可以,如果你新集群已经装好了高版本,你可以把 2.6 搞到一个中间集群2.6上,升级它的数据,再搞到 3.3 的新集群里。


我们集群部署采用的是docker。请问这种情况怎么去升级数据呢?

可以试着用二进制包里的 binary upgrade 工具,放到 storaged 里执行,可以在测试环境先演练一下

是。这个没专门验证过,不过理论上是可行的。数据解析的格式改了

1 个赞

在2.6的服务的storaged执行以下命令

./db_upgrader
–src_db_path=/usr/local/nebula/data/storage
–dst_db_path=/usr/local/nebula/data/storage-v3
–upgrade_meta_server=192.168.66.79:49232,192.168.66.79:49233,192.168.66.79:49228
–upgrade_version=2:3

upgrade_meta_server是3.3的
db_upgrader也是3.3的

数据路径:

日志

===========================PARAMS============================
meta server: 192.168.66.79:49232,192.168.66.79:49233,192.168.66.79:49228
source data path: /usr/local/nebula/data/storage
destination data path: /usr/local/nebula/data/storage-v3
The size of the batch written: 100
upgrade data from version: 2:3
whether to compact all data: true
maximum number of concurrent parts allowed:10
maximum number of concurrent spaces allowed: 5
===========================PARAMS============================

I20221220 03:15:17.504247   560 DbUpgraderTool.cpp:112] Prepare phase begin
I20221220 03:15:17.504603   560 MetaClient.cpp:80] Create meta client to "192.168.66.79":49233
I20221220 03:15:17.504608   560 MetaClient.cpp:81] root path: , data path size: 0
E20221220 03:15:18.513824   561 ThriftClientManager-inl.h:70] Failed to resolve address for 'metad1': Name or service not known (error=-2): Unknown error -2
I20221220 03:15:19.533504   560 MetaClient.cpp:3108] Load leader of "storaged0":9779 in 3 space
I20221220 03:15:19.533535   560 MetaClient.cpp:3108] Load leader of "storaged1":9779 in 3 space
I20221220 03:15:19.533538   560 MetaClient.cpp:3108] Load leader of "storaged2":9779 in 3 space
I20221220 03:15:19.533541   560 MetaClient.cpp:3114] Load leader ok
I20221220 03:15:19.536188   560 MetaClient.cpp:162] Register time task for heartbeat!
I20221220 03:15:19.536207   560 DbUpgraderTool.cpp:171] Prepare phase end
I20221220 03:15:19.536209   560 DbUpgraderTool.cpp:174] Upgrade phase begin
I20221220 03:15:19.536991   568 DbUpgraderTool.cpp:185] Upgrade from path /usr/local/nebula/data/storage to path /usr/local/nebula/data/storage-v3 begin
I20221220 03:15:19.537659   568 DbUpgrader.cpp:1169] Upgrade from path /usr/local/nebula/data/storage to path /usr/local/nebula/data/storage-v3 in DbUpgrader run begin
E20221220 03:15:19.537685   568 MetaClient.cpp:1393] Space 806 not found!
E20221220 03:15:19.537737   568 DbUpgrader.cpp:76] Space id 806 no found
E20221220 03:15:19.537744   568 DbUpgrader.cpp:59] Init /usr/local/nebula/data/storage space id 806 failed
W20221220 03:15:19.537756   568 DbUpgrader.cpp:1180] Upgrade from path /usr/local/nebula/data/storage space id 806 to path /usr/local/nebula/data/storage-v3 init failed
W20221220 03:15:19.537766   568 DbUpgrader.cpp:1182] Ignore upgrade /usr/local/nebula/data/storage space id 806
E20221220 03:15:19.537781   568 MetaClient.cpp:1393] Space 0 not found!
E20221220 03:15:19.537787   568 DbUpgrader.cpp:76] Space id 0 no found
E20221220 03:15:19.537799   568 DbUpgrader.cpp:59] Init /usr/local/nebula/data/storage space id 0 failed
W20221220 03:15:19.537806   568 DbUpgrader.cpp:1180] Upgrade from path /usr/local/nebula/data/storage space id 0 to path /usr/local/nebula/data/storage-v3 init failed
W20221220 03:15:19.537812   568 DbUpgrader.cpp:1182] Ignore upgrade /usr/local/nebula/data/storage space id 0
E20221220 03:15:19.537818   568 MetaClient.cpp:1393] Space 796 not found!
E20221220 03:15:19.537827   568 DbUpgrader.cpp:76] Space id 796 no found
E20221220 03:15:19.537834   568 DbUpgrader.cpp:59] Init /usr/local/nebula/data/storage space id 796 failed
W20221220 03:15:19.537840   568 DbUpgrader.cpp:1180] Upgrade from path /usr/local/nebula/data/storage space id 796 to path /usr/local/nebula/data/storage-v3 init failed
W20221220 03:15:19.537845   568 DbUpgrader.cpp:1182] Ignore upgrade /usr/local/nebula/data/storage space id 796
E20221220 03:15:19.537853   568 MetaClient.cpp:1393] Space 609 not found!
E20221220 03:15:19.537860   568 DbUpgrader.cpp:76] Space id 609 no found
E20221220 03:15:19.537866   568 DbUpgrader.cpp:59] Init /usr/local/nebula/data/storage space id 609 failed
W20221220 03:15:19.537878   568 DbUpgrader.cpp:1180] Upgrade from path /usr/local/nebula/data/storage space id 609 to path /usr/local/nebula/data/storage-v3 init failed
W20221220 03:15:19.537883   568 DbUpgrader.cpp:1182] Ignore upgrade /usr/local/nebula/data/storage space id 609
I20221220 03:15:19.537889   568 DbUpgrader.cpp:1196] Max concurrent spaces: 0
I20221220 03:15:19.537894   568 DbUpgrader.cpp:1206] Upgrade from path /usr/local/nebula/data/storage to path /usr/local/nebula/data/storage-v3 in DbUpgrader run end
I20221220 03:15:19.537896   568 DbUpgraderTool.cpp:193] Upgrade from path /usr/local/nebula/data/storage to path /usr/local/nebula/data/storage-v3 end
I20221220 03:15:19.537964   560 DbUpgraderTool.cpp:202] Upgrade phase end
[root@nebula-cluster-storaged-3 bin]# 

@wey @min.wu

1 个赞

meta启动了吗?


启动了的,一个3.3的完整服务


地址似乎有问题,工具解析不到。查下你容器内外的网络联通性


space 还是没找到

你正常登录下你的服务,看看这些space还在不在。meta好像都不认为他们存在



数据都是有的

–upgrade_meta_server=192.168.66.79:49232,192.168.66.79:49233,192.168.66.79:49228

是指的3.3服务的meta吗?

我是在2.6的环境下进行db_upgrader,2.6的服务都是正常的,space能正常操作

如果我没记错,你需要启动3.X的meta (graph有必要也可以启动 但一定不要启动storage),然后用3.X的db_upgrader来升级2.6的storage,升级成功才能启动storage