-
nebula 版本：
-
部署方式：单机
-
安装方式：Docker
-
是否上生产环境： N
-
硬件信息
- 磁盘：SSD
- CPU、内存信息：4c 8g
-
问题的具体描述
- nebula-br 和 agent 的部署使用参考了大佬的模板
nebula-up/backup_restore at main · wey-gu/nebula-up (github.com) - 我把最新的 br 和 agent 塞进了 nebula-br:3.6.0 里面，备份是正常没有问题的，但还原的时候报错了：Error: restore cluster meta failed: restore meta service metad0:9559 failed: call metad0:9559:RestoreMeta failed: E_RESTORE_FAILURE
- 找了一下论坛和issue，相关的信息都比较旧，看着说当时是不支持k8s和docker的还原，想问下现在有正确的处理方式么？
- nebula-br 和 agent 的部署使用参考了大佬的模板
-
相关的 meta / storage / graph info 日志信息
[root@fat-dba-10.2.2.101 nebula-br]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
acb6c0e63833 harbor.xxx.com/devops/nebula-br:3.6.0 "/usr/local/bin/agen…" About a minute ago Up About a minute nebula-metad0-agent
ab91fde2748a harbor.xxx.com/devops/nebula-br:3.6.0 "/usr/local/bin/agen…" About a minute ago Up About a minute nebula-storaged0-agent
3636f98495b6 harbor.xxx.com/devops/minio:latest "sh -c 'mkdir -p /mi…" 3 hours ago Up 3 hours (healthy) 0.0.0.0:9000-9001->9000-9001/tcp nebula-minio
c47cb90c83b2 harbor.xxx.com/devops/nebula-br:3.6.0 "/usr/local/bin/agen…" 4 hours ago Up 4 hours nebula-graphd-agent
1f547721092f harbor.xxx.com/devops/nebula-console:v3.6.0 "sh -c 'for i in `se…" 5 hours ago Up 5 hours nebula-console
eecbb75f0ce4 harbor.xxx.com/devops/nebula-graphd:v3.8.0 "/usr/local/nebula/b…" 5 hours ago Up 5 hours (healthy) 0.0.0.0:9669->9669/tcp, 0.0.0.0:19669->19669/tcp, 0.0.0.0:1039->19670/tcp nebula-graphd
041c39259447 harbor.xxx.com/devops/nebula-storaged:v3.8.0 "/usr/local/nebula/b…" 5 hours ago Up 5 hours (healthy) 9777-9778/tcp, 0.0.0.0:9779->9779/tcp, 9780/tcp, 0.0.0.0:19779->19779/tcp, 0.0.0.0:1038->19780/tcp nebula-storaged0
c268b9af60de harbor.xxx.com/devops/nebula-metad:v3.8.0 "/usr/local/nebula/b…" 5 hours ago Up 5 hours (healthy) 0.0.0.0:9559->9559/tcp, 0.0.0.0:19559->19559/tcp, 9560/tcp, 0.0.0.0:1037->19560/tcp nebula-metad0
16512217a6e0 prom/node-exporter:latest "/bin/node_exporter …" 4 days ago Up 4 days 0.0.0.0:9182->9100/tcp node-exporter
9556dafa006f harbor.xxx.com/devops/nebula-graph-studio:v3.10.0 "./server" 10 days ago Up 10 days 0.0.0.0:7001->7001/tcp nebula-graph-studio
[root@fat-pmp-10.2.30.61 nebula]# docker exec -it nebula-console /usr/local/bin/nebula-console -u root -p nebula --address=graphd --port=9669
Welcome!
(root@nebula) [(none)]> SHOW HOSTS AGENT;
+-------------+------+-----------+---------+--------------+---------+
| Host | Port | Status | Role | Git Info Sha | Version |
+-------------+------+-----------+---------+--------------+---------+
| "agent" | 8888 | "OFFLINE" | "AGENT" | "0bbd267" | |
| "graphd" | 8888 | "ONLINE" | "AGENT" | "0bbd267" | |
| "metad0" | 8888 | "ONLINE" | "AGENT" | "0bbd267" | |
| "storaged0" | 8888 | "ONLINE" | "AGENT" | "0bbd267" | |
+-------------+------+-----------+---------+--------------+---------+
[root@fat-dba-10.2.2.101 nebula-br]#
[root@fat-dba-10.2.2.101 nebula]# docker exec -it nebula-metad0-agent br show --s3.access_key=AAAAAAAAAAAAAAAAAAAB --s3.secret_key=BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB --storage="s3://nebula/backup/" --s3.endpoint "http://10.2.2.101:9000"
+----------------------------+---------------------+---------------------------------------------------------------------------------------------------------------------+-------------+------------+
| NAME | CREATE TIME | SPACES | FULL BACKUP | ALL SPACES |
+----------------------------+---------------------+---------------------------------------------------------------------------------------------------------------------+-------------+------------+
| BACKUP_2024_06_17_07_31_07 | 2024-06-17 07:31:12 | bapd00ptbnd1794877711697182091536169,bapd4f109686d52f2a7dff08bb3b564c5af9,bapd00ptbnd1154883264169449833938816,bapd | true | true |
| BACKUP_2024_06_17_07_43_33 | 2024-06-17 07:43:38 | bapd00ptbnd1154883264169449833938816,bapd,bapd00ptbnd1794877711697182091536169,bapd4f109686d52f2a7dff08bb3b564c5af9 | true | true |
+----------------------------+---------------------+---------------------------------------------------------------------------------------------------------------------+-------------+------------+
[root@fat-dba-10.2.2.101 nebula-br]#
[root@fat-dba-10.2.2.101 nebula-br]# docker exec -it nebula-metad0-agent br restore full --meta "metad0:9559" --s3.access_key=AAAAAAAAAAAAAAAAAAAA --s3.secret_key=BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB --storage="s3://nebula/backup/" --s3.endpoint "http://10.2.2.101:9000" --name BACKUP_2024_06_17_07_43_33
{"level":"info","meta address":"metad0:9559","msg":"Try to connect meta service.","time":"2024-06-17T07:44:05.082Z"}
{"level":"info","meta address":"metad0:9559","msg":"Connect meta server successfully.","time":"2024-06-17T07:44:05.085Z"}
{"level":"info","meta address":"metad0:9559","msg":"Try to connect meta service.","time":"2024-06-17T07:44:05.086Z"}
{"level":"info","meta address":"metad0:9559","msg":"Connect meta server successfully.","time":"2024-06-17T07:44:05.089Z"}
{"host info":"map[graphd:graphd:8888[AGENT]: (data: , root: ) | graphd:9669[GRAPH]: (data: , root: /usr/local/nebula) metad0:metad0:8888[AGENT]: (data: , root: ) | metad0:9559[META]: (data: /data/meta, root: /usr/local/nebula) storaged0:storaged0:9779[STORAGE]: (data: /data/storage, root: /usr/local/nebula) | storaged0:8888[AGENT]: (data: , root: )]","level":"info","msg":"Get cluster topology from the nebula.","time":"2024-06-17T07:44:05.090Z"}
{"backup":"BACKUP_2024_06_17_07_43_33","level":"info","msg":"Check backup dir successfully.","time":"2024-06-17T07:44:05.097Z","uri":"s3://nebula/backup/BACKUP_2024_06_17_07_43_33"}
{"dir":"/usr/local/nebula","host":"storaged0","level":"info","msg":"Stop services.","role":"STORAGE","time":"2024-06-17T07:44:05.100Z"}
{"dir":"/usr/local/nebula","host":"graphd","level":"info","msg":"Stop services.","role":"GRAPH","time":"2024-06-17T07:44:05.150Z"}
{"dir":"/usr/local/nebula","host":"metad0","level":"info","msg":"Stop services.","role":"META","time":"2024-06-17T07:44:05.202Z"}
{"backup":"BACKUP_2024_06_17_07_43_33","level":"info","msg":"Stop cluster successfully.","time":"2024-06-17T07:44:05.251Z"}
{"addr":"storaged0:9779","backup path":"/data/storage/nebula_old_1718610245","level":"info","msg":"Backup origin storage data path successfully.","origin path":"/data/storage/nebula","time":"2024-06-17T07:44:05.253Z"}
{"addr":"storaged0:9779","backup path":"/usr/local/nebula/cluster.id_old_1718610245","level":"info","msg":"Backup origin cluster.id path successfully.","origin not exist":true,"origin path":"/usr/local/nebula/cluster.id","time":"2024-06-17T07:44:05.253Z"}
{"addr":"metad0:9559","backup path":"/data/meta/nebula_old_1718610245","level":"info","msg":"Backup origin meta data path successfully.","origin path":"/data/meta/nebula","time":"2024-06-17T07:44:05.254Z"}
{"backup":"BACKUP_2024_06_17_07_43_33","level":"info","msg":"Backup origin cluster data successfully.","time":"2024-06-17T07:44:05.255Z"}
{"level":"info","msg":"Download meta data to cluster successfully.","time":"2024-06-17T07:44:05.286Z"}
{"addr":"storaged0:9779","external":"s3://nebula/backup/BACKUP_2024_06_17_07_43_33/data/storaged0:9779/data0/","level":"info","local":"/data/storage/nebula","msg":"Download storage data successfully.","time":"2024-06-17T07:44:06.867Z"}
{"level":"info","msg":"Download storage data to cluster successfully.","time":"2024-06-17T07:44:06.868Z"}
{"addr":"metad0:9559","level":"info","msg":"Start meta service successfully.","time":"2024-06-17T07:44:06.925Z"}
{"level":"info","msg":"Start meta service successfully.","time":"2024-06-17T07:44:16.926Z"}
{"level":"info","meta address":"metad0:9559","msg":"Try to connect meta service.","time":"2024-06-17T07:44:16.926Z"}
{"level":"info","meta address":"metad0:9559","msg":"Connect meta server successfully.","time":"2024-06-17T07:44:16.930Z"}
{"level":"info","msg":"All services are OK.","time":"2024-06-17T07:44:17.096Z"}
Error: restore cluster meta failed: restore meta service metad0:9559 failed: call metad0:9559:RestoreMeta failed: E_RESTORE_FAILURE
Usage:
br restore full [flags]
Flags:
-h, --help help for full
Global Flags:
--concurrency int Max concurrency for download data (default 5)
--debug Output log in debug level or not
--log string Specify br detail log path (default "br.log")
--meta string Specify meta server
--name string Specify backup name
--s3.access_key string S3 Option: set access key id
--s3.endpoint string S3 Option: set the S3 endpoint URL, please specify the http or https scheme explicitly
--s3.region string S3 Option: set region or location to upload or download backup
--s3.secret_key string S3 Option: set secret key for access id
--storage string backup target url, format: <SCHEME>://<PATH>.
<SCHEME>: a string indicating which backend type. optional: local, s3.
now only s3-compatible is supported.
example:
for local - "local:///the/local/path/to/backup"
for s3 - "s3://example/url/to/the/backup"
{"level":"fatal","msg":"restore cluster meta failed: restore meta service metad0:9559 failed: call metad0:9559:RestoreMeta failed: E_RESTORE_FAILURE","time":"2024-06-17T07:44:17.098Z"}
