Star

nebula meta节点一直都是unhealthy

提问参考模版:

  • nebula 版本:v1.2.0
  • 部署方式(分布式 / 单机 / Docker / DBaaS):docker单机
  • 硬件信息
    • 磁盘:HDD
    • CPU、内存信息:4核,16G
  • 问题的具体描述
  • 开发服docker单机部署nebula,出现meta节点一直都是unhealthy状态,看了日志如下

  • 这个问题怎么处理,重启很多次meta节点都没用

meta配置看一下

services:
  metad0:
    image: vesoft/nebula-metad:v1.2.0
    environment:
      USER: root
      TZ:   "${TZ}"
    command:
      - --meta_server_addrs=172.28.1.1:45500,172.28.1.2:45500,172.28.1.3:45500
      - --local_ip=172.28.1.1
      - --ws_ip=172.28.1.1
      - --port=45500
      - --data_path=/data/meta
      - --log_dir=/logs
      - --v=0
      - --minloglevel=0
    healthcheck:
      test: ["CMD", "curl", "-f", "http://172.28.1.1:11000/status"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 20s
    ports:
      - "45500:45500"
      - 11000
      - 11002
    volumes:
      - /mnt2/data/meta0:/data/meta
      - /mnt2/logs/meta0:/logs
    networks:
      nebula-net:
        ipv4_address: 172.28.1.1
    restart: on-failure
    cap_add:
      - SYS_PTRACE

metad1:
    image: vesoft/nebula-metad:v1.2.0
    environment:
      USER: root
      TZ:   "${TZ}"
    command:
      - --meta_server_addrs=172.28.1.1:45500,172.28.1.2:45500,172.28.1.3:45500
      - --local_ip=172.28.1.2
      - --ws_ip=172.28.1.2
      - --port=45500
      - --data_path=/data/meta
      - --log_dir=/logs
      - --v=0
      - --minloglevel=0
    healthcheck:
      test: ["CMD", "curl", "-f", "http://172.28.1.2:11000/status"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 20s
    ports:
      - "45501:45500"
      - 11000
      - 11002
    volumes:
      - /mnt2/data/meta1:/data/meta
      - /mnt2/logs/meta1:/logs
    networks:
      nebula-net:
        ipv4_address: 172.28.1.2
    restart: on-failure
    cap_add:
      - SYS_PTRACE

  metad2:
    image: vesoft/nebula-metad:v1.2.0
    environment:
      USER: root
      TZ:   "${TZ}"
    command:
      - --meta_server_addrs=172.28.1.1:45500,172.28.1.2:45500,172.28.1.3:45500
      - --local_ip=172.28.1.3
      - --ws_ip=172.28.1.3
      - --port=45500
      - --data_path=/data/meta
      - --log_dir=/logs
      - --v=0
      - --minloglevel=0
    healthcheck:
      test: ["CMD", "curl", "-f", "http://172.28.1.3:11000/status"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 20s
    ports:
      - "45502:45500"
      - 11000
      - 11002
    volumes:
      - /mnt2/data/meta2:/data/meta
      - /mnt2/logs/meta2:/logs
    networks:
      nebula-net:
        ipv4_address: 172.28.1.3
    restart: on-failure
    cap_add:
      - SYS_PTRACE

配置贴上了

–meta_server_addrs
只有一个meta,就填一个地址

什么意思,只填一个地址,那meta1和meta2都不需要了?

看错了,以为你是单机的

这个问题怎么解决啊

吧meta的log都贴一下


meta0,1,2的info日志

多贴几行

I0224 08:40:16.585942    50 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:16.642772     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:17.529587    47 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 39, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:17.642917     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:18.531493    47 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:18.643127     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:19.301972    48 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 40, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:19.643276     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:20.304126    48 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:20.643461     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:21.285346    49 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 41, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:21.643627     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:22.287544    49 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:22.643805     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:23.644009     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:23.655102    50 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 42, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:24.644150     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:24.656946    50 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:25.644302     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:25.891319    47 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 43, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:26.644465     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:26.893460    47 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:27.644627     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:28.644804     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:28.878537    48 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 44, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:29.644976     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:29.878109    48 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:30.645218     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:31.255642    49 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 45, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:31.645336     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:32.255687    49 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:32.645484     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:33.240873    50 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 46, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:33.645632     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:34.239048    50 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:34.645782     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:35.645925     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:35.779754    47 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 47, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:36.646076     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:36.781786    47 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:37.646265     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:37.705845    48 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 48, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:38.646458     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:38.707386    48 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:39.286044    49 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 49, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:39.646669     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:40.287583    49 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:40.646862     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:40.928328    50 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 50, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:41.647025     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:41.928426    50 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:42.647215     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:43.586294    47 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 51, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:43.647488     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:44.588016    47 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:44.647673     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:45.377974    48 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 52, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:45.647810     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:46.379891    48 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:46.647974     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:47.648185     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:48.157814    49 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 53, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:48.648377     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:49.159860    49 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:49.648563     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:50.355198    50 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 54, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:50.648730     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:51.357383    50 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:51.648948     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:52.649099     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:53.063297    47 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 55, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:53.649329     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:54.065521    47 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:54.649492     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:55.190881    48 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 56, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:55.649732     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:56.190944    48 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:56.649991     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:57.650269     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:57.980063    49 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 57, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:40:58.650429     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:40:58.982218    49 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:40:59.650583     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:41:00.507033    50 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 58, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:41:00.650784     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:41:01.508353    50 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:41:01.651000     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:41:02.651146     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:41:03.058151    47 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 59, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:41:03.651307     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:41:04.059854    47 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:41:04.590459    48 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 60, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:41:04.651473     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:41:05.592214    48 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:41:05.651628     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:41:06.651765     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:41:07.433797    49 RaftPart.cpp:1124] [Port: 45501, Space: 0, Part: 0] Sending out an election request (space = 0, part = 0, term = 61, lastLogId = 0, lastLogTerm = 0, candidateIP = 172.28.1.3, candidatePort = 45501)
I0224 08:41:07.651965     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s
I0224 08:41:08.435485    49 RaftPart.cpp:1206] [Port: 45501, Space: 0, Part: 0] No one is elected, continue the election
I0224 08:41:08.652122     1 MetaDaemon.cpp:106] Leader has not been elected, sleep 1s

这是log,重复出现

三个meta的log都看看

一样的,重复的。。。

那看看meta间网络连通

怎么看,有什么命令?

到容器里ping一下吧

互相ping不通,所以导致选举主节点失败是吧

三个meta节点互相ping不通,怎么处理这个网络问题?

浙ICP备20010487号