Heartbeat failed, status:Machine not existed!

jingnny · 2022 年10 月 17 日 01:42

1、2台机器
2、机器Ameta 配置

--meta_server_addrs=10.58.20.125:9559,10.58.20.126:9559
# Local IP used to identify the nebula-metad process.
# Change it to an address other than loopback if the service is distributed or
# will be accessed remotely.
--local_ip=10.58.20.125
# Meta daemon listening port
--port=9559
# HTTP service ip
--ws_ip=0.0.0.0
# HTTP service port
--ws_http_port=19559
# HTTP2 service port
--ws_h2_port=19560
# Port to listen on Storage with HTTP protocol, it corresponds to ws_http_port in storage's configuration file
--ws_storage_http_port=19779

graph配置
########## networking ##########
# Comma separated Meta Server Addresses
--meta_server_addrs=10.58.20.125:9559,10.58.20.126:9559
# Local IP used to identify the nebula-graphd process.
# Change it to an address other than loopback if the service is distributed or
# will be accessed remotely.
--local_ip=10.58.20.125
# Network device to listen on
--listen_netdev=any
# Port to listen on
--port=9669
# To turn on SO_REUSEPORT or not
--reuse_port=false
# Backlog of the listen socket, adjust this together with net.core.somaxconn
--listen_backlog=1024
# The number of seconds Nebula service waits before closing the idle connections
--client_idle_timeout_secs=28800
# The number of seconds before idle sessions expire
# The range should be in [1, 604800]
--session_idle_timeout_secs=28800
# The number of threads to accept incoming connections
--num_accept_threads=1
# The number of networking IO threads, 0 for # of CPU cores
--num_netio_threads=0
# The number of threads to execute user queries, 0 for # of CPU cores
--num_worker_threads=0
# HTTP service ip
--ws_ip=0.0.0.0
# HTTP service port
--ws_http_port=19669
# HTTP2 service port
--ws_h2_port=19670


########## networking ##########
# Comma separated Meta server addresses
--meta_server_addrs=10.58.20.125:9559,10.58.20.126:9559
# Local IP used to identify the nebula-storaged process.
# Change it to an address other than loopback if the service is distributed or
# will be accessed remotely.
--local_ip=10.58.20.125
# Storage daemon listening port
--port=9779
# HTTP service ip
--ws_ip=0.0.0.0
# HTTP service port
--ws_http_port=19779
# HTTP2 service port
--ws_h2_port=19780
# heartbeat with meta service
--heartbeat_interval_secs=10

机器B配置
meta
########## networking ##########
# Comma separated Meta Server addresses
--meta_server_addrs=10.58.20.126:9559,10.58.20.125:9559
# Local IP used to identify the nebula-metad process.
# Change it to an address other than loopback if the service is distributed or
# will be accessed remotely.
--local_ip=10.58.20.126
# Meta daemon listening port
--port=9559
# HTTP service ip
--ws_ip=0.0.0.0
# HTTP service port
--ws_http_port=19559
# HTTP2 service port
--ws_h2_port=19560
# Port to listen on Storage with HTTP protocol, it corresponds to ws_http_port in storage's configuration file
--ws_storage_http_port=19779

graph
# Comma separated Meta Server Addresses
--meta_server_addrs=10.58.20.126:9559,10.58.20.125:9559
# Local IP used to identify the nebula-graphd process.
# Change it to an address other than loopback if the service is distributed or
# will be accessed remotely.
--local_ip=10.58.20.126
# Network device to listen on
--listen_netdev=any
# Port to listen on
--port=9669
# To turn on SO_REUSEPORT or not
--reuse_port=false
# Backlog of the listen socket, adjust this together with net.core.somaxconn
--listen_backlog=1024
# The number of seconds Nebula service waits before closing the idle connections
--client_idle_timeout_secs=28800
# The number of seconds before idle sessions expire
# The range should be in [1, 604800]
--session_idle_timeout_secs=288000
# The number of threads to accept incoming connections
--num_accept_threads=1
# The number of networking IO threads, 0 for # of CPU cores
--num_netio_threads=0
# The number of threads to execute user queries, 0 for # of CPU cores
--num_worker_threads=0
# HTTP service ip
--ws_ip=0.0.0.0
# HTTP service port
--ws_http_port=19669
# HTTP2 service port
--ws_h2_port=19670
# storage client timeout
--storage_client_timeout_ms=60000000
# Port to listen on Meta with HTT

storage
########## networking ##########
# Comma separated Meta server addresses
--meta_server_addrs=10.58.20.126:9559,10.58.20.125:9559
# Local IP used to identify the nebula-storaged process.
# Change it to an address other than loopback if the service is distributed or
# will be accessed remotely.
--local_ip=10.58.20.126
# Storage daemon listening port
--port=9779
# HTTP service ip
--ws_ip=0.0.0.0
# HTTP service port
--ws_http_port=19779
# HTTP2 service port
--ws_h2_port=19780
# heartbeat with meta service
--heartbeat_interval_secs=10

问题：

启动后一天左右storage服务自动退出
启动info日志（10/14启动）


Log file created at: 2022/10/14 12:58:36
Running on machine: ZH07-TXN-20126
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
I20221014 12:58:36.970633 10780 StorageDaemon.cpp:128] localhost = "10.58.20.126":9779
I20221014 12:58:36.971027 10780 StorageDaemon.cpp:143] data path= /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage
I20221014 12:58:36.998814 10780 MetaClient.cpp:84] Create meta client to "10.58.20.125":9559
I20221014 12:58:36.998893 10780 MetaClient.cpp:85] root path: /usr/local/nebula-graph-3.0.1.el7.x86_64, data path size: 1
I20221014 12:58:40.024912 10780 FileBasedClusterIdMan.cpp:53] Get clusterId: 2661116903471194233
I20221014 12:58:41.647307 10780 MetaClient.cpp:3228] Load leader of "10.58.20.125":9779 in 313 space
I20221014 12:58:41.647369 10780 MetaClient.cpp:3228] Load leader of "10.58.20.126":9779 in 313 space
I20221014 12:58:41.647375 10780 MetaClient.cpp:3234] Load leader ok
I20221014 12:58:41.659112 10780 MetaClient.cpp:144] Register time task for heartbeat!
I20221014 12:58:41.659149 10780 StorageServer.cpp:192] Init schema manager
I20221014 12:58:41.659175 10780 StorageServer.cpp:195] Init index manager
I20221014 12:58:41.659180 10780 StorageServer.cpp:198] Init kvstore
I20221014 12:58:41.659209 10780 NebulaStore.cpp:51] Start the raft service...
I20221014 12:58:41.660383 10780 NebulaSnapshotManager.cpp:25] Send snapshot is rate limited to 10485760 for each part by default
I20221014 12:58:41.660660 10780 RaftexService.cpp:63] Init thrift server for raft service, port: 9780
I20221014 12:58:41.660749 11046 RaftexService.cpp:94] Starting the Raftex Service
I20221014 12:58:41.666854 11046 RaftexService.cpp:84] Starting the Raftex Service on 9780
I20221014 12:58:41.666885 11046 RaftexService.cpp:106] Start the Raftex Service successfully
I20221014 12:58:41.667009 10780 NebulaStore.cpp:83] Scan the local path, and init the spaces_
I20221014 12:58:41.667341 10780 NebulaStore.cpp:89] Scan path "/usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252"
I20221014 12:58:41.667627 10780 RocksEngineConfig.cpp:343] Emplace rocksdb option max_bytes_for_level_base=268435456
I20221014 12:58:41.667654 10780 RocksEngineConfig.cpp:343] Emplace rocksdb option max_write_buffer_number=4
I20221014 12:58:41.667661 10780 RocksEngineConfig.cpp:343] Emplace rocksdb option write_buffer_size=67108864
I20221014 12:58:41.667932 10780 RocksEngineConfig.cpp:343] Emplace rocksdb option block_size=8192
I20221014 12:58:41.681746 10780 RocksEngine.cpp:142] open rocksdb on /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/data
I20221014 12:58:41.681764 10780 NebulaStore.cpp:113] Load space 252 from disk
I20221014 12:58:41.681829 10780 NebulaStore.cpp:141] Need to open 50 parts of space 252
I20221014 12:58:41.742007 11044 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 7] lastLogId in wal is 48264, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/7/0000000000000031868.wal
I20221014 12:58:41.742206 11044 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 7] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48264, lastLogTerm 11, committedLogId 48264, committedLogTerm 11, term 11
I20221014 12:58:41.742406 11044 NebulaStore.cpp:145] Load part 252, 7 from disk
I20221014 12:58:41.744592 11041 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 1] lastLogId in wal is 48289, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/1/0000000000000031875.wal
I20221014 12:58:41.744760 11041 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 1] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48289, lastLogTerm 11, committedLogId 48289, committedLogTerm 11, term 11
I20221014 12:58:41.744953 11041 NebulaStore.cpp:145] Load part 252, 1 from disk
I20221014 12:58:41.745452 11043 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 5] lastLogId in wal is 48286, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/5/0000000000000031877.wal
I20221014 12:58:41.745559 11043 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 5] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48286, lastLogTerm 11, committedLogId 48286, committedLogTerm 11, term 11
I20221014 12:58:41.745750 11043 NebulaStore.cpp:145] Load part 252, 5 from disk
I20221014 12:58:41.751300 11042 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 3] lastLogId in wal is 48288, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/3/0000000000000031884.wal
I20221014 12:58:41.751435 11042 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 3] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48288, lastLogTerm 11, committedLogId 48288, committedLogTerm 11, term 11
I20221014 12:58:41.751595 11042 NebulaStore.cpp:145] Load part 252, 3 from disk
I20221014 12:58:41.821789 11042 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 11] lastLogId in wal is 48289, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/11/0000000000000031874.wal
I20221014 12:58:41.821913 11042 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 11] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48289, lastLogTerm 11, committedLogId 48289, committedLogTerm 11, term 11
I20221014 12:58:41.821980 11042 NebulaStore.cpp:145] Load part 252, 11 from disk
I20221014 12:58:41.829073 11041 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 9] lastLogId in wal is 48293, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/9/0000000000000031879.wal
I20221014 12:58:41.829226 11041 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 9] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48293, lastLogTerm 11, committedLogId 48293, committedLogTerm 11, term 11
I20221014 12:58:41.829308 11041 NebulaStore.cpp:145] Load part 252, 9 from disk
I20221014 12:58:41.896246 11042 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 19] lastLogId in wal is 48282, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/19/0000000000000031875.wal
I20221014 12:58:41.896363 11042 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 19] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48282, lastLogTerm 11, committedLogId 48282, committedLogTerm 11, term 11
I20221014 12:58:41.896430 11042 NebulaStore.cpp:145] Load part 252, 19 from disk
I20221014 12:58:41.913297 11041 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 17] lastLogId in wal is 48278, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/17/0000000000000031877.wal
I20221014 12:58:41.913444 11041 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 17] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48278, lastLogTerm 11, committedLogId 48278, committedLogTerm 11, term 11
I20221014 12:58:41.913527 11041 NebulaStore.cpp:145] Load part 252, 17 from disk
I20221014 12:58:41.949375 11044 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 15] lastLogId in wal is 48281, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/15/0000000000000000001.wal
I20221014 12:58:41.949489 11044 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 15] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48281, lastLogTerm 11, committedLogId 48281, committedLogTerm 11, term 11
I20221014 12:58:41.949556 11044 NebulaStore.cpp:145] Load part 252, 15 from disk
I20221014 12:58:41.992776 11043 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 13] lastLogId in wal is 48288, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/13/0000000000000000001.wal
I20221014 12:58:41.992930 11043 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 13] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48288, lastLogTerm 11, committedLogId 48288, committedLogTerm 11, term 11
I20221014 12:58:41.993024 11043 NebulaStore.cpp:145] Load part 252, 13 from disk
I20221014 12:58:41.997540 11041 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 25] lastLogId in wal is 48278, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/25/0000000000000031874.wal
I20221014 12:58:41.997640 11041 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 25] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48278, lastLogTerm 11, committedLogId 48278, committedLogTerm 11, term 11
I20221014 12:58:41.997706 11041 NebulaStore.cpp:145] Load part 252, 25 from disk
I20221014 12:58:42.076238 11043 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 21] lastLogId in wal is 48280, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/21/0000000000000031869.wal
I20221014 12:58:42.076392 11043 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 21] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48280, lastLogTerm 11, committedLogId 48280, committedLogTerm 11, term 11
I20221014 12:58:42.076474 11043 NebulaStore.cpp:145] Load part 252, 21 from disk
I20221014 12:58:42.107128 11042 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 27] lastLogId in wal is 48273, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/27/0000000000000000001.wal
I20221014 12:58:42.107246 11042 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 27] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48273, lastLogTerm 11, committedLogId 48273, committedLogTerm 11, term 11
I20221014 12:58:42.107312 11042 NebulaStore.cpp:145] Load part 252, 27 from disk
I20221014 12:58:42.121151 11044 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 23] lastLogId in wal is 48286, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/23/0000000000000000001.wal
I20221014 12:58:42.121260 11044 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 23] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48286, lastLogTerm 11, committedLogId 48286, committedLogTerm 11, term 11
I20221014 12:58:42.121315 11044 NebulaStore.cpp:145] Load part 252, 23 from disk
I20221014 12:58:42.179291 11044 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 31] lastLogId in wal is 48280, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/31/0000000000000031876.wal
I20221014 12:58:42.179410 11044 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 31] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48280, lastLogTerm 11, committedLogId 48280, committedLogTerm 11, term 11
I20221014 12:58:42.179476 11044 NebulaStore.cpp:145] Load part 252, 31 from disk
I20221014 12:58:42.179622 11042 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 35] lastLogId in wal is 48287, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/35/0000000000000031875.wal
I20221014 12:58:42.179700 11042 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 35] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48287, lastLogTerm 11, committedLogId 48287, committedLogTerm 11, term 11
I20221014 12:58:42.179754 11042 NebulaStore.cpp:145] Load part 252, 35 from disk
I20221014 12:58:42.234723 11044 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 39] lastLogId in wal is 48280, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/39/0000000000000031873.wal
I20221014 12:58:42.234841 11044 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 39] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48280, lastLogTerm 11, committedLogId 48280, committedLogTerm 11, term 11
I20221014 12:58:42.234895 11044 NebulaStore.cpp:145] Load part 252, 39 from disk
I20221014 12:58:42.242352 11041 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 33] lastLogId in wal is 48284, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/33/0000000000000000001.wal
I20221014 12:58:42.242501 11041 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 33] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48284, lastLogTerm 11, committedLogId 48284, committedLogTerm 11, term 11
I20221014 12:58:42.242588 11041 NebulaStore.cpp:145] Load part 252, 33 from disk
I20221014 12:58:42.291098 11044 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 47] lastLogId in wal is 48285, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/47/0000000000000031876.wal
I20221014 12:58:42.291189 11044 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 47] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48285, lastLogTerm 11, committedLogId 48285, committedLogTerm 11, term 11
I20221014 12:58:42.291242 11044 NebulaStore.cpp:145] Load part 252, 47 from disk
I20221014 12:58:42.321319 11043 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 29] lastLogId in wal is 48289, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/29/0000000000000000001.wal
I20221014 12:58:42.321475 11043 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 29] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48289, lastLogTerm 11, committedLogId 48289, committedLogTerm 11, term 11
I20221014 12:58:42.321552 11043 NebulaStore.cpp:145] Load part 252, 29 from disk
I20221014 12:58:42.325867 11041 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 41] lastLogId in wal is 48293, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/41/0000000000000031884.wal
I20221014 12:58:42.325933 11041 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 41] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48293, lastLogTerm 11, committedLogId 48293, committedLogTerm 11, term 11
I20221014 12:58:42.325994 11041 NebulaStore.cpp:145] Load part 252, 41 from disk
I20221014 12:58:42.347733 11044 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 55] lastLogId in wal is 48292, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/55/0000000000000031892.wal
I20221014 12:58:42.347836 11044 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 55] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48292, lastLogTerm 11, committedLogId 48292, committedLogTerm 11, term 11
I20221014 12:58:42.347888 11044 NebulaStore.cpp:145] Load part 252, 55 from disk
I20221014 12:58:42.359925 11042 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 43] lastLogId in wal is 48285, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/43/0000000000000000001.wal
I20221014 12:58:42.360026 11042 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 43] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48285, lastLogTerm 11, committedLogId 48285, committedLogTerm 11, term 11
I20221014 12:58:42.360078 11042 NebulaStore.cpp:145] Load part 252, 43 from disk
I20221014 12:58:42.387814 11041 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 49] lastLogId in wal is 48283, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/49/0000000000000031877.wal
I20221014 12:58:42.387912 11041 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 49] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48283, lastLogTerm 11, committedLogId 48283, committedLogTerm 11, term 11
I20221014 12:58:42.387969 11041 NebulaStore.cpp:145] Load part 252, 49 from disk
I20221014 12:58:42.402444 11044 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 63] lastLogId in wal is 48283, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/63/0000000000000031877.wal
I20221014 12:58:42.402572 11044 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 63] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48283, lastLogTerm 11, committedLogId 48283, committedLogTerm 11, term 11
I20221014 12:58:42.402639 11044 NebulaStore.cpp:145] Load part 252, 63 from disk
I20221014 12:58:42.415148 11042 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 51] lastLogId in wal is 48290, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/51/0000000000000031877.wal
I20221014 12:58:42.415256 11042 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 51] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48290, lastLogTerm 11, committedLogId 48290, committedLogTerm 11, term 11
I20221014 12:58:42.415311 11042 NebulaStore.cpp:145] Load part 252, 51 from disk
I20221014 12:58:42.461297 11044 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 71] lastLogId in wal is 48282, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/71/0000000000000031879.wal
I20221014 12:58:42.461390 11044 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 71] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48282, lastLogTerm 11, committedLogId 48282, committedLogTerm 11, term 11
I20221014 12:58:42.461438 11044 NebulaStore.cpp:145] Load part 252, 71 from disk
I20221014 12:58:42.518890 11044 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 79] lastLogId in wal is 48278, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/79/0000000000000031875.wal
I20221014 12:58:42.519002 11044 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 79] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48278, lastLogTerm 11, committedLogId 48278, committedLogTerm 11, term 11
I20221014 12:58:42.519057 11044 NebulaStore.cpp:145] Load part 252, 79 from disk
I20221014 12:58:42.520678 11043 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 37] lastLogId in wal is 48294, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/37/0000000000000000001.wal
I20221014 12:58:42.520773 11043 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 37] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48294, lastLogTerm 11, committedLogId 48294, committedLogTerm 11, term 11
I20221014 12:58:42.520835 11043 NebulaStore.cpp:145] Load part 252, 37 from disk
I20221014 12:58:42.556841 11041 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 57] lastLogId in wal is 48286, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/57/0000000000000000001.wal
I20221014 12:58:42.556960 11041 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 57] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48286, lastLogTerm 11, committedLogId 48286, committedLogTerm 11, term 11
I20221014 12:58:42.557021 11041 NebulaStore.cpp:145] Load part 252, 57 from disk
I20221014 12:58:42.593310 11042 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 59] lastLogId in wal is 48301, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/59/0000000000000000001.wal
I20221014 12:58:42.593420 11042 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 59] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48301, lastLogTerm 11, committedLogId 48301, committedLogTerm 11, term 11
I20221014 12:58:42.593475 11042 NebulaStore.cpp:145] Load part 252, 59 from disk
I20221014 12:58:42.621454 11041 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 65] lastLogId in wal is 48281, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/65/0000000000000031876.wal
I20221014 12:58:42.621606 11041 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 65] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48281, lastLogTerm 11, committedLogId 48281, committedLogTerm 11, term 11
I20221014 12:58:42.621668 11041 NebulaStore.cpp:145] Load part 252, 65 from disk
I20221014 12:58:42.648543 11042 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 67] lastLogId in wal is 48288, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/67/0000000000000031876.wal
I20221014 12:58:42.648671 11042 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 67] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48288, lastLogTerm 11, committedLogId 48288, committedLogTerm 11, term 11
I20221014 12:58:42.648728 11042 NebulaStore.cpp:145] Load part 252, 67 from disk
I20221014 12:58:42.676599 11041 FileBasedWal.cpp:66] [Port: 9780, Space: 252, Part: 73] lastLogId in wal is 48286, lastLogTerm is 11, path is /usr/local/nebula-graph-3.0.1.el7.x86_64/data/storage/nebula/252/wal/73/0000000000000031875.wal
I20221014 12:58:42.676699 11041 RaftPart.cpp:299] [Port: 9780, Space: 252, Part: 73] There are 0 peer hosts, and total 1 copies. The quorum is 1, as learner 0, lastLogId 48286, lastLogTerm 11, committedLogId 48286, committedLogTerm 11, term 11
I20221014 12:58:42.676752 11041 NebulaStore.cpp:145] Load part 252, 73 from disk

报错日志

Log file created at: 2022/10/15 11:03:02
Running on machine: ZH07-TXN-20126
Running duration (h:mm:ss): 0:00:16
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
E20221015 11:03:02.703413 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 14:06:50.853341 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 14:07:03.887933 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 14:21:01.273442 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 14:54:02.572485 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 15:03:10.788877 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 15:04:12.903571 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 16:12:05.364883 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 16:46:17.993677 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 16:46:31.025183 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 17:21:50.655273 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 17:53:51.596510 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 18:08:00.850071 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 19:53:50.661836 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 20:27:09.011018 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 20:49:12.205754 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221015 22:49:27.855502 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 00:36:32.423722 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 02:19:54.166790 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 02:20:07.197805 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 02:36:11.001114 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 02:54:55.713732 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 02:55:08.744925 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 03:15:41.084328 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 03:15:54.113514 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 03:16:07.146996 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 03:32:15.242765 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 04:03:19.911015 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 08:11:09.519274 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 09:22:16.874522 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 09:27:51.651263 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!
E20221016 12:30:23.238937 11040 MetaClient.cpp:171] Heartbeat failed, status:Machine not existed!

SuperYoko · 2022 年10 月 17 日 02:39

add hosts了么？

而且为啥要用两个主机来，可以看看部署方面的建议。

jingnny · 2022 年10 月 18 日 00:50

1、因为想测试集群部署模式。那只有2台机器的情况下，meta，graph，storage有相关的建议吗？发现了一个现象是storage内存一直在慢慢添加，不能释放，从50%多2天后到90%了，机器内存24G

jingnny · 2022 年10 月 18 日 00:52

“add hosts了么？” 加了，注意刚刚启动是正常的，1天内都能正常访问，大概2天左右或者1天左右就自动退出了。前面提到的现象还有storage的内存一直在慢慢添加，不能释放

SuperYoko · 2022 年10 月 18 日 02:20

能带一下版本吗？这样我好确定是以前版本的已修复问题还是新的问题。
关于后面的问题，你的ip有改动吗？一般这个Machine not existed!错误是检查机器是否已经作为hosts add到集群中，而这个签名是以 local ip+port作为签名的，你可以检查一下。

另外建议如果多节点的话至少是三节点部署哈~

jingnny · 2022 年10 月 18 日 09:34

版本：nebula-graph-3.0.1.el7.x86_64

关于后面的问题，你的ip有改动吗？一般这个Machine not existed!错误是检查机器是否已经作为hosts add到集群中，而这个签名是以 local ip+port作为签名的，你可以检查一下。
已经加了，问题原因应该就是内存的问题，一直慢慢增加，到100%导致进程自动退出了。就报这个错误

另外建议如果多节点的话至少是三节点部署哈~
如果现在有数据的情况下，我能切回单节点吗？还是说2个节点完全不能集群，用一个meta，两个storage，两个graph不行？

SuperYoko · 2022 年10 月 18 日 09:59

graph倒是随便，因为没有同步问题，
storage/meta同步机制依赖raft，为什么不建议用两个节点与其相关。
内存问题可以换用新的版本，后续有解决。
可以看看历史问题

jingnny · 2022 年10 月 19 日 06:38

内存问题可以换用新的版本，后续有解决。用哪个版本呢？

SuperYoko · 2022 年10 月 19 日 06:52

一个是可以看是否是我后续贴的问题导致的
如果想要升级3.1以后应该是ok的~个人建议肯定是最新版， 3.x版本当前主要在提升稳定性，理论上越靠后的版本会越稳定。

jingnny · 2022 年10 月 23 日 09:32

这个是什么文件？一直在增长，慢慢把磁盘占满了

SuperYoko · 2022 年10 月 23 日 11:39

这是coredump，程序崩溃时的内存转储。

system · 2022 年11 月 22 日 11:39

此话题已在最后回复的 30 天后被自动关闭。不再允许新回复。