- nebula 版本：3.6
- 部署方式：分布式（graphd/metad/storaged均采用分布式容器化部署，且均为3实例）
- 安装方式：源码编译
- 是否上生产环境：Y
- 硬件信息
- 问题的具体描述
- 背景：nebula-storgaed
故障节点的storaged日志如下（开启最高日志权限v=5,trace_raft=1,minloglevel=0）：
Log file created at: 2026/02/24 17:27:37
Running on machine: infinitygraph-storaged-2
Running duration (h:mm:ss): 3:16:36
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
I20260224 17:27:37.612732 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.612967 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143165 to 143189
I20260224 17:27:37.612978 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143165 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.613008 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.613018 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.613025 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143162 to 143189
I20260224 17:27:37.613032 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143162 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.613054 1964 CollectNSucceeded-inl.h:59] Set Value [completed=2, total=2, Result list size=2]
I20260224 17:27:37.613077 1876 RaftPart.cpp:984] [Port: 26747, Space: 1, Part: 19] Received enough response
I20260224 17:27:37.613111 2353 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.613131 2353 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.613139 2353 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.613155 2353 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.615192 2354 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.615229 2354 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.615239 2354 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.615259 2354 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.622951 1876 RaftPart.cpp:946] [Port: 26747, Space: 1, Part: 19] About to replicate logs in range [143189, 143189] to all peer hosts
I20260224 17:27:37.623003 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.623045 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.623081 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.623124 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.623133 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143165 to 143189
I20260224 17:27:37.623140 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143165 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.623185 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.623194 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.623201 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143162 to 143189
I20260224 17:27:37.623208 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143162 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.623240 1964 CollectNSucceeded-inl.h:59] Set Value [completed=2, total=2, Result list size=2]
I20260224 17:27:37.623246 2355 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.623303 2355 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.623293 2352 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.623327 2355 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.623376 2352 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.623411 2352 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.623412 2355 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.623494 2352 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.623713 1876 RaftPart.cpp:984] [Port: 26747, Space: 1, Part: 19] Received enough response
I20260224 17:27:37.624831 1876 RaftPart.cpp:946] [Port: 26747, Space: 1, Part: 19] About to replicate logs in range [143189, 143189] to all peer hosts
I20260224 17:27:37.624866 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.624894 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.624929 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.624977 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.624989 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143165 to 143189
I20260224 17:27:37.624996 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143165 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.625028 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.625038 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.625046 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143162 to 143189
I20260224 17:27:37.625052 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143162 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.625077 1964 CollectNSucceeded-inl.h:59] Set Value [completed=2, total=2, Result list size=2]
I20260224 17:27:37.625140 2353 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.625180 2353 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.625190 2353 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.625209 2353 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.625394 1876 RaftPart.cpp:984] [Port: 26747, Space: 1, Part: 19] Received enough response
I20260224 17:27:37.625662 2354 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.625715 2354 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.625730 2354 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.625752 2354 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.626507 1876 RaftPart.cpp:946] [Port: 26747, Space: 1, Part: 19] About to replicate logs in range [143189, 143189] to all peer hosts
I20260224 17:27:37.626566 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.626596 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.626621 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.626664 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.626675 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143165 to 143189
I20260224 17:27:37.626683 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143165 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.626717 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.626731 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.626739 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143162 to 143189
I20260224 17:27:37.626745 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143162 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.626750 2355 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.626770 1964 CollectNSucceeded-inl.h:59] Set Value [completed=2, total=2, Result list size=2]
I20260224 17:27:37.626804 2355 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.626819 2355 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.626827 2352 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.626847 2355 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.626893 2352 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.626928 2352 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.626861 1876 RaftPart.cpp:984] [Port: 26747, Space: 1, Part: 19] Received enough response
I20260224 17:27:37.626955 2352 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.628069 1876 RaftPart.cpp:946] [Port: 26747, Space: 1, Part: 19] About to replicate logs in range [143189, 143189] to all peer hosts
I20260224 17:27:37.628118 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.628149 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.628165 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.628206 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.628216 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143165 to 143189
I20260224 17:27:37.628224 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143165 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.628250 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.628259 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.628266 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143162 to 143189
I20260224 17:27:37.628273 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143162 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.628294 1964 CollectNSucceeded-inl.h:59] Set Value [completed=2, total=2, Result list size=2]
I20260224 17:27:37.628286 2353 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.628326 1876 RaftPart.cpp:984] [Port: 26747, Space: 1, Part: 19] Received enough response
I20260224 17:27:37.628332 2354 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.628337 2353 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.628417 2353 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.628396 2354 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.628436 2354 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.628440 2353 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.628474 2354 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.629421 1876 RaftPart.cpp:946] [Port: 26747, Space: 1, Part: 19] About to replicate logs in range [143189, 143189] to all peer hosts
I20260224 17:27:37.629446 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.629467 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.629475 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.629505 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.629515 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143165 to 143189
I20260224 17:27:37.629523 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143165 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.629554 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.629563 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.629571 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143162 to 143189
I20260224 17:27:37.629577 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143162 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.629591 2355 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.629601 1964 CollectNSucceeded-inl.h:59] Set Value [completed=2, total=2, Result list size=2]
I20260224 17:27:37.629623 2355 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.629634 2355 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.629638 2352 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.629652 2355 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.629686 2352 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.629698 1876 RaftPart.cpp:984] [Port: 26747, Space: 1, Part: 19] Received enough response
I20260224 17:27:37.629725 2352 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.629762 2352 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.630812 1876 RaftPart.cpp:946] [Port: 26747, Space: 1, Part: 19] About to replicate logs in range [143189, 143189] to all peer hosts
I20260224 17:27:37.630856 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.630887 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.630897 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.630949 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.630960 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143165 to 143189
I20260224 17:27:37.630968 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143165 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.630997 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.631006 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.631013 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143162 to 143189
I20260224 17:27:37.631021 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143162 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.631036 1964 CollectNSucceeded-inl.h:59] Set Value [completed=2, total=2, Result list size=2]
I20260224 17:27:37.631085 2353 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.631121 2353 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.631145 2353 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.631165 2353 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.631073 2354 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.631074 1876 RaftPart.cpp:984] [Port: 26747, Space: 1, Part: 19] Received enough response
I20260224 17:27:37.631204 2354 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.631244 2354 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.631261 2354 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.632324 1876 RaftPart.cpp:946] [Port: 26747, Space: 1, Part: 19] About to replicate logs in range [143189, 143189] to all peer hosts
I20260224 17:27:37.632364 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.632391 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.632406 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.632427 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.632447 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143165 to 143189
I20260224 17:27:37.632454 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143165 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.632485 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.632503 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.632511 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143162 to 143189
I20260224 17:27:37.632519 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143162 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.632522 2355 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.632547 2355 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.632556 2355 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.632539 1964 CollectNSucceeded-inl.h:59] Set Value [completed=2, total=2, Result list size=2]
I20260224 17:27:37.632591 2355 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.632620 2352 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.632661 2352 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.632629 1876 RaftPart.cpp:984] [Port: 26747, Space: 1, Part: 19] Received enough response
I20260224 17:27:37.632678 2352 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.632736 2352 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.633790 1876 RaftPart.cpp:946] [Port: 26747, Space: 1, Part: 19] About to replicate logs in range [143189, 143189] to all peer hosts
I20260224 17:27:37.633831 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.633858 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.633873 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.633891 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.633919 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143165 to 143189
I20260224 17:27:37.633927 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143165 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.633970 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.633989 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.633996 2353 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.633997 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143162 to 143189
I20260224 17:27:37.634052 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143162 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.634052 2353 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.634078 2353 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.634105 2353 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.634078 1964 CollectNSucceeded-inl.h:59] Set Value [completed=2, total=2, Result list size=2]
I20260224 17:27:37.634157 1876 RaftPart.cpp:984] [Port: 26747, Space: 1, Part: 19] Received enough response
I20260224 17:27:37.634109 2354 NebulaSnapshotManager.cpp:67] Space 1 Part 19 start send snapshot of commitLogId 143166 commitLogTerm 3, rate limited to 10485760, batch size is 1048576
I20260224 17:27:37.634189 2354 NebulaSnapshotManager.cpp:113] [spaceId:1, partId:19] access prefix failed, error code:-4
I20260224 17:27:37.634197 2354 SnapshotManager.cpp:58] [Port: 26747, Space: 1, Part: 19] Snapshot send failed, the leader changed?
I20260224 17:27:37.634214 2354 Host.cpp:369] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Send snapshot failed!
I20260224 17:27:37.635250 1876 RaftPart.cpp:946] [Port: 26747, Space: 1, Part: 19] About to replicate logs in range [143189, 143189] to all peer hosts
I20260224 17:27:37.635277 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.635301 1876 RaftPart.cpp:959] [Port: 26747, Space: 1, Part: 19] Appending logs to [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747]
I20260224 17:27:37.635330 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.635362 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request
I20260224 17:27:37.635388 1964 Host.cpp:290] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Prepare AppendLogs request from Log 143165 to 143189
I20260224 17:27:37.635397 1964 Host.cpp:351] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-0.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Can't find log 143165 in wal, send the snapshot, logIdToSend = 143189, firstLogId in wal = 143167, lastLogId in wal = 143189
I20260224 17:27:37.635439 1964 Host.cpp:98] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] Entering Host::appendLogs()
I20260224 17:27:37.635452 1964 Host.cpp:128] [Port: 26747, Space: 1, Part: 19] [Host: infinitygraph-storaged-1.infinitygraph-storaged-headless.kube-system.svc.cluster.local:26747] About to send the AppendLog request