-
nebula 版本:2.0GA
-
部署方式:分布式 3台 storage 1台graph
-
硬件信息
- 磁盘: SSD
- CPU、内存信息
-
问题的具体描述
数据导入后创建索引,手动compact和rebuild index 已完成。
先执行一个不指定属性的查询,出现E_RPC_FAILURE,查看其它帖子是由于加载全部数据查询超时导致。
再执行一个指定属性的查询,出现The leader has changed. Try again later
show hosts 查看节点状态并没有发生变化。 -
相关的 meta / storage / graph info 日志信息
graph info日志
E0331 16:43:20.931025 215814 StorageClientBase.inl:209] Request to "10.100.2.244":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
I0331 16:43:20.936769 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 13]
I0331 16:43:20.936801 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 10]
I0331 16:43:20.936823 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 51]
I0331 16:43:20.936844 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 35]
I0331 16:43:20.936866 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 44]
I0331 16:43:20.936885 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 91]
I0331 16:43:20.936980 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 67]
I0331 16:43:20.937008 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 79]
I0331 16:43:20.937029 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 32]
I0331 16:43:20.937048 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 42]
I0331 16:43:20.937069 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 89]
I0331 16:43:20.937090 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 80]
I0331 16:43:20.937110 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 34]
I0331 16:43:20.937130 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 83]
I0331 16:43:20.937150 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 76]
I0331 16:43:20.937170 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 99]
I0331 16:43:20.937191 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 5]
I0331 16:43:20.937211 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 45]
I0331 16:43:20.937232 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 22]
I0331 16:43:20.937252 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 63]
I0331 16:43:20.937271 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 16]
I0331 16:43:20.937292 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 96]
I0331 16:43:20.937312 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 49]
I0331 16:43:20.937332 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 47]
I0331 16:43:20.937352 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 94]
I0331 16:43:20.937373 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 95]
I0331 16:43:20.937393 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 48]
I0331 16:43:20.937413 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 77]
I0331 16:43:20.937433 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 100]
I0331 16:43:20.937454 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 72]
I0331 16:43:20.937474 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 43]
I0331 16:43:20.937494 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 90]
I0331 16:43:20.937513 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 19]
I0331 16:43:20.937534 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 58]
I0331 16:43:20.937554 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 11]
I0331 16:43:20.937574 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 62]
I0331 16:43:20.937593 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 84]
I0331 16:43:20.937613 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 70]
I0331 16:43:20.937633 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 39]
I0331 16:43:20.937654 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 71]
I0331 16:43:20.937674 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 26]
I0331 16:43:20.937692 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 31]
I0331 16:43:20.937712 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 87]
I0331 16:43:20.937731 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 97]
I0331 16:43:20.937752 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 56]
I0331 16:43:20.937772 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 1]
I0331 16:43:20.937791 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 98]
I0331 16:43:20.937811 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 38]
I0331 16:43:20.937831 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 68]
I0331 16:43:20.937850 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 12]
I0331 16:43:20.937870 215814 StorageClientBase.inl:161] Invalidate the leader for [340, 8]
E0331 16:43:21.081640 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 8
E0331 16:43:21.081836 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 12
E0331 16:43:21.081889 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 68
E0331 16:43:21.081957 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 1
E0331 16:43:21.082011 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 56
E0331 16:43:21.082060 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 71
E0331 16:43:21.082108 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 39
E0331 16:43:21.082156 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 70
E0331 16:43:21.082206 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 31
E0331 16:43:21.082253 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 84
E0331 16:43:21.082301 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 62
E0331 16:43:21.082350 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 11
E0331 16:43:21.082398 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 90
E0331 16:43:21.082446 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 19
E0331 16:43:21.082494 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 72
E0331 16:43:21.082542 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 77
E0331 16:43:21.082590 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 48
E0331 16:43:21.082639 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 94
E0331 16:43:21.082687 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 100
E0331 16:43:21.082736 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 47
E0331 16:43:21.082783 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 49
E0331 16:43:21.082831 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 43
E0331 16:43:21.082950 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 96
E0331 16:43:21.083014 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 16
E0331 16:43:21.083062 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 22
E0331 16:43:21.083110 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 98
E0331 16:43:21.083158 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 45
E0331 16:43:21.083204 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 58
E0331 16:43:21.083251 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 5
E0331 16:43:21.083298 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 99
E0331 16:43:21.083345 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 76
E0331 16:43:21.083392 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 83
E0331 16:43:21.083439 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 87
E0331 16:43:21.083487 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 34
E0331 16:43:21.083534 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 80
E0331 16:43:21.083581 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 89
E0331 16:43:21.083678 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 95
E0331 16:43:21.083729 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 42
E0331 16:43:21.083777 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 32
E0331 16:43:21.083825 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E0331 16:43:21.083874 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 79
E0331 16:43:21.083922 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 67
E0331 16:43:21.083982 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 38
E0331 16:43:21.084033 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 91
E0331 16:43:21.084080 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 97
E0331 16:43:21.084128 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 44
E0331 16:43:21.084177 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 35
E0331 16:43:21.084225 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 51
E0331 16:43:21.084275 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 63
E0331 16:43:21.084322 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 10
E0331 16:43:21.084370 215731 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 13
E0331 16:43:21.084426 215731 StorageAccessExecutor.h:112] Storage Error: part: 8, error: E_RPC_FAILURE(-3).
E0331 16:43:21.084707 215774 QueryInstance.cpp:103] Storage Error: part: 8, error: E_RPC_FAILURE(-3).
I0331 16:43:33.974162 215822 StorageClientBase.inl:142] Update the leader for [340, 13] to "10.100.2.244":9779
I0331 16:43:33.974392 215822 StorageClientBase.inl:142] Update the leader for [340, 19] to "10.100.2.244":9779
I0331 16:43:33.974426 215822 StorageClientBase.inl:142] Update the leader for [340, 22] to "10.100.2.244":9779
I0331 16:43:33.974448 215822 StorageClientBase.inl:142] Update the leader for [340, 31] to "10.100.2.244":9779
I0331 16:43:33.974491 215822 StorageClientBase.inl:142] Update the leader for [340, 34] to "10.100.2.244":9779
I0331 16:43:33.974522 215822 StorageClientBase.inl:142] Update the leader for [340, 38] to "10.100.2.244":9779
I0331 16:43:33.974545 215822 StorageClientBase.inl:142] Update the leader for [340, 45] to "10.100.2.244":9779
I0331 16:43:33.974567 215822 StorageClientBase.inl:142] Update the leader for [340, 51] to "10.100.2.244":9779
I0331 16:43:33.974589 215822 StorageClientBase.inl:142] Update the leader for [340, 58] to "10.100.2.244":9779
I0331 16:43:33.974611 215822 StorageClientBase.inl:142] Update the leader for [340, 62] to "10.100.2.244":9779
I0331 16:43:33.974633 215822 StorageClientBase.inl:142] Update the leader for [340, 77] to "10.100.2.244":9779
I0331 16:43:33.974656 215822 StorageClientBase.inl:142] Update the leader for [340, 80] to "10.100.2.244":9779
I0331 16:43:33.974679 215822 StorageClientBase.inl:142] Update the leader for [340, 83] to "10.100.2.244":9779
I0331 16:43:33.974700 215822 StorageClientBase.inl:142] Update the leader for [340, 95] to "10.100.2.244":9779
I0331 16:43:33.974722 215822 StorageClientBase.inl:142] Update the leader for [340, 96] to "10.100.2.244":9779
I0331 16:43:33.974745 215822 StorageClientBase.inl:142] Update the leader for [340, 97] to "10.100.2.244":9779
I0331 16:43:33.974766 215822 StorageClientBase.inl:142] Update the leader for [340, 100] to "10.100.2.244":9779
I0331 16:43:33.982084 215823 StorageClientBase.inl:142] Update the leader for [340, 5] to "10.100.2.244":9779
I0331 16:43:33.982149 215823 StorageClientBase.inl:142] Update the leader for [340, 11] to "10.100.2.244":9779
I0331 16:43:33.982228 215823 StorageClientBase.inl:142] Update the leader for [340, 12] to "10.100.2.244":9779
I0331 16:43:33.982270 215823 StorageClientBase.inl:142] Update the leader for [340, 26] to "10.100.2.244":9779
I0331 16:43:33.982296 215823 StorageClientBase.inl:142] Update the leader for [340, 35] to "10.100.2.244":9779
I0331 16:43:33.982319 215823 StorageClientBase.inl:142] Update the leader for [340, 39] to "10.100.2.244":9779
I0331 16:43:33.982344 215823 StorageClientBase.inl:142] Update the leader for [340, 42] to "10.100.2.244":9779
I0331 16:43:33.982365 215823 StorageClientBase.inl:142] Update the leader for [340, 43] to "10.100.2.244":9779
I0331 16:43:33.982388 215823 StorageClientBase.inl:142] Update the leader for [340, 44] to "10.100.2.244":9779
I0331 16:43:33.982409 215823 StorageClientBase.inl:142] Update the leader for [340, 47] to "10.100.2.244":9779
I0331 16:43:33.982430 215823 StorageClientBase.inl:142] Update the leader for [340, 48] to "10.100.2.244":9779
I0331 16:43:33.982451 215823 StorageClientBase.inl:142] Update the leader for [340, 56] to "10.100.2.244":9779
I0331 16:43:33.982473 215823 StorageClientBase.inl:142] Update the leader for [340, 63] to "10.100.2.244":9779
I0331 16:43:33.982494 215823 StorageClientBase.inl:142] Update the leader for [340, 67] to "10.100.2.244":9779
I0331 16:43:33.982515 215823 StorageClientBase.inl:142] Update the leader for [340, 71] to "10.100.2.244":9779
I0331 16:43:33.982537 215823 StorageClientBase.inl:142] Update the leader for [340, 72] to "10.100.2.244":9779
I0331 16:43:33.982558 215823 StorageClientBase.inl:142] Update the leader for [340, 79] to "10.100.2.244":9779
I0331 16:43:33.982579 215823 StorageClientBase.inl:142] Update the leader for [340, 84] to "10.100.2.244":9779
I0331 16:43:33.982626 215823 StorageClientBase.inl:142] Update the leader for [340, 99] to "10.100.2.244":9779
E0331 16:43:33.983220 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 84
E0331 16:43:33.983319 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 79
E0331 16:43:33.983350 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 72
E0331 16:43:33.983381 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 71
E0331 16:43:33.983409 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 67
E0331 16:43:33.983494 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 63
E0331 16:43:33.983530 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 56
E0331 16:43:33.983558 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 47
E0331 16:43:33.983587 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 44
E0331 16:43:33.983614 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 43
E0331 16:43:33.983642 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 42
E0331 16:43:33.983670 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 12
E0331 16:43:33.983698 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 35
E0331 16:43:33.983726 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 62
E0331 16:43:33.983754 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 39
E0331 16:43:33.983799 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 51
E0331 16:43:33.983829 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 97
E0331 16:43:33.983856 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 99
E0331 16:43:33.983883 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 5
E0331 16:43:33.983952 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 38
E0331 16:43:33.983985 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 19
E0331 16:43:33.984012 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 45
E0331 16:43:33.984040 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 22
E0331 16:43:33.984067 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 34
E0331 16:43:33.984094 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 80
E0331 16:43:33.984122 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 58
E0331 16:43:33.984149 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 11
E0331 16:43:33.984176 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 13
E0331 16:43:33.984205 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 77
E0331 16:43:33.984231 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 31
E0331 16:43:33.984258 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 100
E0331 16:43:33.984287 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 83
E0331 16:43:33.984313 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 48
E0331 16:43:33.984341 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 95
E0331 16:43:33.984369 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 26
E0331 16:43:33.984395 215731 StorageAccessExecutor.h:35] IndexScanExecutor failed, error E_LEADER_CHANGED, part 96
E0331 16:43:33.984501 215731 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
storage info/error 没有打印日志
为什么不指定属性会加载全部的数据再limit结果?
超时为什么会导致leader change?