graphd进程崩溃(非oom)

现在已经定位到是find noloop path upto 10 steps 语句的导致的,此时机器内存使用不多。

麻烦把 这个core文件 帧栈的前30层 发一下

这个可以复现吗

上面那个coredump文件调试信息中间加了省略号表示#5 - #1611结果都是一样的。

稍等下,我现在把昨天都退出过的graphd 节点调试信息整理一下发出来。

有没有完整的语句看下,用了哪些表达式

find noloop path from -2886766351330904521 to 8372371496602176544 over Invest upto 10 steps

coredump_debug.log (35.1 KB)
里面有两类错误信息

1 个赞

看着是 go 语句报的错, 帮忙看看这个时间点。把nebula-graphd.ERROR 和 graphd-stderr.log 贴一下

graphd节点重启时间点

    Line 10: 2021-09-07 16:08:01,hostname: nebula-server-1,Nebula Role: graphd, Status: 重启成功.
	Line 17: 2021-09-07 16:25:01,hostname: nebula-server-1,Nebula Role: graphd, Status: 重启成功.
	Line 22: 2021-09-07 16:38:01,hostname: nebula-server-1,Nebula Role: graphd, Status: 重启成功.
	Line 28: 2021-09-07 17:03:01,hostname: nebula-server-1,Nebula Role: graphd, Status: 重启成功.
	Line 42: 2021-09-07 17:30:01,hostname: nebula-server-1,Nebula Role: graphd, Status: 重启成功.
	Line 73: 2021-09-07 21:56:01,hostname: nebula-server-1,Nebula Role: graphd, Status: 重启成功.

	Line 6: 2021-09-07 14:48:01, hostname:nebula-server-2,Nebula Role: graphd. Status: 重启成功.
	Line 34: 2021-09-07 17:12:01,hostname:nebula-server-2,Nebula Role: graphd. Status: 重启成功.
	Line 53: 2021-09-07 18:02:01,hostname:nebula-server-2,Nebula Role: graphd. Status: 重启成功.
	Line 61: 2021-09-07 19:12:01,hostname:nebula-server-2,Nebula Role: graphd. Status: 重启成功.
	Line 71: 2021-09-07 21:27:01,hostname:nebula-server-2,Nebula Role: graphd. Status: 重启成功.

	Line 1: 2021-09-07 14:33:01 hostname: nebula-server-3,Nebula Role: graphd. Status: 重启成功.
	Line 8: 2021-09-07 15:27:01 hostname: nebula-server-3,Nebula Role: graphd. Status: 重启成功.
	Line 14: 2021-09-07 16:23:01 hostname: nebula-server-3,Nebula Role: graphd. Status: 重启成功.
	Line 32: 2021-09-07 17:08:01 hostname: nebula-server-3,Nebula Role: graphd. Status: 重启成功.
	Line 46: 2021-09-07 17:32:01 hostname: nebula-server-3,Nebula Role: graphd. Status: 重启成功.
	Line 48: 2021-09-07 17:38:01 hostname: nebula-server-3,Nebula Role: graphd. Status: 重启成功.
	Line 57: 2021-09-07 18:32:01 hostname: nebula-server-3,Nebula Role: graphd. Status: 重启成功.
	
	Line 12: 2021-09-07 16:22:01 hostname: nebula-server-4,Nebula Role: graphd, Status: 重启成功.
	Line 25: 2021-09-07 16:56:01 hostname: nebula-server-4,Nebula Role: graphd, Status: 重启成功.
	Line 44: 2021-09-07 17:31:01 hostname: nebula-server-4,Nebula Role: graphd, Status: 重启成功.
	Line 52: 2021-09-07 18:01:01 hostname: nebula-server-4,Nebula Role: graphd, Status: 重启成功.
	Line 54: 2021-09-07 18:05:01 hostname: nebula-server-4,Nebula Role: graphd, Status: 重启成功.
	Line 63: 2021-09-07 19:23:01 hostname: nebula-server-4,Nebula Role: graphd, Status: 重启成功.
	
	
	Line 37: 2021-09-07 17:17:01 hostname:nebula-server-5,Nebula Role: graphd. Status: 重启成功.
	Line 50: 2021-09-07 17:55:01 hostname:nebula-server-5,Nebula Role: graphd. Status: 重启成功.
	Line 69: 2021-09-07 21:27:01 hostname:nebula-server-5,Nebula Role: graphd. Status: 重启成功.
	
	Line 4: 2021-09-07 14:33:01 Hostname: nebula-server-6,Nebula Role: graphd, Status: 重启成功.
	Line 19: 2021-09-07 16:27:01 hostname: nebula-server-6,Nebula Role: graphd, Status: 重启成功.
	Line 39: 2021-09-07 17:19:01 hostname: nebula-server-6,Nebula Role: graphd, Status: 重启成功.
	Line 56: 2021-09-07 18:13:01 hostname: nebula-server-6,Nebula Role: graphd, Status: 重启成功.
	Line 58: 2021-09-07 18:40:01 hostname: nebula-server-6,Nebula Role: graphd, Status: 重启成功.
	Line 66: 2021-09-07 21:27:01 hostname: nebula-server-6,Nebula Role: graphd, Status: 重启成功.

graphd-stderr.log 信息 nebula-server-1, nebula-server-2, nebula-server-3

================================nebula-server-1=========================
E0907 14:31:11.816169 11408 GraphService.cpp:81] Session not found, id[22460018]
E0907 15:32:40.260087 11410 GraphService.cpp:81] Session not found, id[200898]
E0907 15:54:02.472220 11420 GraphService.cpp:81] Session not found, id[4380112]
E0907 16:08:02.377914  7429 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 28
E0907 16:08:02.378125  7429 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 5
E0907 16:08:02.378136  7429 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 4
E0907 16:08:02.378145  7429 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 27
E0907 16:08:02.378155  7429 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 16
E0907 16:08:02.378165  7429 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 23
E0907 16:08:02.378173  7429 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 17
E0907 16:08:02.378181  7429 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 6
E0907 16:08:02.378281  7429 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 7
E0907 16:08:02.378298  7429 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 10
E0907 16:08:02.378310  7429 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 11
E0907 16:08:02.378322  7429 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 25
E0907 16:08:02.378388  7427 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 16:20:36.477999  7422 GraphService.cpp:81] Session not found, id[6792733]
E0907 16:21:18.563454  7420 GraphService.cpp:81] Session not found, id[344475]
E0907 16:21:18.706684  7419 GraphService.cpp:81] Session not found, id[344451]
E0907 16:21:18.753572  7427 GraphService.cpp:81] Session not found, id[344463]
E0907 16:21:19.193536  7425 GraphService.cpp:81] Session not found, id[344455]
E0907 16:21:19.408237  7420 GraphService.cpp:81] Session not found, id[344468]
E0907 16:21:20.789549  7415 GraphService.cpp:81] Session not found, id[344473]
E0907 16:25:27.418139  9283 GraphService.cpp:81] Session not found, id[847682]
E0907 16:25:27.476275  9281 GraphService.cpp:81] Session not found, id[847690]
E0907 16:51:36.420161 10637 GraphService.cpp:81] Session not found, id[638694]
E0907 17:03:03.263010 13090 GraphService.cpp:81] Session not found, id[64]
E0907 17:30:01.366215 15673 GraphService.cpp:81] Session not found, id[91]
E0907 17:30:01.883803 15664 QueryInstance.cpp:103] EdgeName `Invest'  is nonexistent
E0907 17:45:32.571168 15674 GraphService.cpp:81] Session not found, id[27766]
E0907 21:25:25.179664 15673 GraphService.cpp:81] Session not found, id[433317]
E0907 21:25:33.363988 15670 GraphService.cpp:81] Session not found, id[495502]
E0907 21:25:33.728281 15667 GraphService.cpp:81] Session not found, id[437834]
E0907 21:25:33.861763 15667 GraphService.cpp:81] Session not found, id[433452]
E0907 21:25:44.733098 15643 StorageClientBase.inl:214] Request to "10.0.7.140":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:44.779111 15662 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 13
E0907 21:25:44.789105 15662 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 5
E0907 21:25:44.792143 15662 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 10
E0907 21:25:44.794955 15662 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 7
E0907 21:25:44.796483 15662 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 11
E0907 21:25:44.802773 15662 StorageAccessExecutor.h:112] Storage Error: part: 13, error: E_RPC_FAILURE(-3).
E0907 21:25:44.807971 15663 QueryInstance.cpp:103] Storage Error: part: 13, error: E_RPC_FAILURE(-3).
E0907 21:25:45.005359 15661 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 10
E0907 21:25:45.006867 15661 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:25:46.820344 15670 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:25:46.821107 15670 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:25:46.822065 15670 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:25:46.823076 15670 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:25:46.829576 15667 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:25:46.913975 15668 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:25:46.915349 15668 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:25:46.917089 15668 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:25:46.920079 15668 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:25:46.929139 15669 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:12.362610 15675 GraphService.cpp:81] Session not found, id[995610]
E0907 21:26:12.375145 15675 GraphService.cpp:81] Session not found, id[438455]
E0907 21:26:12.414742 15669 GraphService.cpp:81] Session not found, id[995636]
E0907 21:26:13.513455 15665 GraphService.cpp:81] Session not found, id[995682]
E0907 21:26:32.235096 15669 GraphService.cpp:81] Session not found, id[434650]
E0907 21:26:32.251950 15674 GraphService.cpp:81] Session not found, id[434655]
E0907 21:26:34.286554 15673 GraphService.cpp:81] Session not found, id[434661]
E0907 21:26:34.422282 15667 GraphService.cpp:81] Session not found, id[434652]
E0907 21:54:53.822078 15645 StorageClientBase.inl:214] Request to "10.0.7.140":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:54:53.819839 15642 StorageClientBase.inl:214] Request to "10.0.7.141":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:54:53.821239 15644 StorageClientBase.inl:214] Request to "10.0.7.252":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:54:54.071358 15664 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 16
E0907 21:54:54.077075 15664 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 26
E0907 21:54:54.085163 15664 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 4
E0907 21:54:54.094089 15668 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:54.141309 15671 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:54:54.147140 15671 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 12
E0907 21:54:54.152870 15671 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:54.211027 15674 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 23
E0907 21:54:54.219128 15675 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:54.319943 15662 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:54:54.332098 15673 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:55.014166 15667 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 10
E0907 21:54:55.017069 15667 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:54:55.020093 15667 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:54:55.023067 15667 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:54:55.028090 15667 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:55.019402 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E0907 21:54:55.052078 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 16
E0907 21:54:55.062361 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 23
E0907 21:54:55.074079 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 7
E0907 21:54:55.094224 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 5
E0907 21:54:55.098371 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 10
E0907 21:54:55.115025 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 28
E0907 21:54:55.128691 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 11
E0907 21:54:55.137255 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 13
E0907 21:54:55.145782 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 2
E0907 21:54:55.149634 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 21
E0907 21:54:55.174080 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 4
E0907 21:54:55.195899 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 12
E0907 21:54:55.203083 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 1
E0907 21:54:55.217988 15665 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 3
E0907 21:54:55.228498 15665 StorageAccessExecutor.h:112] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E0907 21:54:55.299093 15661 QueryInstance.cpp:103] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E0907 21:55:00.553994 15660 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:55:00.572132 15675 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:03.764753 15662 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 3
E0907 21:55:03.773953 15662 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 1
E0907 21:55:03.784178 15662 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:04.657287 15664 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 2
E0907 21:55:04.674955 15664 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:09.568686 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 23
E0907 21:55:09.584218 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 21
E0907 21:55:09.593075 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 26
E0907 21:55:09.597306 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 3
E0907 21:55:09.621407 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 12
E0907 21:55:09.625990 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 1
E0907 21:55:09.642190 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 7
E0907 21:55:09.649152 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 11
E0907 21:55:09.654079 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 13
E0907 21:55:09.665899 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 2
E0907 21:55:09.670156 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 10
E0907 21:55:09.680075 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 4
E0907 21:55:09.698076 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 28
E0907 21:55:09.720098 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 5
E0907 21:55:09.740078 15665 StorageAccessExecutor.h:35] pneO
                                                           __Pr failed, error E_LEADER_CHANGED, part 16
E0907 23:31:58.885834  6868 GraphService.cpp:81] Session not found, id[858318]
================================nebula-server-2=========================
E0907 13:30:09.075769 23322 GraphService.cpp:81] Session not found, id[5619628]
E0907 13:52:42.949167 23333 GraphService.cpp:81] Session not found, id[43615042]
E0907 15:25:17.932757   398 GraphService.cpp:81] Session not found, id[335050]
E0907 16:20:37.548913   397 GraphService.cpp:81] Session not found, id[6792725]
E0907 16:21:18.609414   404 GraphService.cpp:81] Session not found, id[344471]
E0907 16:21:18.629525   397 GraphService.cpp:81] Session not found, id[344474]
E0907 16:21:18.719725   404 GraphService.cpp:81] Session not found, id[344464]
E0907 16:21:18.725920   411 GraphService.cpp:81] Session not found, id[344445]
E0907 17:06:32.551252   410 GraphService.cpp:81] Session not found, id[274604]
E0907 17:12:01.388185 13998 GraphService.cpp:81] Session not found, id[63]
E0907 17:12:01.415726 13997 QueryInstance.cpp:103] EdgeName `Invest'  is nonexistent
E0907 17:12:02.156231 13999 GraphService.cpp:81] Session not found, id[104]
E0907 17:15:14.027144 13999 GraphService.cpp:81] Session not found, id[4933694]
E0907 17:15:15.065169 13992 GraphService.cpp:81] Session not found, id[4933526]
E0907 17:28:58.257220 13991 GraphService.cpp:81] Session not found, id[136447]
E0907 17:36:57.832662 13998 GraphService.cpp:81] Session not found, id[13975]
E0907 17:45:32.573750 13999 GraphService.cpp:81] Session not found, id[170787]
E0907 17:53:23.569126 13990 GraphService.cpp:81] Session not found, id[214979]
E0907 18:04:34.762408 18793 GraphService.cpp:81] Session not found, id[13530]
E0907 18:04:34.795675 18793 GraphService.cpp:81] Session not found, id[13529]
E0907 18:12:21.996568 18787 GraphService.cpp:81] Session not found, id[259361]
E0907 19:11:02.321552 24960 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 11
E0907 19:11:02.321776 24962 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:25:27.692272 25129 GraphService.cpp:81] Session not found, id[495434]
E0907 21:25:33.455902 25131 GraphService.cpp:81] Session not found, id[495505]
E0907 21:26:12.307736 25136 GraphService.cpp:81] Session not found, id[995652]
E0907 21:26:12.327106 25136 GraphService.cpp:81] Session not found, id[438456]
E0907 21:26:12.378038 25136 GraphService.cpp:81] Session not found, id[995588]
E0907 21:26:12.374954 25138 GraphService.cpp:81] Session not found, id[995585]
E0907 21:26:12.412549 25127 GraphService.cpp:81] Session not found, id[995601]
E0907 21:26:13.063091 25129 GraphService.cpp:81] Session not found, id[995620]
E0907 21:26:13.187196 25135 GraphService.cpp:81] Session not found, id[995654]
E0907 21:26:13.322499 25141 GraphService.cpp:81] Session not found, id[995644]
E0907 21:26:13.225984 25126 GraphService.cpp:81] Session not found, id[995655]
E0907 21:26:13.267323 25137 GraphService.cpp:81] Session not found, id[995684]
E0907 21:26:19.382941 25109 StorageClientBase.inl:214] Request to "10.0.7.252":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:19.403229 25112 StorageClientBase.inl:214] Request to "10.0.7.251":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:19.437296 25110 StorageClientBase.inl:214] Request to "10.0.7.140":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:19.439204 25107 StorageClientBase.inl:214] Request to "10.0.7.141":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:19.399567 25108 StorageClientBase.inl:214] Request to "10.0.7.250":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:19.437120 25111 StorageClientBase.inl:214] Request to "10.0.7.139":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:19.597961 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 9
E0907 21:26:19.664868 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 14
E0907 21:26:19.669925 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 8
E0907 21:26:19.670204 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 6
E0907 21:26:19.670703 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 15
E0907 21:26:19.671196 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 1
E0907 21:26:19.672194 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 30
E0907 21:26:19.678607 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 2
E0907 21:26:19.687781 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 12
E0907 21:26:19.693207 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 4
E0907 21:26:19.706452 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 20
E0907 21:26:19.711932 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 22
E0907 21:26:19.717414 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 19
E0907 21:26:19.730266 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 18
E0907 21:26:19.744199 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 17
E0907 21:26:19.751199 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 5
E0907 21:26:19.754199 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 28
E0907 21:26:19.757196 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 11
E0907 21:26:19.762192 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 23
E0907 21:26:19.777015 25134 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 23
E0907 21:26:19.812201 25134 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 26
E0907 21:26:19.819916 25134 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 16
E0907 21:26:19.828850 25134 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:19.808284 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 16
E0907 21:26:19.866195 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 3
E0907 21:26:19.870196 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E0907 21:26:19.874342 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 21
E0907 21:26:19.904711 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 13
E0907 21:26:19.913609 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 10
E0907 21:26:19.923287 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 7
E0907 21:26:19.943218 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 27
E0907 21:26:19.970252 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 25
E0907 21:26:19.989200 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 24
E0907 21:26:19.998009 25127 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 29
E0907 21:26:20.013062 25127 StorageAccessExecutor.h:112] Storage Error: part: 9, error: E_RPC_FAILURE(-3).
E0907 21:26:20.020048 25127 QueryInstance.cpp:103] Storage Error: part: 9, error: E_RPC_FAILURE(-3).
E0907 21:26:20.986338 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 6
E0907 21:26:20.990725 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:26:20.997241 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 14
E0907 21:26:21.043215 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 15
E0907 21:26:21.054040 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 8
E0907 21:26:21.034523 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 6
E0907 21:26:21.073581 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 8
E0907 21:26:21.076678 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 15
E0907 21:26:21.080492 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 14
E0907 21:26:21.092497 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:26:21.113194 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:26:21.125880 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 10
E0907 21:26:21.137198 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:26:21.141703 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:26:21.174204 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 3
E0907 21:26:21.196199 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 26
E0907 21:26:21.208204 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 2
E0907 21:26:21.217715 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 25
E0907 21:26:21.224512 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 1
E0907 21:26:21.233223 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 24
E0907 21:26:21.245216 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 12
E0907 21:26:21.254222 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:26:21.260828 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 4
E0907 21:26:21.272209 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 27
E0907 21:26:21.291558 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 29
E0907 21:26:21.297695 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 18
E0907 21:26:21.066192 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:26:21.398787 25139 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 15
E0907 21:26:21.405671 25139 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 14
E0907 21:26:21.422154 25139 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 14
E0907 21:26:21.429324 25139 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:26:21.441354 25139 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:26:21.449442 25139 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:26:21.460561 25139 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:26:21.414350 25137 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:26:21.323388 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:26:21.558195 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 29
E0907 21:26:21.563794 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 17
E0907 21:26:21.570310 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 2
E0907 21:26:21.597127 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 25
E0907 21:26:21.599265 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 24
E0907 21:26:21.606194 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 1
E0907 21:26:21.609200 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:26:21.617208 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 4
E0907 21:26:21.631392 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 27
E0907 21:26:21.645651 25132 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:21.653455 25132 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:21.668351 25132 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:21.700742 25135 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:21.597064 25141 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:21.649462 25140 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:21.638119 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 26
E0907 21:26:21.858202 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 3
E0907 21:26:21.863566 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 30
E0907 21:26:21.581604 25131 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:21.300400 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 16
E0907 21:26:21.765563 25136 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 8
E0907 21:26:21.908195 25136 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 6
E0907 21:26:21.911201 25136 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:26:21.914196 25136 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 14
E0907 21:26:21.879034 25130 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 6
E0907 21:26:21.933207 25130 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 14
E0907 21:26:21.940726 25130 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:26:21.958415 25130 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 8
E0907 21:26:21.971443 25130 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 15
E0907 21:26:21.992974 25137 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:21.866194 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 12
E0907 21:26:22.011384 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:26:21.921201 25136 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 15
E0907 21:26:21.898767 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 23
E0907 21:26:22.103686 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 30
E0907 21:26:22.113211 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:26:22.128137 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 20
E0907 21:26:22.135196 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 17
E0907 21:26:22.141191 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 22
E0907 21:26:22.148010 25126 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:26:22.021204 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:26:22.220654 25129 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:27:01.999383  4886 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:27:01.999642  4886 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 15
E0907 21:27:01.999671  4887 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:18.085232  4877 GraphService.cpp:81] Session not found, id[885950]



================================nebula-server-3=========================
E0907 13:31:54.942699 17653 GraphService.cpp:81] Session not found, id[44657125]
E0907 14:09:09.263646 17651 GraphService.cpp:81] Session not found, id[171649]
E0907 14:31:13.380036 17653 GraphService.cpp:81] Session not found, id[22460016]
E0907 15:27:02.235539  6538 GraphService.cpp:81] Session not found, id[168]
E0907 16:23:02.824852 11805 GraphService.cpp:81] Session not found, id[140]
E0907 16:23:23.664683 11801 GraphService.cpp:81] Session not found, id[75828]
E0907 16:25:27.373039 11806 GraphService.cpp:81] Session not found, id[847689]
E0907 16:25:27.859417 11807 GraphService.cpp:81] Session not found, id[847594]
E0907 16:36:46.244226 11806 GraphService.cpp:81] Session not found, id[58081]
E0907 17:01:38.754375 11808 GraphService.cpp:81] Session not found, id[128023]
E0907 17:08:03.280349 16054 GraphService.cpp:81] Session not found, id[105]
E0907 17:28:58.256837 16050 GraphService.cpp:81] Session not found, id[136438]
E0907 17:32:02.978972 18356 GraphService.cpp:81] Session not found, id[40]
E0907 17:32:03.106894 18357 GraphService.cpp:81] Session not found, id[56]
E0907 17:45:32.576189 19015 GraphService.cpp:81] Session not found, id[123017]
E0907 17:52:11.409672 19003 GraphService.cpp:81] Session not found, id[82738]
E0907 21:14:52.956315 24098 GraphService.cpp:81] Session not found, id[946642]
E0907 21:25:33.209900 24094 GraphService.cpp:81] Session not found, id[437828]
E0907 21:25:33.426470 24094 GraphService.cpp:81] Session not found, id[437831]
E0907 21:25:33.732043 24101 GraphService.cpp:81] Session not found, id[272170]
E0907 21:25:34.319813 24102 GraphService.cpp:81] Session not found, id[433459]
E0907 21:25:34.688326 24100 GraphService.cpp:81] Session not found, id[272185]
E0907 21:26:12.329469 24108 GraphService.cpp:81] Session not found, id[438457]
E0907 21:26:12.390834 24106 GraphService.cpp:81] Session not found, id[995646]
E0907 21:26:12.305819 24096 GraphService.cpp:81] Session not found, id[995600]
E0907 21:26:12.385259 24100 GraphService.cpp:81] Session not found, id[995583]
E0907 21:26:12.427728 24098 GraphService.cpp:81] Session not found, id[995572]
E0907 21:26:13.048511 24098 GraphService.cpp:81] Session not found, id[995563]
E0907 21:26:13.182906 24096 GraphService.cpp:81] Session not found, id[995657]
E0907 21:26:22.723740 24070 StorageClientBase.inl:214] Request to "10.0.7.140":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:22.748039 24094 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 10
E0907 21:26:22.753693 24094 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 5
E0907 21:26:22.755620 24094 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 7
E0907 21:26:22.756938 24094 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 13
E0907 21:26:22.758819 24094 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 11
E0907 21:26:22.760825 24094 StorageAccessExecutor.h:112] Storage Error: part: 10, error: E_RPC_FAILURE(-3).
E0907 21:26:22.762564 24095 QueryInstance.cpp:103] Storage Error: part: 10, error: E_RPC_FAILURE(-3).
E0907 21:26:23.110855 24100 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:26:23.112838 24100 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:26:23.116084 24100 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 10
E0907 21:26:23.118098 24100 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:26:23.119818 24100 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:26:23.123242 24107 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:23.175218 24093 StorageAccessExecutor.h:35] PassThroughExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:26:23.178120 24093 StorageAccessExecutor.h:35] PassThroughExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:26:23.181453 24093 StorageAccessExecutor.h:35] PassThroughExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:26:23.186295 24093 StorageAccessExecutor.h:35] PassThroughExecutor failed, error E_LEADER_CHANGED, part 10
E0907 21:26:23.188455 24093 StorageAccessExecutor.h:35] PassThroughExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:26:32.282950 24098 GraphService.cpp:81] Session not found, id[434658]
E0907 21:26:34.232328 24104 GraphService.cpp:81] Session not found, id[434654]
E0907 21:26:34.468740 24098 GraphService.cpp:81] Session not found, id[434656]
E0907 21:54:47.422984 24079 StorageClientBase.inl:214] Request to "10.0.7.251":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:54:47.425833 24075 StorageClientBase.inl:214] Request to "10.0.7.250":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:54:47.430881 24078 StorageClientBase.inl:214] Request to "10.0.7.139":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:54:47.425848 24074 StorageClientBase.inl:214] Request to "10.0.7.141":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:54:47.444519 24077 StorageClientBase.inl:214] Request to "10.0.7.140":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:54:47.441530 24076 StorageClientBase.inl:214] Request to "10.0.7.252":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:54:47.753877 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E0907 21:54:47.761621 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 21
E0907 21:54:47.766829 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 16
E0907 21:54:47.772861 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 23
E0907 21:54:47.784826 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 28
E0907 21:54:47.788825 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 3
E0907 21:54:47.797120 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 12
E0907 21:54:47.812167 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 1
E0907 21:54:47.822551 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 30
E0907 21:54:47.829569 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 24
E0907 21:54:47.839187 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 2
E0907 21:54:47.845824 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 25
E0907 21:54:47.853966 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 8
E0907 21:54:47.861380 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 4
E0907 21:54:47.868822 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 27
E0907 21:54:47.874922 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 19
E0907 21:54:47.886698 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 18
E0907 21:54:47.897832 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 17
E0907 21:54:47.902827 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 22
E0907 21:54:47.905820 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 20
E0907 21:54:47.915613 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 15
E0907 21:54:47.925841 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 29
E0907 21:54:47.943847 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 6
E0907 21:54:47.949366 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 14
E0907 21:54:47.960834 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 9
E0907 21:54:47.978454 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 13
E0907 21:54:47.987200 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 7
E0907 21:54:48.005832 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 10
E0907 21:54:48.016767 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 5
E0907 21:54:48.077597 24105 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 11
E0907 21:54:48.080827 24105 StorageAccessExecutor.h:112] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E0907 21:54:48.085883 24105 QueryInstance.cpp:103] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E0907 21:54:48.230013 24095 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 23
E0907 21:54:48.239181 24095 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:48.264889 24095 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 3
E0907 21:54:48.280836 24095 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 18
E0907 21:54:48.299232 24095 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:54:48.304355 24095 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 26
E0907 21:54:48.311331 24095 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 23
E0907 21:54:48.321996 24095 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 12
E0907 21:54:48.338819 24095 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 27
E0907 21:54:48.356940 24095 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:54:48.366494 24095 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:48.911854 24099 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 4
E0907 21:54:48.923074 24099 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 10
E0907 21:54:48.940830 24099 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:54:48.960924 24099 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 20
E0907 21:54:48.974529 24099 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:54:48.996887 24099 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 30
E0907 21:54:49.007261 24099 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 25
E0907 21:54:49.022657 24099 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 24
E0907 21:54:49.045838 24099 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 2
E0907 21:54:49.054934 24099 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:49.155514 24107 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:54:49.176314 24107 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:49.368414 24102 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:54:49.376320 24102 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 19
E0907 21:54:49.378863 24102 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:50.483834 24099 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:54:50.492339 24099 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:50.568837 24098 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:54:50.586889 24098 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:50.515851 24099 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:54:50.621273 24099 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:50.684636 24108 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 22
E0907 21:54:50.688629 24108 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 8
E0907 21:54:50.696130 24108 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 15
E0907 21:54:50.724830 24108 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 29
E0907 21:54:50.725899 24108 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 14
E0907 21:54:50.725935 24108 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 1
E0907 21:54:50.725961 24108 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 16
E0907 21:54:50.723660 24099 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 16
E0907 21:54:50.768050 24094 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:50.671509 24097 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 16
E0907 21:54:50.804708 24097 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:50.817864 24094 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 16
E0907 21:54:50.655370 24093 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 1
E0907 21:54:50.878841 24093 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 8
E0907 21:54:50.892834 24093 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 29
E0907 21:54:50.916539 24093 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 22
E0907 21:54:50.936833 24093 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 17
E0907 21:54:50.950896 24093 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 6
E0907 21:54:50.984238 24106 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:54:50.727821 24108 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:54:51.019884 24108 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 6
E0907 21:54:51.031510 24108 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 17

nebula-server-4,5,6 graphd-stderr.log 信息

=======================================nebula-server-4===============================
E0907 15:16:35.697944 25994 GraphService.cpp:81] Session not found, id[4103533]
E0907 15:59:10.759276 25991 GraphService.cpp:81] Session not found, id[45802265]
E0907 16:25:28.254501 28838 GraphService.cpp:81] Session not found, id[847675]
E0907 17:06:33.086369 31612 GraphService.cpp:81] Session not found, id[274453]
E0907 17:17:27.054302 31609 GraphService.cpp:81] Session not found, id[318582]
E0907 17:31:04.665796  2095 StorageClientBase.inl:214] Request to "10.0.7.140":9779 failed: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: write timed out during connection, type = Timed out
E0907 17:31:04.666136  2095 StorageClientBase.inl:214] Request to "10.0.7.140":9779 failed: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: socket closing after error (peer=10.0.7.140:9779, local=<uninitialized address>), type = Internal error
E0907 17:31:04.666177  2119 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_RPC_FAILURE, part 10
E0907 17:31:04.666229  2116 StorageAccessExecutor.h:35] GetEdgesExecutor failed, error E_RPC_FAILURE, part 11
E0907 17:31:04.666245  2119 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_RPC_FAILURE, part 5
E0907 17:31:04.666283  2116 StorageAccessExecutor.h:112] Storage Error: part: 11, error: E_RPC_FAILURE(-3).
E0907 17:31:04.666285  2119 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_RPC_FAILURE, part 6
E0907 17:31:04.666314  2119 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_RPC_FAILURE, part 8
E0907 17:31:04.666329  2119 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_RPC_FAILURE, part 13
E0907 17:31:04.666335  2120 QueryInstance.cpp:103] Storage Error: part: 11, error: E_RPC_FAILURE(-3).
E0907 17:31:04.666347  2119 StorageAccessExecutor.h:112] Storage Error: part: 10, error: E_RPC_FAILURE(-3).
E0907 17:31:04.666429  2111 QueryInstance.cpp:103] Storage Error: part: 10, error: E_RPC_FAILURE(-3).
E0907 17:31:04.676393  2111 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 6
E0907 17:31:04.676434  2111 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 5
E0907 17:31:04.676445  2111 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 10
E0907 17:31:04.676483  2119 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 17:31:04.744033  2105 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 13
E0907 17:31:04.744087  2105 StorageAccessExecutor.h:35] GetVerticesExecutor failed, error E_LEADER_CHANGED, part 8
E0907 17:31:04.744151  2120 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 17:31:05.223124  2120 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 11
E0907 17:31:05.223259  2119 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 17:36:57.859728  2111 GraphService.cpp:81] Session not found, id[13969]
E0907 18:39:03.564208  5289 GraphService.cpp:81] Session not found, id[83463]
E0907 21:25:33.900532 11375 GraphService.cpp:81] Session not found, id[784773]
E0907 21:26:12.348994 11379 GraphService.cpp:81] Session not found, id[438458]
E0907 21:26:12.394631 11370 GraphService.cpp:81] Session not found, id[995564]
E0907 21:26:12.397418 11370 GraphService.cpp:81] Session not found, id[995606]
E0907 21:26:12.424892 11369 GraphService.cpp:81] Session not found, id[995667]
E0907 21:26:12.461580 11371 GraphService.cpp:81] Session not found, id[438454]
E0907 21:26:32.216281 11375 GraphService.cpp:81] Session not found, id[434663]
E0907 21:26:32.232760 11375 GraphService.cpp:81] Session not found, id[434660]
E0907 21:26:32.378952 11369 GraphService.cpp:81] Session not found, id[434648]
E0907 21:26:34.297472 11382 GraphService.cpp:81] Session not found, id[434662]
E0907 21:26:34.355325 11345 StorageClientBase.inl:214] Request to "10.0.7.140":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:34.390259 11347 StorageClientBase.inl:214] Request to "10.0.7.251":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:34.326166 11349 StorageClientBase.inl:214] Request to "10.0.7.252":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:34.372721 11348 StorageClientBase.inl:214] Request to "10.0.7.139":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:34.389559 11350 StorageClientBase.inl:214] Request to "10.0.7.141":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:34.404037 11346 StorageClientBase.inl:214] Request to "10.0.7.250":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:26:34.868929 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 25
E0907 21:26:34.872356 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 30
E0907 21:26:34.874285 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 24
E0907 21:26:34.875679 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 29
E0907 21:26:34.877144 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 27
E0907 21:26:34.877406 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 9
E0907 21:26:34.879802 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 14
E0907 21:26:34.883373 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 8
E0907 21:26:34.884965 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 6
E0907 21:26:34.887516 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 15
E0907 21:26:34.893267 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 20
E0907 21:26:34.896251 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 17
E0907 21:26:34.899519 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 19
E0907 21:26:34.911144 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 22
E0907 21:26:34.911901 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 11
E0907 21:26:34.914247 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 5
E0907 21:26:34.915284 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 28
E0907 21:26:34.917248 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 18
E0907 21:26:34.919245 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 7
E0907 21:26:34.921957 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 10
E0907 21:26:34.936619 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 13
E0907 21:26:35.132251 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 23
E0907 21:26:35.318584 11370 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:26:35.140251 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 16
E0907 21:26:35.383247 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 21
E0907 21:26:35.387926 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E0907 21:26:35.399979 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 4
E0907 21:26:35.415356 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 12
E0907 21:26:35.418548 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 2
E0907 21:26:35.430161 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 3
E0907 21:26:35.435488 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 1
E0907 21:26:35.444362 11373 StorageAccessExecutor.h:112] Storage Error: part: 25, error: E_RPC_FAILURE(-3).
E0907 21:26:35.511297 11380 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:35.542620 11380 QueryInstance.cpp:103] Storage Error: part: 25, error: E_RPC_FAILURE(-3).
E0907 21:26:35.307390 11368 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:26:35.724440 11368 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 6
E0907 21:26:35.739537 11368 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:26:35.755249 11368 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:26:35.771584 11368 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 22
E0907 21:26:35.852902 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 19
E0907 21:26:35.788259 11368 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:26:35.840798 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 2
E0907 21:26:36.032778 11378 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:36.056480 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 8
E0907 21:26:35.946766 11368 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 3
E0907 21:26:36.323282 11368 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 25
E0907 21:26:36.340234 11368 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 15
E0907 21:26:36.402557 11382 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:36.413436 11382 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:26:36.294687 11375 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:36.534890 11376 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:37.149866 11369 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 27
E0907 21:26:37.155381 11369 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 10
E0907 21:26:37.158248 11369 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 26
E0907 21:26:37.165657 11369 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 1
E0907 21:26:37.241400 11382 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:26:37.251839 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 20
E0907 21:26:37.257244 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 18
E0907 21:26:37.262255 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 23
E0907 21:26:37.266317 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 4
E0907 21:26:37.271250 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 26
E0907 21:26:37.280251 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 17
E0907 21:26:37.285697 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 10
E0907 21:26:37.334260 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 30
E0907 21:26:37.345089 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:26:37.495258 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 24
E0907 21:26:37.501262 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 29
E0907 21:26:37.515247 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 12
E0907 21:26:37.525262 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 14
E0907 21:26:37.535899 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 16
E0907 21:26:37.612490 11368 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:14.006099 11348 StorageClientBase.inl:214] Request to "10.0.7.251":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:55:14.008358 11349 StorageClientBase.inl:214] Request to "10.0.7.139":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:55:14.010036 11345 StorageClientBase.inl:214] Request to "10.0.7.140":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:55:14.006865 11347 StorageClientBase.inl:214] Request to "10.0.7.141":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:55:14.022269 11350 StorageClientBase.inl:214] Request to "10.0.7.250":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:55:14.012357 11346 StorageClientBase.inl:214] Request to "10.0.7.252":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:55:14.205737 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E0907 21:55:14.210237 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 16
E0907 21:55:14.215245 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 21
E0907 21:55:14.219247 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 28
E0907 21:55:14.225244 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 23
E0907 21:55:14.230245 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 3
E0907 21:55:14.237252 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 4
E0907 21:55:14.243242 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 2
E0907 21:55:14.249248 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 12
E0907 21:55:14.254241 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 1
E0907 21:55:14.260244 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 30
E0907 21:55:14.267241 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 14
E0907 21:55:14.274242 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 15
E0907 21:55:14.279234 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 6
E0907 21:55:14.285243 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 17
E0907 21:55:14.291237 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 22
E0907 21:55:14.296814 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 5
E0907 21:55:14.301239 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 18
E0907 21:55:14.309242 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 9
E0907 21:55:14.317245 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 20
E0907 21:55:14.322240 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 8
E0907 21:55:14.327246 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 19
E0907 21:55:14.331244 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 7
E0907 21:55:14.335253 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 10
E0907 21:55:14.342828 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 13
E0907 21:55:14.349248 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 11
E0907 21:55:14.353241 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 27
E0907 21:55:14.360249 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 25
E0907 21:55:14.367241 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 24
E0907 21:55:14.374244 11378 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 29
E0907 21:55:14.380245 11378 StorageAccessExecutor.h:112] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E0907 21:55:14.388319 11374 QueryInstance.cpp:103] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E0907 21:55:17.077292 11382 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:55:17.085309 11373 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:18.081705 11372 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 23
E0907 21:55:18.102396 11377 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:18.112141 11374 GraphService.cpp:81] Session not found, id[885954]
E0907 21:55:18.130255 11373 GraphService.cpp:81] Session not found, id[885951]
E0907 21:55:18.078538 11382 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 23
E0907 21:55:18.158753 11374 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:18.182581 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:55:18.206291 11370 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:55:18.212797 11376 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:18.189162 11375 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:55:18.231473 11375 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:18.187290 11373 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:18.199652 11368 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:55:18.290292 11368 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:18.296463 11371 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:55:18.312418 11379 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:18.261266 11376 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:55:18.372359 11376 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:18.457283 11380 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 19
E0907 21:55:18.482581 11380 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:55:18.507654 11380 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:18.625679 11369 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:55:18.632331 11369 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:18.682036 11375 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 14
E0907 21:55:18.684257 11375 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 6
E0907 21:55:18.684290 11375 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 15
E0907 21:55:18.684310 11375 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:55:18.684330 11375 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 8
E0907 21:55:18.684371 11375 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:19.011301 11381 GraphService.cpp:81] Session not found, id[885953]
E0907 21:55:19.045298 11372 GraphService.cpp:81] Session not found, id[885952]
E0907 21:55:19.204264 11374 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 24
E0907 21:55:19.209244 11374 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:55:19.221244 11374 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 27
E0907 21:55:19.233630 11374 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 16
E0907 21:55:19.240507 11374 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 17
E0907 21:55:19.245240 11374 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:55:19.251375 11374 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 3
E0907 21:55:19.259277 11374 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:21.666323 11371 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 2
E0907 21:55:21.673281 11369 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:22.264780 11369 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 29
E0907 21:55:22.269250 11369 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 4
E0907 21:55:22.282243 11369 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 22
E0907 21:55:22.295246 11369 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 25
E0907 21:55:22.305261 11369 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:28.093281 11374 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 10
E0907 21:55:28.097244 11374 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 26
E0907 21:55:28.103271 11377 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:28.632277 11377 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 20
E0907 21:55:28.640290 11372 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:30.508947 11373 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 30
E0907 21:55:30.515290 11369 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:30.956257 11377 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 12
E0907 21:55:30.964262 11377 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:32.608283 11377 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 18
E0907 21:55:32.615243 11377 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 1
E0907 21:55:32.618290 11382 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later



=======================================nebula-server-5===============================
E0907 16:07:11.224901 27589 GraphService.cpp:81] Session not found, id[45867344]
E0907 16:21:18.613818 27587 GraphService.cpp:81] Session not found, id[344467]
E0907 16:21:18.632097 27587 GraphService.cpp:81] Session not found, id[344448]
E0907 16:21:18.658625 27585 GraphService.cpp:81] Session not found, id[344457]
E0907 16:21:18.731251 27588 GraphService.cpp:81] Session not found, id[344466]
E0907 16:21:18.806205 27592 GraphService.cpp:81] Session not found, id[344447]
E0907 16:21:20.198604 27592 GraphService.cpp:81] Session not found, id[344469]
E0907 16:21:20.403445 27579 GraphService.cpp:81] Session not found, id[344465]
E0907 16:21:20.655804 27586 GraphService.cpp:81] Session not found, id[344456]
E0907 16:21:20.705541 27592 GraphService.cpp:81] Session not found, id[344460]
E0907 16:21:20.736562 27589 GraphService.cpp:81] Session not found, id[344462]
E0907 16:23:24.126397 27583 GraphService.cpp:81] Session not found, id[75810]
E0907 16:25:28.256578 27583 GraphService.cpp:81] Session not found, id[847688]
E0907 17:01:38.202597 27579 GraphService.cpp:81] Session not found, id[128029]
E0907 17:01:38.361356 27587 GraphService.cpp:81] Session not found, id[128027]
E0907 17:01:38.641001 27584 GraphService.cpp:81] Session not found, id[128028]
E0907 17:10:38.201486 27584 GraphService.cpp:81] Session not found, id[759940]
E0907 17:10:38.490022 27582 GraphService.cpp:81] Session not found, id[759943]
E0907 17:10:38.510288 27593 GraphService.cpp:81] Session not found, id[759937]
E0907 17:28:58.146559 21034 GraphService.cpp:81] Session not found, id[136448]
E0907 17:29:35.946132 21022 GraphService.cpp:81] Session not found, id[200522]
E0907 17:36:57.941108 21023 GraphService.cpp:81] Session not found, id[13965]
E0907 18:12:22.054491 24173 GraphService.cpp:81] Session not found, id[259359]
E0907 21:25:33.216337 24174 GraphService.cpp:81] Session not found, id[495501]
E0907 21:25:33.229072 24182 GraphService.cpp:81] Session not found, id[433441]
E0907 21:25:33.831843 24172 GraphService.cpp:81] Session not found, id[784771]
E0907 21:25:34.211422 24176 GraphService.cpp:81] Session not found, id[437846]
E0907 21:25:34.331815 24178 GraphService.cpp:81] Session not found, id[437848]
E0907 21:25:48.804301 24153 StorageClientBase.inl:214] Request to "10.0.7.252":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:48.834033 24150 StorageClientBase.inl:214] Request to "10.0.7.139":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:48.813359 24151 StorageClientBase.inl:214] Request to "10.0.7.140":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:48.834342 24149 StorageClientBase.inl:214] Request to "10.0.7.251":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:48.817281 24147 StorageClientBase.inl:214] Request to "10.0.7.250":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:48.809631 24152 StorageClientBase.inl:214] Request to "10.0.7.141":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:49.480675 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 1
E0907 21:25:49.486259 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 3
E0907 21:25:49.492255 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 12
E0907 21:25:49.498258 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 2
E0907 21:25:49.499272 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 4
E0907 21:25:49.503245 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 18
E0907 21:25:49.510253 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 20
E0907 21:25:49.516255 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 17
E0907 21:25:49.520272 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 22
E0907 21:25:49.521261 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 19
E0907 21:25:49.526262 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 14
E0907 21:25:49.528265 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 9
E0907 21:25:49.534250 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E0907 21:25:49.537801 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 8
E0907 21:25:49.546322 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 23
E0907 21:25:49.555250 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 6
E0907 21:25:49.560254 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 11
E0907 21:25:49.564255 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 16
E0907 21:25:49.570147 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 15
E0907 21:25:49.573405 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 21
E0907 21:25:49.584244 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 13
E0907 21:25:49.593250 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 7
E0907 21:25:49.945135 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 10
E0907 21:25:49.956272 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 28
E0907 21:25:49.965581 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 5
E0907 21:25:49.982293 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 27
E0907 21:25:50.042259 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 24
E0907 21:25:50.056258 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 29
E0907 21:25:50.079255 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 30
E0907 21:25:50.100261 24185 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 25
E0907 21:25:50.112262 24185 StorageAccessExecutor.h:112] Storage Error: part: 1, error: E_RPC_FAILURE(-3).
E0907 21:25:50.155597 24184 QueryInstance.cpp:103] Storage Error: part: 1, error: E_RPC_FAILURE(-3).
E0907 21:25:51.057541 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 8
E0907 21:25:51.109097 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 15
E0907 21:25:51.128279 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 30
E0907 21:25:51.172293 24181 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 12
E0907 21:25:51.173282 24181 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 3
E0907 21:25:51.176275 24181 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:25:51.172480 24171 StorageAccessExecutor.h:35] GetEdgesExecutor failed, error E_LEADER_CHANGED, part 19
E0907 21:25:51.209497 24171 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:25:51.141270 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:25:51.218467 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:25:51.304524 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 10
E0907 21:25:51.316009 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:25:51.322047 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:25:51.334563 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 20
E0907 21:25:51.339268 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 25
E0907 21:25:51.350509 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 26
E0907 21:25:51.281615 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 15
E0907 21:25:51.398857 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 8
E0907 21:25:51.419158 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:25:51.434274 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:25:51.450390 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:25:51.467234 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 25
E0907 21:25:51.485280 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 30
E0907 21:25:51.495339 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:25:51.525722 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 24
E0907 21:25:51.536281 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 23
E0907 21:25:51.548171 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 29
E0907 21:25:51.568979 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 6
E0907 21:25:51.575909 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 18
E0907 21:25:51.648548 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 16
E0907 21:25:51.671268 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 27
E0907 21:25:51.681639 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 26
E0907 21:25:51.689937 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 17
E0907 21:25:51.697824 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 22
E0907 21:25:51.713807 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 19
E0907 21:25:51.724171 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 20
E0907 21:25:51.740636 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:25:51.754658 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:25:51.768296 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 10
E0907 21:25:51.776273 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:25:51.838519 24183 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 14
E0907 21:25:51.839763 24183 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:25:51.367286 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 29
E0907 21:25:51.910255 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 6
E0907 21:25:51.983858 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 24
E0907 21:25:51.984267 24180 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:25:52.062275 24180 StorageAccessExecutor.h:35] `£££ failed, error E_LEADER_CHANGED, part 27
E0907 21:25:52.089643 24180 StorageAccessExecutor.h:35] `£££ failed, error E_LEADER_CHANGED, part 21
E0907 21:25:52.110265 24180 StorageAccessExecutor.h:35] `£££ failed, error E_LEADER_CHANGED, part 16
E0907 21:25:52.127128 24180 StorageAccessExecutor.h:35] `£££ failed, error E_LEADER_CHANGED, part 23
E0907 21:25:52.134414 24180 StorageAccessExecutor.h:35] `£££ failed, error E_LEADER_CHANGED, part 18
E0907 21:25:52.143909 24180 StorageAccessExecutor.h:35] `£££ failed, error E_LEADER_CHANGED, part 22
E0907 21:25:52.151099 24180 StorageAccessExecutor.h:35] `£££ failed, error E_LEADER_CHANGED, part 17
E0907 21:25:52.154981 24180 StorageAccessExecutor.h:35] `£££ failed, error E_LEADER_CHANGED, part 19
E0907 21:25:52.163895 24180 StorageAccessExecutor.h:35] `£££ failed, error E_LEADER_CHANGED, part 9
E0907 21:25:52.172930 24180 StorageAccessExecutor.h:35] `£££ failed, error E_LEADER_CHANGED, part 14
E0907 21:55:18.975433  8462 GraphService.cpp:81] Session not found, id[885947]

=======================================nebula-server-6===============================
E0907 15:32:40.222616 31713 GraphService.cpp:81] Session not found, id[45576042]
E0907 16:21:18.582566 31712 GraphService.cpp:81] Session not found, id[344458]
E0907 16:21:18.701050 31718 GraphService.cpp:81] Session not found, id[344477]
E0907 16:21:18.806548 31722 GraphService.cpp:81] Session not found, id[344472]
E0907 16:21:19.254384 31708 GraphService.cpp:81] Session not found, id[344229]
E0907 16:21:19.395689 31712 GraphService.cpp:81] Session not found, id[344478]
E0907 16:21:20.428084 31718 GraphService.cpp:81] Session not found, id[344470]
E0907 16:21:20.671893 31712 GraphService.cpp:81] Session not found, id[344461]
E0907 16:21:20.725598 31714 GraphService.cpp:81] Session not found, id[344476]
E0907 16:21:20.756912 31711 GraphService.cpp:81] Session not found, id[344459]
E0907 16:54:24.403863  9708 GraphService.cpp:81] Session not found, id[204276]
E0907 17:10:38.445773  9711 GraphService.cpp:81] Session not found, id[759942]
E0907 17:19:01.649158 14494 GraphService.cpp:81] Session not found, id[56]
E0907 17:28:58.248636 14495 GraphService.cpp:81] Session not found, id[136424]
E0907 17:29:36.252630 14494 GraphService.cpp:81] Session not found, id[200520]
E0907 17:36:57.943140 14494 GraphService.cpp:81] Session not found, id[13974]
E0907 18:04:34.666240 14489 GraphService.cpp:81] Session not found, id[13506]
E0907 21:25:33.250537 21747 GraphService.cpp:81] Session not found, id[272161]
E0907 21:25:57.053558 21718 StorageClientBase.inl:214] Request to "10.0.7.139":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:57.077077 21716 StorageClientBase.inl:214] Request to "10.0.7.250":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:57.060541 21730 StorageClientBase.inl:214] Request to "10.0.7.141":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:57.079778 21719 StorageClientBase.inl:214] Request to "10.0.7.252":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:57.061558 21717 StorageClientBase.inl:214] Request to "10.0.7.251":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:57.744686 21731 StorageClientBase.inl:214] Request to "10.0.7.140":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:25:57.779650 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 5
E0907 21:25:57.783537 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 10
E0907 21:25:57.785382 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 13
E0907 21:25:57.789526 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 11
E0907 21:25:57.792064 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 7
E0907 21:25:57.795521 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 28
E0907 21:25:57.800760 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 23
E0907 21:25:57.802836 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 21
E0907 21:25:57.805613 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 16
E0907 21:25:57.807523 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 12
E0907 21:25:57.811704 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 30
E0907 21:25:57.813524 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 1
E0907 21:25:57.815531 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 8
E0907 21:25:57.817824 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 15
E0907 21:25:57.819522 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 29
E0907 21:25:57.825546 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 6
E0907 21:25:57.830531 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 9
E0907 21:25:57.832522 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 14
E0907 21:25:57.838795 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E0907 21:25:57.840204 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 3
E0907 21:25:57.841050 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 27
E0907 21:25:57.849520 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 4
E0907 21:25:57.851653 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 24
E0907 21:25:57.854362 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 25
E0907 21:25:57.861861 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 2
E0907 21:25:57.862529 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 18
E0907 21:25:57.865592 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 19
E0907 21:25:57.874521 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 20
E0907 21:25:57.878367 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 17
E0907 21:25:57.897593 21745 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 22
E0907 21:25:57.911778 21745 StorageAccessExecutor.h:112] Storage Error: part: 5, error: E_RPC_FAILURE(-3).
E0907 21:25:57.921602 21745 QueryInstance.cpp:103] Storage Error: part: 5, error: E_RPC_FAILURE(-3).
E0907 21:25:57.945894 21743 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 22
E0907 21:25:57.950603 21743 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:25:57.982578 21754 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 19
E0907 21:25:57.991639 21754 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:25:58.178870 21754 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 25
E0907 21:25:58.189707 21754 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:25:58.223948 21754 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 25
E0907 21:25:58.246124 21754 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:25:58.366652 21754 StorageAccessExecutor.h:35] PassThroughExecutor failed, error E_LEADER_CHANGED, part 23
E0907 21:25:58.489002 21754 StorageAccessExecutor.h:35] "¸"¸"ʼ failed, error E_LEADER_CHANGED, part 23
E0907 21:25:58.485584 21746 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:25:58.526564 21746 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:07.220240  3477 StorageClientBase.inl:214] Request to "10.0.7.140":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:55:07.223670  3481 StorageClientBase.inl:214] Request to "10.0.7.141":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:55:07.223932  3478 StorageClientBase.inl:214] Request to "10.0.7.250":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:55:07.231611  3476 StorageClientBase.inl:214] Request to "10.0.7.252":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:55:07.239539  3479 StorageClientBase.inl:214] Request to "10.0.7.251":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:55:07.234541  3480 StorageClientBase.inl:214] Request to "10.0.7.139":9779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E0907 21:55:07.673166  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 9
E0907 21:55:07.683538  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 14
E0907 21:55:07.699612  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 8
E0907 21:55:07.700549  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 6
E0907 21:55:07.704799  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 15
E0907 21:55:07.706760  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 21
E0907 21:55:07.707525  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 16
E0907 21:55:07.708518  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 23
E0907 21:55:07.709533  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E0907 21:55:07.710541  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 3
E0907 21:55:07.711517  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 12
E0907 21:55:07.712522  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 29
E0907 21:55:07.713518  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 13
E0907 21:55:07.721532  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 11
E0907 21:55:07.724596  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 10
E0907 21:55:07.727524  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 4
E0907 21:55:07.731524  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 27
E0907 21:55:07.733345  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 28
E0907 21:55:07.734516  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 5
E0907 21:55:07.738523  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 30
E0907 21:55:07.741518  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 1
E0907 21:55:07.745524  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 7
E0907 21:55:07.747530  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 24
E0907 21:55:07.748528  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 2
E0907 21:55:07.755597  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 25
E0907 21:55:07.766533  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 18
E0907 21:55:07.796299  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 19
E0907 21:55:07.826532  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 17
E0907 21:55:07.850318  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 22
E0907 21:55:07.859562  3507 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 20
E0907 21:55:07.952540  3507 StorageAccessExecutor.h:112] Storage Error: part: 9, error: E_RPC_FAILURE(-3).
E0907 21:55:07.986692  3493 QueryInstance.cpp:103] Storage Error: part: 9, error: E_RPC_FAILURE(-3).
E0907 21:55:08.070641  3496 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 9
E0907 21:55:08.082048  3504 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:08.105536  3504 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 23
E0907 21:55:08.183576  3504 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:08.229576  3506 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:55:08.232609  3502 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:08.258945  3505 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 21
E0907 21:55:08.283701  3503 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:08.476006  3497 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 22
E0907 21:55:08.498672  3497 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 16
E0907 21:55:08.514076  3497 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 3
E0907 21:55:08.524529  3497 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 19
E0907 21:55:08.530550  3497 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 15
E0907 21:55:08.538563  3497 StorageAccessExecutor.h:35] InsertVerticesExecutor failed, error E_LEADER_CHANGED, part 4
E0907 21:55:08.561578  3497 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:09.321660  3494 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 13
E0907 21:55:09.339125  3494 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:12.515741  3506 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 24
E0907 21:55:12.527529  3506 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 2
E0907 21:55:12.534526  3506 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 11
E0907 21:55:12.542378  3506 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 27
E0907 21:55:12.550526  3506 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 14
E0907 21:55:12.557662  3506 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 26
E0907 21:55:12.574033  3506 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 1
E0907 21:55:12.588626  3506 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 30
E0907 21:55:12.592520  3506 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 8
E0907 21:55:12.604589  3506 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:12.664114  3493 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 7
E0907 21:55:12.676599  3493 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:14.041811  3494 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 20
E0907 21:55:14.052757  3494 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:14.174096  3493 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 25
E0907 21:55:14.182915  3493 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:19.070595  3499 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 5
E0907 21:55:19.075323  3499 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 29
E0907 21:55:19.094034  3499 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:19.387528  3492 StorageAccessExecutor.h:35] InsertEdgesExecutor failed, error E_LEADER_CHANGED, part 28
E0907 21:55:19.391580  3492 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:24.194559  3506 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 12
E0907 21:55:24.199568  3496 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:25.949591  3506 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 6
E0907 21:55:25.952579  3506 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:33.319559  3494 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 10
E0907 21:55:33.324522  3494 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 17
E0907 21:55:33.329761  3494 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later
E0907 21:55:35.397996  3502 StorageAccessExecutor.h:35] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 18
E0907 21:55:35.402559  3495 QueryInstance.cpp:103] Storage Error: The leader has changed. Try again later

日志中没看到关于core 有效信息, 还是对core文件分析吧
1、看一下 core文件是由哪一个 graphd 进程产生的。
2、gdb core文件,看一下当时的帧栈
可以看一下,最近有执行 go 语句没,执行go语句时候的语句是什么

找到复现的语句了

go from
    -4750792628681723695
over
       Invest
where
       Invest._dst == 1554156494904300803 or Invest._dst == -3635610059008749053 
yield
       Invest._src as src,
       Invest._dst as dst,
       Invest.prop1,
       Invest.startid,
       Invest.endid

where 后跟了2100+个判断条件。

请问一下,这边条件数目有限制吗?

你确定 where后面加了 2100+个条件判断? 为什么会加这么多啊

我们马上会加一个新功能,限制一条语句最大 大小为4MB(可以调整), 太大的话会报错的

现在条件数目没加限制,但是 where语句校验的代码里 用的是递归,条件太多的话,会造成栈溢出

2 个赞

业务上想通过一条语句直接把结果捞出来,所以直接加在where条件上。

这边先通过业务上优化来解决这个问题;感谢帮忙排查!

可是试试 in 语句, where Invest._dst in [a,b,c…]

2 个赞

in 速度很快 :clap:

1 个赞

此话题已在最后回复的 7 天后被自动关闭。不再允许新回复。