- nebula 版本:2.6.1
- 部署方式:分布式
- 安装方式:源码编译
- 是否为线上版本:Y
- 硬件信息
- 磁盘 SSD
- CPU 64核
- 内存 265G
- 问题的具体描述
不定期的程序会挂。storaged、graphd、metad全会一起挂。 - 相关的 storage 日志信息
#### 最后挂掉之前storage的log:
E1202 00:05:24.001101 61370 AddVerticesProcessor.cpp:164] The vertex locked : tag 49, vid fa0b830beda02b9cc28f4af3b6939e03092
E1202 00:05:24.491807 61363 AddVerticesProcessor.cpp:164] The vertex locked : tag 49, vid fa0b830beda02b9cc28f4af3b6939e03092
E1202 00:05:49.775272 61390 GeneratedCodeHelper.cpp:183] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:49.775305 61388 GeneratedCodeHelper.cpp:183] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:49.775346 61388 GeneratedCodeHelper.cpp:115] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:49.775358 61388 GeneratedCodeHelper.cpp:77] invalid message from client in function process
E1202 00:05:49.775367 61390 GeneratedCodeHelper.cpp:115] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:49.775401 61390 GeneratedCodeHelper.cpp:77] invalid message from client in function process
E1202 00:05:49.775405 61391 GeneratedCodeHelper.cpp:183] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:49.775445 61391 GeneratedCodeHelper.cpp:115] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:49.775455 61391 GeneratedCodeHelper.cpp:77] invalid message from client in function process
E1202 00:05:49.776181 61375 GeneratedCodeHelper.cpp:183] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:49.776208 61375 GeneratedCodeHelper.cpp:115] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:49.776217 61375 GeneratedCodeHelper.cpp:77] invalid message from client in function process
E1202 00:05:54.779211 61387 GeneratedCodeHelper.cpp:183] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:54.779266 61387 GeneratedCodeHelper.cpp:115] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:54.779278 61387 GeneratedCodeHelper.cpp:77] invalid message from client in function process
E1202 00:05:54.779286 61390 GeneratedCodeHelper.cpp:183] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:54.779318 61390 GeneratedCodeHelper.cpp:115] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:54.779328 61390 GeneratedCodeHelper.cpp:77] invalid message from client in function process
E1202 00:05:54.779470 61389 GeneratedCodeHelper.cpp:183] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:54.779513 61389 GeneratedCodeHelper.cpp:115] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:54.779526 61389 GeneratedCodeHelper.cpp:77] invalid message from client in function process
E1202 00:05:54.781086 61374 GeneratedCodeHelper.cpp:183] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:54.781113 61374 GeneratedCodeHelper.cpp:115] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:05:54.781123 61374 GeneratedCodeHelper.cpp:77] invalid message from client in function process
E1202 00:05:59.783205 61388 HeaderServerChannel.cpp:103] Received invalid request from client: N6apache6thrift9transport19TTransportExceptionE: Header transport frame is too large: 1330664521 (hex 0x4f505449, ascii 'OPTI') (transport N6apache6thrift36PreReceivedDataAsyncTransportWrapperE, address 10.101.100.20, port 48074)
E1202 00:05:59.783265 61375 HeaderServerChannel.cpp:103] Received invalid request from client: N6apache6thrift9transport19TTransportExceptionE: Header transport frame is too large: 1330664521 (hex 0x4f505449, ascii 'OPTI') (transport N6apache6thrift36PreReceivedDataAsyncTransportWrapperE, address 10.101.100.20, port 59950)
E1202 00:05:59.783290 61388 PeekingManager.h:274] peekSuccess failed, dropping connection: N6apache6thrift9transport19TTransportExceptionE: Channel is !good()
E1202 00:05:59.783321 61389 HeaderServerChannel.cpp:103] Received invalid request from client: N6apache6thrift9transport19TTransportExceptionE: Header transport frame is too large: 1330664521 (hex 0x4f505449, ascii 'OPTI') (transport N6apache6thrift36PreReceivedDataAsyncTransportWrapperE, address 10.101.100.20, port 58916)
E1202 00:05:59.783336 61375 PeekingManager.h:274] peekSuccess failed, dropping connection: N6apache6thrift9transport19TTransportExceptionE: Channel is !good()
E1202 00:05:59.783394 61389 PeekingManager.h:274] peekSuccess failed, dropping connection: N6apache6thrift9transport19TTransportExceptionE: Channel is !good()
E1202 00:05:59.783483 61392 HeaderServerChannel.cpp:103] Received invalid request from client: N6apache6thrift9transport19TTransportExceptionE: Header transport frame is too large: 1330664521 (hex 0x4f505449, ascii 'OPTI') (transport N6apache6thrift36PreReceivedDataAsyncTransportWrapperE, address 10.101.100.20, port 41422)
E1202 00:05:59.783556 61392 PeekingManager.h:274] peekSuccess failed, dropping connection: N6apache6thrift9transport19TTransportExceptionE: Channel is !good()
E1202 00:05:59.783856 61387 HeaderServerChannel.cpp:103] Received invalid request from client: N6apache6thrift9transport19TTransportExceptionE: Header transport frame is too large: 2147483688 (hex 0x80000028) (transport N6apache6thrift36PreReceivedDataAsyncTransportWrapperE, address 10.101.100.20, port 48092)
E1202 00:05:59.783937 61387 PeekingManager.h:274] peekSuccess failed, dropping connection: N6apache6thrift9transport19TTransportExceptionE: Channel is !good()
E1202 00:05:59.783939 61374 HeaderServerChannel.cpp:103] Received invalid request from client: N6apache6thrift9transport19TTransportExceptionE: Header transport frame is too large: 2147483688 (hex 0x80000028) (transport N6apache6thrift36PreReceivedDataAsyncTransportWrapperE, address 10.101.100.20, port 59968)
E1202 00:05:59.784003 61374 PeekingManager.h:274] peekSuccess failed, dropping connection: N6apache6thrift9transport19TTransportExceptionE: Channel is !good()
E1202 00:05:59.784006 61388 HeaderServerChannel.cpp:103] Received invalid request from client: N6apache6thrift9transport19TTransportExceptionE: Header transport frame is too large: 2147483688 (hex 0x80000028) (transport N6apache6thrift36PreReceivedDataAsyncTransportWrapperE, address 10.101.100.20, port 58934)
E1202 00:05:59.784065 61388 PeekingManager.h:274] peekSuccess failed, dropping connection: N6apache6thrift9transport19TTransportExceptionE: Channel is !good()
E1202 00:05:59.784135 61385 HeaderServerChannel.cpp:103] Received invalid request from client: N6apache6thrift9transport19TTransportExceptionE: Header transport frame is too large: 2147483688 (hex 0x80000028) (transport N6apache6thrift36PreReceivedDataAsyncTransportWrapperE, address 10.101.100.20, port 41440)
E1202 00:05:59.784207 61385 PeekingManager.h:274] peekSuccess failed, dropping connection: N6apache6thrift9transport19TTransportExceptionE: Channel is !good()
E1202 00:06:01.802224 61366 AddEdgesProcessor.cpp:172] edge locked : src JH174069797R90500000000_1, type 50, rank 1, dst e878aff00df1880bb24588a683dd3cf4092
E1202 00:06:12.292932 61392 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:06:12.292968 61393 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:06:12.293056 61374 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:06:12.293159 61395 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:06:12.294559 61385 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:06:12.294615 61394 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:06:12.294634 61392 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:06:12.294756 61396 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:06:18.298079 61455 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:43.327188 61457 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:43.333395 61458 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:50.846383 61456 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:50.865031 61457 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:50.877558 61458 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:50.882467 61455 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:55.898000 61393 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:06:55.898766 61395 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:06:55.899483 61385 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:06:55.900235 61397 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:07:05.044692 61391 MetaClient.cpp:636] Send request to "172.30.93.44":1559, exceed retry limit
E1202 00:07:05.044764 61401 MetaClient.cpp:139] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:18.057801 61394 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:18.057857 61401 MetaClient.cpp:139] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:31.071550 61387 MetaClient.cpp:636] Send request to "172.30.93.47":1559, exceed retry limit
E1202 00:07:31.071609 61401 MetaClient.cpp:139] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:44.084372 61390 MetaClient.cpp:636] Send request to "172.30.93.44":1559, exceed retry limit
E1202 00:07:44.084448 61401 MetaClient.cpp:139] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
graph的log
E1202 00:06:20.654259 62481 StorageAccessExecutor.h:41] InsertVerticesExecutor failed, error E_RPC_FAILURE, part 23
E1202 00:06:20.654285 62481 StorageAccessExecutor.h:124] Storage Error: part: 23, error: E_RPC_FAILURE(-3).
E1202 00:06:20.654314 62442 QueryInstance.cpp:108] Storage Error: part: 23, error: E_RPC_FAILURE(-3).
E1202 00:06:20.741328 62388 StorageClientBase-inl.h:143] Request to "172.30.93.47":1779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:20.741398 62439 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 27
E1202 00:06:20.741432 62439 StorageAccessExecutor.h:124] Storage Error: part: 27, error: E_RPC_FAILURE(-3).
E1202 00:06:20.741466 62481 QueryInstance.cpp:108] Storage Error: part: 27, error: E_RPC_FAILURE(-3).
E1202 00:06:20.773772 62390 StorageClientBase-inl.h:143] Request to "172.30.93.46":2779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:20.773821 62481 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 23
E1202 00:06:20.773844 62481 StorageAccessExecutor.h:124] Storage Error: part: 23, error: E_RPC_FAILURE(-3).
E1202 00:06:20.773875 62456 QueryInstance.cpp:108] Storage Error: part: 23, error: E_RPC_FAILURE(-3).
E1202 00:06:20.794982 62393 StorageClientBase-inl.h:143] Request to "172.30.93.47":1779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:20.795032 62442 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E1202 00:06:20.795058 62442 StorageAccessExecutor.h:124] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:20.795086 62481 QueryInstance.cpp:108] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:20.804795 62397 StorageClientBase-inl.h:143] Request to "172.30.93.47":2779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:20.804839 62481 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 24
E1202 00:06:20.804857 62481 StorageAccessExecutor.h:124] Storage Error: part: 24, error: E_RPC_FAILURE(-3).
E1202 00:06:20.804888 62442 QueryInstance.cpp:108] Storage Error: part: 24, error: E_RPC_FAILURE(-3).
E1202 00:06:20.859989 62497 StorageClientBase-inl.h:143] Request to "172.30.93.48":3779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:20.860054 62439 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 27
E1202 00:06:20.860083 62439 StorageAccessExecutor.h:124] Storage Error: part: 27, error: E_RPC_FAILURE(-3).
E1202 00:06:20.860112 62456 QueryInstance.cpp:108] Storage Error: part: 27, error: E_RPC_FAILURE(-3).
E1202 00:06:20.861400 62513 StorageClientBase-inl.h:143] Request to "172.30.93.47":1779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:20.861443 62481 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E1202 00:06:20.861462 62481 StorageAccessExecutor.h:124] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:20.861517 62442 QueryInstance.cpp:108] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:20.862392 62508 StorageClientBase-inl.h:143] Request to "172.30.93.47":1779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:20.862444 62442 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E1202 00:06:20.862471 62442 StorageAccessExecutor.h:124] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:20.862499 62481 QueryInstance.cpp:108] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:20.965025 62516 StorageClientBase-inl.h:143] Request to "172.30.93.47":1779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:20.965076 62447 StorageAccessExecutor.h:41] InsertVerticesExecutor failed, error E_RPC_FAILURE, part 26
E1202 00:06:20.965101 62519 StorageClientBase-inl.h:143] Request to "172.30.93.46":2779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:20.965108 62447 StorageAccessExecutor.h:124] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:20.965150 62481 StorageAccessExecutor.h:41] DeleteEdgesExecutor failed, error E_RPC_FAILURE, part 23
E1202 00:06:20.965195 62481 StorageAccessExecutor.h:41] DeleteEdgesExecutor failed, error E_LEADER_CHANGED, part 1
E1202 00:06:20.965209 62481 StorageAccessExecutor.h:41] DeleteEdgesExecutor failed, error E_LEADER_CHANGED, part 24
E1202 00:06:20.965220 62481 StorageAccessExecutor.h:41] DeleteEdgesExecutor failed, error E_LEADER_CHANGED, part 29
E1202 00:06:20.965234 62481 StorageAccessExecutor.h:124] Storage Error: part: 23, error: E_RPC_FAILURE(-3).
E1202 00:06:20.965168 62442 QueryInstance.cpp:108] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:20.965258 62439 QueryInstance.cpp:108] Storage Error: part: 23, error: E_RPC_FAILURE(-3).
E1202 00:06:21.084614 62381 StorageClientBase-inl.h:143] Request to "172.30.93.47":3779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:21.084667 62481 StorageAccessExecutor.h:41] InsertVerticesExecutor failed, error E_RPC_FAILURE, part 23
E1202 00:06:21.084689 62481 StorageAccessExecutor.h:124] Storage Error: part: 23, error: E_RPC_FAILURE(-3).
E1202 00:06:21.084710 62447 QueryInstance.cpp:108] Storage Error: part: 23, error: E_RPC_FAILURE(-3).
E1202 00:06:21.105823 62382 StorageClientBase-inl.h:143] Request to "172.30.93.48":3779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:21.105876 62447 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E1202 00:06:21.105901 62447 StorageAccessExecutor.h:124] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:21.105926 62481 QueryInstance.cpp:108] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:21.115645 62383 StorageClientBase-inl.h:143] Request to "172.30.93.47":3779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:21.115697 62447 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 24
E1202 00:06:21.115720 62447 StorageAccessExecutor.h:124] Storage Error: part: 24, error: E_RPC_FAILURE(-3).
E1202 00:06:21.115743 62447 QueryInstance.cpp:108] Storage Error: part: 24, error: E_RPC_FAILURE(-3).
E1202 00:06:21.170869 62384 StorageClientBase-inl.h:143] Request to "172.30.93.48":3779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:21.170925 62469 StorageAccessExecutor.h:41] InsertVerticesExecutor failed, error E_RPC_FAILURE, part 27
E1202 00:06:21.170953 62469 StorageAccessExecutor.h:124] Storage Error: part: 27, error: E_RPC_FAILURE(-3).
E1202 00:06:21.170974 62442 QueryInstance.cpp:108] Storage Error: part: 27, error: E_RPC_FAILURE(-3).
E1202 00:06:21.175278 62389 StorageClientBase-inl.h:143] Request to "172.30.93.48":1779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:21.175320 62442 StorageAccessExecutor.h:41] InsertVerticesExecutor failed, error E_RPC_FAILURE, part 28
E1202 00:06:21.175343 62442 StorageAccessExecutor.h:124] Storage Error: part: 28, error: E_RPC_FAILURE(-3).
E1202 00:06:21.175362 62447 QueryInstance.cpp:108] Storage Error: part: 28, error: E_RPC_FAILURE(-3).
E1202 00:06:21.395512 62391 StorageClientBase-inl.h:143] Request to "172.30.93.46":2779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:21.395570 62442 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 23
E1202 00:06:21.395601 62442 StorageAccessExecutor.h:124] Storage Error: part: 23, error: E_RPC_FAILURE(-3).
E1202 00:06:21.395637 62439 QueryInstance.cpp:108] Storage Error: part: 23, error: E_RPC_FAILURE(-3).
E1202 00:06:21.416709 62392 StorageClientBase-inl.h:143] Request to "172.30.93.48":3779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:21.416764 62439 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E1202 00:06:21.416786 62439 StorageAccessExecutor.h:124] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:21.416810 62439 QueryInstance.cpp:108] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:21.426476 62393 StorageClientBase-inl.h:143] Request to "172.30.93.47":2779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:21.426534 62439 StorageAccessExecutor.h:41] InsertVerticesExecutor failed, error E_RPC_FAILURE, part 24
E1202 00:06:21.426558 62439 StorageAccessExecutor.h:124] Storage Error: part: 24, error: E_RPC_FAILURE(-3).
E1202 00:06:21.426578 62439 QueryInstance.cpp:108] Storage Error: part: 24, error: E_RPC_FAILURE(-3).
E1202 00:06:21.486099 62394 StorageClientBase-inl.h:143] Request to "172.30.93.48":3779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:21.486155 62439 StorageAccessExecutor.h:41] InsertVerticesExecutor failed, error E_RPC_FAILURE, part 26
E1202 00:06:21.486178 62439 StorageAccessExecutor.h:124] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:21.486207 62439 QueryInstance.cpp:108] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:21.799878 62487 StorageClientBase-inl.h:143] Request to "172.30.93.47":2779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:21.799934 62447 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E1202 00:06:21.799965 62447 StorageAccessExecutor.h:124] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:21.799989 62447 QueryInstance.cpp:108] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:22.031592 62490 StorageClientBase-inl.h:143] Request to "172.30.93.47":2779 failed: N6apache6thrift9transport19TTransportExceptionE: Timed Out
E1202 00:06:22.031666 62456 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_RPC_FAILURE, part 26
E1202 00:06:22.031700 62456 StorageAccessExecutor.h:124] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:22.031733 62456 QueryInstance.cpp:108] Storage Error: part: 26, error: E_RPC_FAILURE(-3).
E1202 00:06:22.370551 62456 StorageAccessExecutor.h:41] GetNeighborsExecutor failed, error E_LEADER_CHANGED, part 23
E1202 00:06:22.370775 62456 QueryInstance.cpp:108] Storage Error: The leader has changed. Try again later
E1202 00:06:43.327251 62305 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:43.333051 62306 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:50.845225 62304 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:50.863744 62305 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:50.878021 62306 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:50.882834 62303 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:06:55.897100 62516 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:07:05.046905 62499 MetaClient.cpp:636] Send request to "172.30.93.48":1559, exceed retry limit
E1202 00:07:05.047003 62395 MetaClient.cpp:139] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.047473 62492 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:05.047487 62495 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:05.047533 62439 SwitchSpaceExecutor.cpp:26] RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.047567 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.047611 62439 GraphService.cpp:86] Create session for userName: root, ip: 10.171.241.38 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.047627 62456 QueryInstance.cpp:108] RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.047756 62494 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:05.047796 62456 SwitchSpaceExecutor.cpp:26] RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.047827 62439 QueryInstance.cpp:108] RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.048182 62469 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.048341 62439 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.048641 62439 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.048913 62439 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.049010 62498 MetaClient.cpp:636] Send request to "172.30.93.47":1559, exceed retry limit
E1202 00:07:05.049048 62498 GraphSessionManager.cpp:205] Update sessions failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.049090 62396 GraphSessionManager.cpp:229] Update sessions failed: Update sessions failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.049477 62496 MetaClient.cpp:636] Send request to "172.30.93.48":1559, exceed retry limit
E1202 00:07:05.049479 62497 MetaClient.cpp:636] Send request to "172.30.93.48":1559, exceed retry limit
E1202 00:07:05.049521 62447 SwitchSpaceExecutor.cpp:26] RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.049546 62439 SwitchSpaceExecutor.cpp:26] RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.049577 62469 QueryInstance.cpp:108] RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.049592 62456 QueryInstance.cpp:108] RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:05.050154 62456 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.050164 62439 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.050576 62439 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.050648 62439 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.050773 62439 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.051007 62456 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.051151 62456 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.051404 62439 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.051410 62456 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.051656 62456 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.051682 62439 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.051946 62439 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.052141 62456 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:05.052417 62447 QueryInstance.cpp:108] SemanticError: Space was not chosen.
E1202 00:07:27.766528 62500 MetaClient.cpp:636] Send request to "172.30.93.45":1559, exceed retry limit
E1202 00:07:27.766592 62439 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:27.766644 62439 GraphService.cpp:86] Create session for userName: root, ip: 10.171.229.175 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:27.767508 62501 MetaClient.cpp:636] Send request to "172.30.93.44":1559, exceed retry limit
E1202 00:07:27.767562 62439 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:27.767591 62439 GraphService.cpp:86] Create session for userName: root, ip: 10.172.35.191 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:27.767598 62502 MetaClient.cpp:636] Send request to "172.30.93.44":1559, exceed retry limit
E1202 00:07:27.767663 62439 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:27.767689 62439 GraphService.cpp:86] Create session for userName: root, ip: 10.171.168.33 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:28.767508 62505 MetaClient.cpp:636] Send request to "172.30.93.45":1559, exceed retry limit
E1202 00:07:28.767575 62395 MetaClient.cpp:139] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:28.768496 62503 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:28.768543 62439 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:28.768550 62504 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:28.768577 62439 GraphService.cpp:86] Create session for userName: root, ip: 10.171.122.139 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:28.768632 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:28.768682 62447 GraphService.cpp:86] Create session for userName: root, ip: 10.171.9.34 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:28.768677 62506 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:28.768744 62506 GraphSessionManager.cpp:205] Update sessions failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:28.768787 62396 GraphSessionManager.cpp:229] Update sessions failed: Update sessions failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:41.780252 62507 MetaClient.cpp:636] Send request to "172.30.93.44":1559, exceed retry limit
E1202 00:07:41.780318 62395 MetaClient.cpp:139] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:41.784071 62508 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:41.784104 62508 GraphSessionManager.cpp:205] Update sessions failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:41.784133 62396 GraphSessionManager.cpp:229] Update sessions failed: Update sessions failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:46.821509 62509 MetaClient.cpp:636] Send request to "172.30.93.47":1559, exceed retry limit
E1202 00:07:46.821573 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:46.821635 62447 GraphService.cpp:86] Create session for userName: root, ip: 10.171.110.5 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:46.823532 62510 MetaClient.cpp:636] Send request to "172.30.93.48":1559, exceed retry limit
E1202 00:07:46.823603 62456 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:46.823650 62456 GraphService.cpp:86] Create session for userName: root, ip: 10.171.9.34 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.822583 62511 MetaClient.cpp:636] Send request to "172.30.93.44":1559, exceed retry limit
E1202 00:07:49.822660 62456 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.822727 62456 GraphService.cpp:86] Create session for userName: root, ip: 10.172.202.39 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.823602 62513 MetaClient.cpp:636] Send request to "172.30.93.47":1559, exceed retry limit
E1202 00:07:49.823662 62456 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.823701 62456 GraphService.cpp:86] Create session for userName: root, ip: 10.171.241.38 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.824488 62512 MetaClient.cpp:636] Send request to "172.30.93.48":1559, exceed retry limit
E1202 00:07:49.824537 62456 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.824566 62456 GraphService.cpp:86] Create session for userName: root, ip: 10.172.35.191 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.825480 62516 MetaClient.cpp:636] Send request to "172.30.93.48":1559, exceed retry limit
E1202 00:07:49.825495 62514 MetaClient.cpp:636] Send request to "172.30.93.48":1559, exceed retry limit
E1202 00:07:49.825532 62456 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.825565 62456 GraphService.cpp:86] Create session for userName: root, ip: 10.171.122.139 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.825560 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.825616 62456 GraphService.cpp:86] Create session for userName: root, ip: 10.172.5.251 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.825623 62519 MetaClient.cpp:636] Send request to "172.30.93.45":1559, exceed retry limit
E1202 00:07:49.825678 62456 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.825707 62456 GraphService.cpp:86] Create session for userName: root, ip: 10.171.168.33 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.826478 62515 MetaClient.cpp:636] Send request to "172.30.93.44":1559, exceed retry limit
E1202 00:07:49.826488 62518 MetaClient.cpp:636] Send request to "172.30.93.44":1559, exceed retry limit
E1202 00:07:49.826490 62517 MetaClient.cpp:636] Send request to "172.30.93.44":1559, exceed retry limit
E1202 00:07:49.826524 62456 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.826546 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.826584 62456 GraphService.cpp:86] Create session for userName: root, ip: 10.172.110.203 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.826579 62439 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.826602 62447 GraphService.cpp:86] Create session for userName: root, ip: 10.171.128.237 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.826650 62439 GraphService.cpp:86] Create session for userName: root, ip: 10.172.179.203 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.827462 62520 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:49.827471 62521 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:49.827482 62522 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:49.827507 62439 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.827533 62456 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.827551 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.827572 62439 GraphService.cpp:86] Create session for userName: root, ip: 10.172.239.147 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.827589 62456 GraphService.cpp:86] Create session for userName: root, ip: 10.171.235.44 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.827610 62447 GraphService.cpp:86] Create session for userName: root, ip: 10.172.80.108 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.828459 62523 MetaClient.cpp:636] Send request to "172.30.93.47":1559, exceed retry limit
E1202 00:07:49.828497 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.828526 62447 GraphService.cpp:86] Create session for userName: root, ip: 10.171.46.165 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.829466 62524 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:49.829514 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.829543 62447 GraphService.cpp:86] Create session for userName: root, ip: 10.171.127.163 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.830000 62372 MetaClient.cpp:636] Send request to "172.30.93.48":1559, exceed retry limit
E1202 00:07:49.830046 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.830075 62447 GraphService.cpp:86] Create session for userName: root, ip: 10.172.128.175 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.830485 62309 MetaClient.cpp:636] Send request to "172.30.93.44":1559, exceed retry limit
E1202 00:07:49.830492 62308 MetaClient.cpp:636] Send request to "172.30.93.44":1559, exceed retry limit
E1202 00:07:49.830528 62456 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.830549 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.830567 62456 GraphService.cpp:86] Create session for userName: root, ip: 10.172.99.180 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.830585 62447 GraphService.cpp:86] Create session for userName: root, ip: 10.172.242.225 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.834499 62374 MetaClient.cpp:636] Send request to "172.30.93.44":1559, exceed retry limit
E1202 00:07:49.834547 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.834575 62447 GraphService.cpp:86] Create session for userName: root, ip: 10.172.209.254 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.834933 62375 MetaClient.cpp:636] Send request to "172.30.93.46":1559, exceed retry limit
E1202 00:07:49.834978 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.835004 62447 GraphService.cpp:86] Create session for userName: root, ip: 10.172.71.251 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.838001 62376 MetaClient.cpp:636] Send request to "172.30.93.47":1559, exceed retry limit
E1202 00:07:49.838043 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:49.838071 62447 GraphService.cpp:86] Create session for userName: root, ip: 10.171.129.148 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:54.912139 62379 MetaClient.cpp:636] Send request to "172.30.93.47":1559, exceed retry limit
E1202 00:07:54.912161 62380 MetaClient.cpp:636] Send request to "172.30.93.47":1559, exceed retry limit
E1202 00:07:54.912335 62378 MetaClient.cpp:636] Send request to "172.30.93.47":1559, exceed retry limit
E1202 00:07:54.912532 62377 MetaClient.cpp:636] Send request to "172.30.93.47":1559, exceed retry limit
E1202 00:07:54.912912 62456 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:54.912915 62447 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:54.913012 62439 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:54.913224 62469 GraphSessionManager.cpp:102] Create session failed:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:54.913514 62456 GraphService.cpp:86] Create session for userName: root, ip: 10.171.229.175 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:54.913846 62456 GraphService.cpp:86] Create session for userName: root, ip: 10.171.9.34 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:54.913997 62442 GraphService.cpp:86] Create session for userName: root, ip: 10.171.110.5 failed: Create session failed: RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E1202 00:07:54.914260 62451 GraphService.cpp:86] Create session for userName: root, ip
挂之前meta的log
E1202 00:04:47.149371 6211 GeneratedCodeHelper.cpp:183] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:04:47.149384 6035 GeneratedCodeHelper.cpp:183] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:04:47.149425 6211 GeneratedCodeHelper.cpp:115] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:04:47.149447 6035 GeneratedCodeHelper.cpp:115] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:04:47.149454 6211 GeneratedCodeHelper.cpp:77] invalid message from client in function process
E1202 00:04:47.149468 6035 GeneratedCodeHelper.cpp:77] invalid message from client in function process
E1202 00:04:52.278714 6037 GeneratedCodeHelper.cpp:183] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:04:52.278757 6037 GeneratedCodeHelper.cpp:115] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:04:52.278769 6037 GeneratedCodeHelper.cpp:77] invalid message from client in function process
E1202 00:04:52.280206 6212 GeneratedCodeHelper.cpp:183] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:04:52.280261 6212 GeneratedCodeHelper.cpp:115] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E1202 00:04:52.280273 6212 GeneratedCodeHelper.cpp:77] invalid message from client in function process
E1202 00:04:57.526290 6038 HeaderServerChannel.cpp:103] Received invalid request from client: N6apache6thrift9transport19TTransportExceptionE: Header transport frame is too large: 1330664521 (hex 0x4f505449, ascii 'OPTI') (transport N6apache6thrift36PreReceivedDataAsyncTransportWrapperE, address 10.101.100.20, port 47534)
E1202 00:04:57.526373 6038 PeekingManager.h:274] peekSuccess failed, dropping connection: N6apache6thrift9transport19TTransportExceptionE: Channel is !good()
E1202 00:04:57.526466 6213 HeaderServerChannel.cpp:103] Received invalid request from client: N6apache6thrift9transport19TTransportExceptionE: Header transport frame is too large: 1330664521 (hex 0x4f505449, ascii 'OPTI') (transport N6apache6thrift36PreReceivedDataAsyncTransportWrapperE, address 10.101.100.20, port 57170)
E1202 00:04:57.526535 6213 PeekingManager.h:274] peekSuccess failed, dropping connection: N6apache6thrift9transport19TTransportExceptionE: Channel is !good()
E1202 00:04:57.853184 6039 HeaderServerChannel.cpp:103] Received invalid request from client: N6apache6thrift9transport19TTransportExceptionE: Header transport frame is too large: 2147483688 (hex 0x80000028) (transport N6apache6thrift36PreReceivedDataAsyncTransportWrapperE, address 10.101.100.20, port 53886)
E1202 00:04:57.853274 6039 PeekingManager.h:274] peekSuccess failed, dropping connection: N6apache6thrift9transport19TTransportExceptionE: Channel is !good()
E1202 00:04:57.853353 6214 HeaderServerChannel.cpp:103] Received invalid request from client: N6apache6thrift9transport19TTransportExceptionE: Header transport frame is too large: 2147483688 (hex 0x80000028) (transport N6apache6thrift36PreReceivedDataAsyncTransportWrapperE, address 10.101.100.20, port 35290)
E1202 00:04:57.853421 6214 PeekingManager.h:274] peekSuccess failed, dropping connection: N6apache6thrift9transport19TTransportExceptionE: Channel is !good()
E1202 00:05:10.969748 6043 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:05:10.969938 6217 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:05:11.159124 6044 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:05:11.159232 6218 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:05:11.612587 6080 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:05:36.902297 6078 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:05:36.904124 6079 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:05:44.413158 6077 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:05:44.415843 6078 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:05:44.418382 6079 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:05:44.421169 6080 HTTP2Codec.cpp:1766] Connection error PROTOCOL_ERROR with ingress=
E1202 00:05:54.970489 6027 PeekingManager.h:122] Received SSL connection on non SSL port
E1202 00:05:54.972021 6164 PeekingManager.h:122] Received SSL connection on non SSL port