版本:
- Nebula 1.2.0
- Exchange 1.1.0
batch 大小以及调节到20 还是报错。
这和 batch size 没什么关系
你的 Storage 节点都正常吗?
看一下storage的log
有个参数 叫 rate limit 默认 1024 目前设的是多大? 调小一些试一下
现在用的默认值
rate: {
limit: 1024
timeout: 1000
}
在每个tag的配置中有两个配置可以降低对服务端的压力:
batch: 可以适当调小 比如100
partition:可以适当调小
上面提到batch 已经设置20了,partition 是32
你们集群那个时候有可能比较繁忙 所以建议把 限流控制的小一些 比如 256
是 rate limit 这个参数吗?我刚才设置100重跑了还是一样
看下日志 主要是storage 和 meta的日志 里面有报错么
storage 日志
Log file created at: 2021/02/25 07:38:43
Running on machine: sec-ocr-serving01.py
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0225 07:38:43.186273 78 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:38:43.186669 78 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:38:43.186679 76 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:38:43.187427 76 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 50
E0225 07:38:54.417650 92 GeneratedCodeHelper.cpp:116] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E0225 07:38:54.417794 92 GeneratedCodeHelper.cpp:73] invalid message from client in function process
E0225 07:40:49.691318 76 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:40:49.697590 76 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:40:50.416115 70 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:40:56.668377 70 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:53:57.914351 77 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:53:57.916229 77 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:53:57.980991 72 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:53:57.981134 78 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:53:57.981717 78 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:53:57.986555 77 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:55:57.022992 70 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:55:57.023200 70 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
E0225 07:55:57.026242 76 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 50
E0225 07:55:57.733237 76 ExecutionPlan.cpp:80] Execute failed: Insert vertex not complete, completeness: 0
meta日志:
Log file created at: 2021/02/25 06:52:51
Running on machine: sec-ocr-serving02.py
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0225 06:52:51.592993 109 JobManager.cpp:95] [JobManager] load a invalid job from queue 0
问下 如果通过 console 单独写一条数据进去 是正确的吧?
docker-compose ps看一下 各个服务的状态 可能graph或者stroage有问题
storage 报这个错误
E0225 10:59:40.810616 64 MetaClient.cpp:110] Heartbeat failed, status:Unknown error(409): Leader changed!
E0225 10:59:53.825803 58 MetaClient.cpp:524] Send request to [10.86.87.15:8045], exceed retry limit
E0225 10:59:53.825928 64 MetaClient.cpp:110] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift21TApplicationExceptionE: Method name heartBeat not found
E0225 11:00:06.840531 59 MetaClient.cpp:524] Send request to [10.86.87.15:8045], exceed retry limit
E0225 11:00:06.840678 64 MetaClient.cpp:110] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift21TApplicationExceptionE: Method name heartBeat not found
E0225 11:00:19.854878 64 MetaClient.cpp:110] Heartbeat failed, status:Unknown error(409): Leader changed!
E0225 11:00:42.873064 50 MetaClient.cpp:524] Send request to [10.86.87.15:8045], exceed retry limit
E0225 11:00:42.873216 64 MetaClient.cpp:110] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift21TApplicationExceptionE: Method name heartBeat not found
E0225 11:00:55.886155 53 MetaClient.cpp:524] Send request to [10.86.87.15:8045], exceed retry limit
E0225 11:00:55.886286 64 MetaClient.cpp:110] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift21TApplicationExceptionE: Method name heartBeat not found
E0225 11:01:08.900907 55 MetaClient.cpp:524] Send request to [10.86.87.15:8045], exceed retry limit
E0225 11:01:08.901034 64 MetaClient.cpp:110] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift21TApplicationExceptionE: Method name heartBeat not found
E0225 11:02:21.971554 53 MetaClient.cpp:524] Send request to [10.86.87.15:8045], exceed retry limit
E0225 11:02:21.971699 64 MetaClient.cpp:110] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift21TApplicationExceptionE: Method name heartBeat not found
E0225 11:03:46.046794 64 MetaClient.cpp:110] Heartbeat failed, status:Unknown error(409): Leader changed!
E0225 11:04:09.060923 64 MetaClient.cpp:110] Heartbeat failed, status:Unknown error(409): Leader changed!
E0225 11:04:32.084555 64 MetaClient.cpp:110] Heartbeat failed, status:Unknown error(409): Leader changed!
E0225 11:04:55.110662 51 MetaClient.cpp:524] Send request to [10.86.87.15:8045], exceed retry limit
E0225 11:04:55.110800 64 MetaClient.cpp:110] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift21TApplicationExceptionE: Method name heartBeat not found
E0225 11:05:08.117949 53 MetaClient.cpp:524] Send request to [10.86.87.15:8045], exceed retry limit
E0225 11:05:08.118093 64 MetaClient.cpp:110] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift21TApplicationExceptionE: Method name heartBeat not found
E0225 11:05:21.132897 64 MetaClient.cpp:110] Heartbeat failed, status:Unknown error(409): Leader changed!
E0225 11:05:34.145349 58 MetaClient.cpp:524] Send request to [10.86.87.15:8045], exceed retry limit
E0225 11:05:34.145476 64 MetaClient.cpp:110] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift21TApplicationExceptionE: Method name heartBeat not found
E0225 11:05:47.159322 64 MetaClient.cpp:110] Heartbeat failed, status:Unknown error(409): Leader changed!
E0225 11:06:00.168772 64 MetaClient.cpp:110] Heartbeat failed, status:Unknown error(409): Leader changed!
E0225 11:06:23.187750 64 MetaClient.cpp:110] Heartbeat failed, status:Unknown error(409): Leader changed!
E0225 11:06:36.202520 64 MetaClient.cpp:110] Heartbeat failed, status:Unknown error(409): Leader changed!
E0225 11:06:49.209812 55 MetaClient.cpp:524] Send request to [10.86.87.15:8045], exceed retry limit
E0225 11:06:49.209975 64 MetaClient.cpp:110] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift21TApplicationExceptionE: Method name heartBeat not found
E0225 11:08:02.270318 64 MetaClient.cpp:110] Heartbeat failed, status:Unknown error(409): Leader changed!