无法查询，查询oom

问答 问题
43

03的日志级别调了下

image
image2882×440 139 KB

44

我怀疑 @dillon.wang 是创建索引—> rebuild ongoing/failed —> 删除索引 —> 循环了几次

中间的 rebuild job 可能有一些没有善终？

45

我也怀疑是的，这个有办法 全部停掉吗

46

但是我昨天集群，全部都stop了很久，按理说 即使有任务 他也会停掉

47

@Shylock-Hg 可能是这个造成的么？是的话，有啥办法能把状态弄好么？

48

直接stop job吧

49

全部stop了，还是不行

50

show jobs看看

51 
(root@nebula) [xygraph]> show jobs
+--------+----------------------+------------+----------------------------+----------------------------+
| Job Id | Command              | Status     | Start Time                 | Stop Time                  |
+--------+----------------------+------------+----------------------------+----------------------------+
| 484    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T07:37:21.000000 | 2022-10-26T07:37:21.000000 |
| 483    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T07:36:10.000000 | 2022-10-26T07:36:10.000000 |
| 482    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T07:33:55.000000 | 2022-10-26T07:33:55.000000 |
| 481    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T07:33:27.000000 | 2022-10-26T07:33:27.000000 |
| 480    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T07:17:56.000000 | 2022-10-26T07:18:54.000000 |
| 479    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T07:08:15.000000 | 2022-10-26T07:09:13.000000 |
| 478    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T03:08:16.000000 | 2022-10-26T03:09:14.000000 |
| 477    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T02:44:20.000000 | 2022-10-26T02:45:17.000000 |
| 476    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T02:39:53.000000 | 2022-10-26T02:40:49.000000 |
| 475    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T02:38:40.000000 | 2022-10-26T02:39:37.000000 |
| 474    | "LEADER_BALANCE"     | "FINISHED" | 2022-10-26T02:13:04.000000 | 2022-10-26T02:13:05.000000 |
| 473    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-26T02:10:27.000000 | 2022-10-26T02:12:44.000000 |
| 472    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-26T02:05:42.000000 | 2022-10-26T02:07:07.000000 |
| 471    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T01:39:42.000000 | 2022-10-26T01:40:06.000000 |
| 470    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T01:38:44.000000 | 2022-10-26T01:39:10.000000 |
| 469    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T01:33:21.000000 | 2022-10-26T01:33:45.000000 |
| 468    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-26T00:00:09.000000 | 2022-10-26T00:00:34.000000 |
| 467    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-25T23:55:07.000000 | 2022-10-25T23:55:07.000000 |
| 466    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-25T23:54:15.000000 | 2022-10-25T23:54:44.000000 |
| 465    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-25T13:04:40.000000 | 2022-10-25T13:05:05.000000 |
| 464    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-25T13:02:32.000000 | 2022-10-25T13:03:04.000000 |
| 463    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-25T07:36:20.000000 | 2022-10-25T11:47:16.000000 |
| 462    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-25T04:05:00.000000 | 2022-10-25T07:16:44.000000 |
| 461    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-25T03:51:24.000000 | 2022-10-25T03:51:24.000000 |
| 460    | "STATS"              | "FINISHED" | 2022-10-25T03:33:00.000000 | 2022-10-25T03:51:24.000000 |
| 459    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-25T03:31:34.000000 | 2022-10-25T03:31:34.000000 |
| 458    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-25T00:47:20.000000 | 2022-10-25T02:40:42.000000 |
| 457    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-24T12:17:46.000000 | 2022-10-24T15:07:49.000000 |
| 456    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-24T12:01:53.000000 | 2022-10-24T12:16:58.000000 |
| 455    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-24T11:24:15.000000 | 2022-10-24T12:01:16.000000 |
| 454    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-24T10:46:05.000000 | 2022-10-24T11:19:53.000000 |
| 453    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-24T10:44:36.000000 | 2022-10-24T10:44:36.000000 |
| 452    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-24T10:42:38.000000 | 2022-10-24T10:42:38.000000 |
| 451    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-24T10:42:17.000000 | 2022-10-24T10:42:17.000000 |
| 450    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-24T10:41:20.000000 | 2022-10-24T10:41:20.000000 |
| 449    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-24T10:32:12.000000 | 2022-10-24T10:33:12.000000 |
| 448    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-24T10:29:15.000000 | 2022-10-24T10:30:01.000000 |
| 447    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-24T10:25:50.000000 | 2022-10-24T10:26:02.000000 |
| 446    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-23T00:23:32.000000 | 2022-10-23T02:05:52.000000 |
| 445    | "STATS"              | "FINISHED" | 2022-10-23T00:06:16.000000 | 2022-10-23T00:23:32.000000 |
| 444    | "STATS"              | "FINISHED" | 2022-10-22T09:21:09.000000 | 2022-10-22T09:39:03.000000 |
| 443    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-22T00:06:27.000000 | 2022-10-22T01:12:57.000000 |
| 442    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T17:43:30.000000 | 2022-10-21T17:51:38.000000 |
| 441    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T17:35:22.000000 | 2022-10-21T17:43:30.000000 |
| 440    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T17:27:09.000000 | 2022-10-21T17:35:22.000000 |
| 439    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T17:18:14.000000 | 2022-10-21T17:27:09.000000 |
| 438    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T17:09:44.000000 | 2022-10-21T17:18:14.000000 |
| 437    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T17:01:26.000000 | 2022-10-21T17:09:44.000000 |
| 436    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T16:53:34.000000 | 2022-10-21T17:01:26.000000 |
| 435    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T16:45:24.000000 | 2022-10-21T16:53:34.000000 |
| 434    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T16:36:58.000000 | 2022-10-21T16:45:24.000000 |
| 433    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T16:22:36.000000 | 2022-10-21T16:36:58.000000 |
| 432    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T16:13:41.000000 | 2022-10-21T16:22:36.000000 |
| 431    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T16:05:30.000000 | 2022-10-21T16:13:41.000000 |
| 430    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T15:57:28.000000 | 2022-10-21T16:05:30.000000 |
| 429    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T15:49:29.000000 | 2022-10-21T15:57:28.000000 |
| 428    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T15:41:23.000000 | 2022-10-21T15:49:29.000000 |
| 427    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T15:33:27.000000 | 2022-10-21T15:41:23.000000 |
| 426    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T12:54:22.000000 | 2022-10-21T13:03:10.000000 |
| 425    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T12:45:32.000000 | 2022-10-21T12:54:22.000000 |
| 424    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T12:36:52.000000 | 2022-10-21T12:45:32.000000 |
| 423    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T12:27:32.000000 | 2022-10-21T12:36:52.000000 |
| 422    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T12:18:45.000000 | 2022-10-21T12:27:32.000000 |
| 421    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T12:09:40.000000 | 2022-10-21T12:18:45.000000 |
| 420    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T12:00:49.000000 | 2022-10-21T12:09:40.000000 |
| 419    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T11:51:08.000000 | 2022-10-21T12:00:49.000000 |
| 418    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T11:42:12.000000 | 2022-10-21T11:51:08.000000 |
| 417    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T11:33:12.000000 | 2022-10-21T11:42:12.000000 |
| 416    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T11:24:30.000000 | 2022-10-21T11:33:12.000000 |
| 415    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T11:15:40.000000 | 2022-10-21T11:24:30.000000 |
| 414    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T11:06:22.000000 | 2022-10-21T11:15:40.000000 |
| 413    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T10:57:13.000000 | 2022-10-21T11:06:22.000000 |
| 412    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T04:16:34.000000 | 2022-10-21T10:57:13.000000 |
| 411    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T04:07:28.000000 | 2022-10-21T04:16:34.000000 |
| 410    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T03:58:32.000000 | 2022-10-21T04:07:28.000000 |
| 409    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T03:49:14.000000 | 2022-10-21T03:58:32.000000 |
| 408    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T03:40:06.000000 | 2022-10-21T03:49:14.000000 |
| 407    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T03:30:49.000000 | 2022-10-21T03:40:06.000000 |
| 406    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T03:00:51.000000 | 2022-10-21T03:09:56.000000 |
| 405    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T02:51:47.000000 | 2022-10-21T03:00:51.000000 |
| 404    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T02:42:15.000000 | 2022-10-21T02:51:47.000000 |
| 403    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T02:32:38.000000 | 2022-10-21T02:42:15.000000 |
| 402    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T02:22:56.000000 | 2022-10-21T02:32:38.000000 |
| 401    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T02:13:28.000000 | 2022-10-21T02:22:56.000000 |
| 400    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T02:04:00.000000 | 2022-10-21T02:13:28.000000 |
| 399    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T01:54:31.000000 | 2022-10-21T02:04:00.000000 |
| 398    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T01:44:57.000000 | 2022-10-21T01:54:30.000000 |
| 397    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T01:35:24.000000 | 2022-10-21T01:44:57.000000 |
| 396    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T01:15:17.000000 | 2022-10-21T01:34:46.000000 |
| 395    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T01:05:35.000000 | 2022-10-21T01:15:17.000000 |
| 394    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-21T00:56:02.000000 | 2022-10-21T01:05:35.000000 |
| 393    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-20T13:16:55.000000 | 2022-10-21T00:56:02.000000 |
| 392    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-20T13:07:05.000000 | 2022-10-20T13:16:55.000000 |
| 391    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-20T12:57:19.000000 | 2022-10-20T13:07:05.000000 |
| 390    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-20T12:47:16.000000 | 2022-10-20T12:57:19.000000 |
| 389    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-20T12:37:14.000000 | 2022-10-20T12:47:16.000000 |
| 388    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-20T12:26:01.000000 | 2022-10-20T12:37:14.000000 |
| 387    | "REBUILD_EDGE_INDEX" | "FINISHED" | 2022-10-20T12:15:43.000000 | 2022-10-20T12:26:01.000000 |
| 386    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T11:38:46.000000 | 2022-10-20T12:14:33.000000 |
| 385    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:36:52.000000 | 2022-10-20T11:38:31.000000 |
| 384    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:34:56.000000 | 2022-10-20T11:36:36.000000 |
| 383    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:32:42.000000 | 2022-10-20T11:34:29.000000 |
| 382    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:30:39.000000 | 2022-10-20T11:32:14.000000 |
| 381    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:27:51.000000 | 2022-10-20T11:29:28.000000 |
| 380    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:25:38.000000 | 2022-10-20T11:27:14.000000 |
| 379    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:22:56.000000 | 2022-10-20T11:24:33.000000 |
| 378    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:21:02.000000 | 2022-10-20T11:22:39.000000 |
| 377    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:18:54.000000 | 2022-10-20T11:20:29.000000 |
| 376    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:16:58.000000 | 2022-10-20T11:18:34.000000 |
| 375    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:14:58.000000 | 2022-10-20T11:16:34.000000 |
| 374    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:12:23.000000 | 2022-10-20T11:14:01.000000 |
| 373    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:10:21.000000 | 2022-10-20T11:12:00.000000 |
| 372    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:08:25.000000 | 2022-10-20T11:10:05.000000 |
| 371    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:06:01.000000 | 2022-10-20T11:07:36.000000 |
| 370    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:03:53.000000 | 2022-10-20T11:05:31.000000 |
| 369    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T11:01:35.000000 | 2022-10-20T11:03:12.000000 |
| 368    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T10:55:13.000000 | 2022-10-20T11:00:23.000000 |
| 367    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T10:52:44.000000 | 2022-10-20T10:54:20.000000 |
| 366    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T10:50:35.000000 | 2022-10-20T10:52:23.000000 |
| 365    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T10:48:15.000000 | 2022-10-20T10:49:50.000000 |
| 364    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T10:45:16.000000 | 2022-10-20T10:46:51.000000 |
| 363    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T10:43:11.000000 | 2022-10-20T10:44:48.000000 |
| 362    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T10:40:48.000000 | 2022-10-20T10:42:23.000000 |
| 361    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T10:36:59.000000 | 2022-10-20T10:38:34.000000 |
| 360    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T09:39:00.000000 | 2022-10-20T10:32:56.000000 |
| 359    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T09:35:53.000000 | 2022-10-20T09:37:19.000000 |
| 358    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:13:26.000000 | 2022-10-26T07:13:26.000000 |
| 357    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T09:19:01.000000 | 2022-10-20T09:35:03.000000 |
| 356    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T07:50:48.000000 | 2022-10-20T08:54:26.000000 |
| 355    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 354    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 353    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 352    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 351    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 350    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 349    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 348    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 347    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 346    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 345    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 344    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 343    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 342    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 341    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 340    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 339    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 338    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 337    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 336    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:06.000000 | 2022-10-26T07:16:06.000000 |
| 335    | "REBUILD_TAG_INDEX"  | "STOPPED"  | 2022-10-26T07:16:07.000000 | 2022-10-26T07:16:07.000000 |
| 334    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:06:02.000000 | 2022-10-20T07:50:16.000000 |
| 333    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 332    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 331    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 330    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 329    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 328    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 327    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 326    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 325    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 324    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 323    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 322    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 321    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 320    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:26.000000 | 2022-10-20T07:04:26.000000 |
| 319    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:26.000000 |
| 318    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 317    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 316    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 315    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 314    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 313    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 312    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 311    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 310    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 309    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 308    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 307    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 306    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 305    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 304    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 303    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 302    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 301    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 300    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 299    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 298    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 297    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 296    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 295    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 294    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 293    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 292    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 291    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 290    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 289    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 288    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 287    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 286    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 285    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 284    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 283    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 282    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 281    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 280    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 279    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 278    | "REBUILD_EDGE_INDEX" | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 277    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 276    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 275    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 274    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 273    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 272    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 271    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 270    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 269    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 268    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 267    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 266    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 265    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 264    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 263    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 262    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 261    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:25.000000 | 2022-10-20T07:04:25.000000 |
| 260    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:24.000000 | 2022-10-20T07:04:25.000000 |
| 259    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:24.000000 | 2022-10-20T07:04:24.000000 |
| 258    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:24.000000 | 2022-10-20T07:04:24.000000 |
| 257    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:24.000000 | 2022-10-20T07:04:24.000000 |
| 256    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:24.000000 | 2022-10-20T07:04:24.000000 |
| 255    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:24.000000 | 2022-10-20T07:04:24.000000 |
| 254    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T07:04:24.000000 | 2022-10-20T07:04:24.000000 |
| 253    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T07:03:52.000000 | 2022-10-20T07:05:20.000000 |
| 252    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T07:02:08.000000 | 2022-10-20T07:03:52.000000 |
| 251    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T07:00:50.000000 | 2022-10-20T07:02:08.000000 |
| 250    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:59:37.000000 | 2022-10-20T07:00:50.000000 |
| 249    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:58:28.000000 | 2022-10-20T06:59:37.000000 |
| 248    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:57:16.000000 | 2022-10-20T06:58:28.000000 |
| 247    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:56:05.000000 | 2022-10-20T06:57:16.000000 |
| 246    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:55:01.000000 | 2022-10-20T06:56:05.000000 |
| 245    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:53:49.000000 | 2022-10-20T06:55:01.000000 |
| 244    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:52:36.000000 | 2022-10-20T06:53:49.000000 |
| 243    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:51:17.000000 | 2022-10-20T06:52:36.000000 |
| 242    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:49:35.000000 | 2022-10-20T06:51:17.000000 |
| 241    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:48:13.000000 | 2022-10-20T06:49:35.000000 |
| 240    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:46:50.000000 | 2022-10-20T06:48:13.000000 |
| 239    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:45:26.000000 | 2022-10-20T06:46:50.000000 |
| 238    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:44:03.000000 | 2022-10-20T06:45:26.000000 |
| 237    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:42:34.000000 | 2022-10-20T06:44:03.000000 |
| 236    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:41:03.000000 | 2022-10-20T06:42:34.000000 |
| 235    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:39:34.000000 | 2022-10-20T06:41:03.000000 |
| 234    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:38:03.000000 | 2022-10-20T06:39:34.000000 |
| 233    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:36:34.000000 | 2022-10-20T06:38:03.000000 |
| 232    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:35:03.000000 | 2022-10-20T06:36:34.000000 |
| 231    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:33:33.000000 | 2022-10-20T06:35:03.000000 |
| 230    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:31:53.000000 | 2022-10-20T06:33:33.000000 |
| 229    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:30:18.000000 | 2022-10-20T06:31:53.000000 |
| 228    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:28:49.000000 | 2022-10-20T06:30:18.000000 |
| 227    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:27:19.000000 | 2022-10-20T06:28:49.000000 |
| 226    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:25:50.000000 | 2022-10-20T06:27:19.000000 |
| 225    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:24:11.000000 | 2022-10-20T06:25:50.000000 |
| 224    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:21:10.000000 | 2022-10-20T06:24:11.000000 |
| 223    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:19:38.000000 | 2022-10-20T06:21:10.000000 |
| 222    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:18:04.000000 | 2022-10-20T06:19:38.000000 |
| 221    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:13:59.000000 | 2022-10-20T06:15:35.000000 |
| 220    | "REBUILD_TAG_INDEX"  | "FINISHED" | 2022-10-20T06:09:58.000000 | 2022-10-20T06:11:27.000000 |
| 219    | "REBUILD_TAG_INDEX"  | "FAILED"   | 2022-10-20T03:24:46.000000 | 2022-10-20T07:04:24.000000 |
| 218    | "STATS"              | "FINISHED" | 2022-10-20T02:50:29.000000 | 2022-10-20T02:56:59.000000 |
+--------+----------------------+------------+----------------------------+----------------------------+
52

这不是ok么，都停了

53

但是一rebuild index就faild

54

我这个问题主要是构建索引失败。你看到这些finish是我之前构建的 都成功了，现在构建的 都是faild

55

看看meta和storage的日志呢

56

这个node是说要接触到的点的数量？

57

image
image1104×331 84.7 KB

日志级别已经调过了，目前·就是这些日志
image
image1103×310 68.6 KB

59

rebuild tag index 再看看

60

image
image1602×635 25.8 KB

61

image
image1215×241 11.3 KB

62

再看一下日志

63

image
image1256×331 85.8 KB

tail -100f nebula-storaged.INFO
image
image1498×848 64.1 KB

tail -100f nebula-metad.INFO
image
image1331×1049 59.3 KB

image
image1186×400 91.6 KB