Star

重建边索引Failed

用户问答 使用问题
#1

提问参考模版：

  • nebula 版本：v2.0.0-rc1
  • 部署方式：单机compose
  • 硬件信息
    • 磁盘：SSD
    • CPU、内存信息：Intel® Xeon® CPU E5-2630L v2 @ 2.40GHz、32G
  • 问题的具体描述

准备试一下LOOKUP寻找边的功能，就给givenName边建立了索引，建立成功，但重建FAILED。
1、建立索引

(root@nebula) [yago]> CREATE EDGE INDEX givenName_rank_index on givenName(rank);                                                                                
Execution succeeded (time spent 8903/9792 us)                                                                                                                   
                                                                                                                                                                
Fri, 15 Jan 2021 02:50:39 EST

2、重建索引

(root@nebula) [yago]> REBUILD EDGE INDEX givenName_rank_index;                                                                                                  
+------------+                                                                                                                                                  
| New Job Id |                                                                                                                                                  
+------------+                                                                                                                                                  
| 363        |                                                                                                                                                  
+------------+                                                                                                                                                  
Got 1 rows (time spent 5897/6824 us)                                                                                                                            
                                                                                                                                                                
Fri, 15 Jan 2021 02:51:22 EST                                                                                                                                   
                                                                                                                                                                
(root@nebula) [yago]> show job 363                                                                                                                              
+----------------+----------------------+----------+------------+------------+                                                                                  
| Job Id(TaskId) | Command(Dest)        | Status   | Start Time | Stop Time  |                                                                                  
+----------------+----------------------+----------+------------+------------+                                                                                  
| 363            | "REBUILD_EDGE_INDEX" | "FAILED" | 1610697082 | 1610697082 |                                                                                  
+----------------+----------------------+----------+------------+------------+                                                                                  
| 0              | "storaged1"          | "FAILED" | 1610697082 | 1610697114 |                                                                                  
+----------------+----------------------+----------+------------+------------+                                                                                  
| 1              | "storaged2"          | "FAILED" | 1610697082 | 1610697114 |                                                                                  
+----------------+----------------------+----------+------------+------------+                                                                                  
| 2              | "storaged0"          | "FAILED" | 1610697082 | 1610697114 |                                                                                  
+----------------+----------------------+----------+------------+------------+                                                                                  
Got 4 rows (time spent 2969/4135 us)                                                                                                                            
                                                                                                                                                                
Fri, 15 Jan 2021 02:51:33 EST

3、查询索引是否存在

(root@nebula) [yago]> show EDGE indexes                                                                                                                                                                                                                                                                                         
+------------------------+                                                                                                                                                                                                                                                                                                      
| Names                  |                                                                                                                                                                                                                                                                                                      
+------------------------+                                                                                                                                                                                                                                                                                                      
| "givenName_rank_index" |                                                                                                                                                                                                                                                                                                      
+------------------------+                                                                                                                                                                                                                                                                                                      
Got 1 rows (time spent 2884/4319 us)

4、查询边类型是否存在：

(root@nebula) [yago]> LOOKUP ON Person WHERE Person.name == "Donald_Trump" | GO FROM $-.VertexID OVER givenName YIELD $-.VertexID AS startVid, givenName._dst AS endVID | FETCH PROP ON givenName $-.startVid -> $-.endVID;                                                                                                     
+------------------------------------+------------------------------------+-----------------+----------------+                                                                                                                                                                                                                  
| givenName._src                     | givenName._dst                     | givenName._rank | givenName.rank |                                                                                                                                                                                                                  
+------------------------------------+------------------------------------+-----------------+----------------+                                                                                                                                                                                                                  
| "e51acdc4ae8125e4966b4feec468fcef" | "d6460d863cc7403c4d48eb8682d87784" | 0               | 100            |                                                                                                                                                                                                                  
+------------------------------------+------------------------------------+-----------------+----------------+                                                                                                                                                                                                                  
Got 1 rows (time spent 10211/11327 us)

5、BALANCE LEADER后重建索引失败

(root@nebula) [yago]> balance leader                                            
Execution succeeded (time spent 5060327/5060161 us)                             
                                                                                
Fri, 15 Jan 2021 02:53:13 EST                                                   
                                                                                
(root@nebula) [yago]> show hosts                                                
+-------------+-----------+-----------+--------------+---------------------+------------------------+                                                           
| Host        | Port      | Status    | Leader count | Leader distribution | Partition distribution |                                                           
+-------------+-----------+-----------+--------------+---------------------+------------------------+                                                           
| "storaged0" | 9779      | "ONLINE"  | 10           | "dns:5, yago:5"     | "dns:15, yago:15"      |                                                           
+-------------+-----------+-----------+--------------+---------------------+------------------------+                                                           
| "storaged1" | 9779      | "ONLINE"  | 10           | "dns:5, yago:5"     | "dns:15, yago:15"      |                                                           
+-------------+-----------+-----------+--------------+---------------------+------------------------+                                                           
| "storaged2" | 9779      | "ONLINE"  | 10           | "dns:5, yago:5"     | "dns:15, yago:15"      |                                                           
+-------------+-----------+-----------+--------------+---------------------+------------------------+                                                           
| "Total"     | __EMPTY__ | __EMPTY__ | 30           | "dns:15, yago:15"   | "dns:45, yago:45"      |
+-------------+-----------+-----------+--------------+---------------------+------------------------+
Got 4 rows (time spent 2492/3489 us)                                            

Fri, 15 Jan 2021 02:53:18 EST                                                   

(root@nebula) [yago]> REBUILD EDGE INDEX givenName_rank_index;
+------------+                                                                  
| New Job Id |                                                                  
+------------+                                                                  
| 365        |                                                                  
+------------+                                                                  
Got 1 rows (time spent 5889/7073 us)                                            

Fri, 15 Jan 2021 02:53:22 EST                                                   

(root@nebula) [yago]> show job 365                                              
+----------------+----------------------+----------+------------+------------+
| Job Id(TaskId) | Command(Dest)        | Status   | Start Time | Stop Time  |
+----------------+----------------------+----------+------------+------------+
| 365            | "REBUILD_EDGE_INDEX" | "FAILED" | 1610697202 | 1610697202 |
+----------------+----------------------+----------+------------+------------+
| 0              | "storaged1"          | "FAILED" | 1610697202 | 1610697234 |
+----------------+----------------------+----------+------------+------------+
| 1              | "storaged0"          | "FAILED" | 1610697202 | 1610697234 |
+----------------+----------------------+----------+------------+------------+
| 2              | "storaged2"          | "FAILED" | 1610697202 | 1610697234 |
+----------------+----------------------+----------+------------+------------+
Got 4 rows (time spent 2360/3525 us)

6、手动Compact之后重建索引失败

(root@nebula) [yago]> SUBMIT JOB COMPACT;                                                                                                                       
+------------+                                                                                                                                                  
| New Job Id |                                                                                                                                                  
+------------+                                                                                                                                                  
| 367        |                                                                                                                                                  
+------------+                                                                                                                                                  
Got 1 rows (time spent 5245/7397 us)                                                                                                                            
                                                                                
Fri, 15 Jan 2021 03:22:14 EST                                                   
                                                                                
(root@nebula) [yago]> show job 367                                              
+----------------+---------------+-----------+------------+-----------+         
| Job Id(TaskId) | Command(Dest) | Status    | Start Time | Stop Time |         
+----------------+---------------+-----------+------------+-----------+         
| 367            | "COMPACT"     | "RUNNING" | 1610698934 | 0         |         
+----------------+---------------+-----------+------------+-----------+         
Got 1 rows (time spent 3985/5294 us)                                            
                                                                                
Fri, 15 Jan 2021 03:22:26 EST  

(root@nebula) [yago]> show job 367                                                                                                                              
+----------------+---------------+------------+------------+------------+                                                                                       
| Job Id(TaskId) | Command(Dest) | Status     | Start Time | Stop Time  |                                                                                       
+----------------+---------------+------------+------------+------------+                                                                                       
| 367            | "COMPACT"     | "FINISHED" | 1610698934 | 1610699315 |                                                                                       
+----------------+---------------+------------+------------+------------+                                                                                       
| 0              | "storaged0"   | "FINISHED" | 1610699315 | 1610699348 |                                                                                       
+----------------+---------------+------------+------------+------------+                                                                                       
| 1              | "storaged2"   | "FINISHED" | 1610699315 | 1610699348 |                                                                                       
+----------------+---------------+------------+------------+------------+       
| 2              | "storaged1"   | "FINISHED" | 1610699315 | 1610699348 |       
+----------------+---------------+------------+------------+------------+       
Got 4 rows (time spent 3807/6269 us)                                            
                                                                                
Fri, 15 Jan 2021 03:30:24 EST 

(root@nebula) [yago]> REBUILD EDGE INDEX givenName_rank_index;                  
+------------+                                                                  
| New Job Id |                                                                  
+------------+                                                                  
| 368        |                                                                  
+------------+                                                                  
Got 1 rows (time spent 6720/8729 us)                                            
                                                                                
Fri, 15 Jan 2021 03:39:07 EST                                                   
                                                                                
(root@nebula) [yago]> show job 368                                              
+----------------+----------------------+----------+------------+------------+
| Job Id(TaskId) | Command(Dest)        | Status   | Start Time | Stop Time  |  
+----------------+----------------------+----------+------------+------------+  
| 368            | "REBUILD_EDGE_INDEX" | "FAILED" | 1610699947 | 1610699947 |  
+----------------+----------------------+----------+------------+------------+  
| 0              | "storaged2"          | "FAILED" | 1610699947 | 1610699980 |  
+----------------+----------------------+----------+------------+------------+
| 1              | "storaged1"          | "FAILED" | 1610699947 | 1610699980 |
+----------------+----------------------+----------+------------+------------+  
| 2              | "storaged0"          | "FAILED" | 1610699947 | 1610699980 |
+----------------+----------------------+----------+------------+------------+  
Got 4 rows (time spent 1950/2883 us)                                            
                                                                                
Fri, 15 Jan 2021 03:39:14 EST
#2

storage0日志如下(本机时间02:52，日志时间多五个小时，是07:52)：

I0115 07:52:32.783504    28 AdminTask.cpp:19] createAdminTask (364, 0)
I0115 07:52:32.784533    28 AdminTaskManager.cpp:35] enqueue task(364, 0), con req=2147483647                                                                   
I0115 07:52:32.784584   119 AdminTaskManager.cpp:101] dequeue task(364, 0)
E0115 07:52:32.785440   119 MetaClient.cpp:2065] Space 1, Tag Index 362 not found!
E0115 07:52:32.785503   119 RebuildIndexTask.cpp:38] Index not found
E0115 07:52:32.785539   119 AdminTaskManager.cpp:115] job 364, genSubTask failed, err=-37
I0115 07:52:32.785764   119 AdminTask.h:96] task(364, 0) finished, rc=[-37]
I0115 07:52:32.786020   119 AdminTaskManager.cpp:89] waiting for incoming task

I0115 07:53:22.513033    40 AdminTask.cpp:19] createAdminTask (365, 0)
I0115 07:53:22.514176    40 AdminTaskManager.cpp:35] enqueue task(365, 0), con req=2147483647
I0115 07:53:22.514235   119 AdminTaskManager.cpp:101] dequeue task(365, 0)
E0115 07:53:22.514828   119 MetaClient.cpp:2065] Space 1, Tag Index 362 not found!
E0115 07:53:22.514884   119 RebuildIndexTask.cpp:38] Index not found
E0115 07:53:22.514935   119 AdminTaskManager.cpp:115] job 365, genSubTask failed, err=-37
I0115 07:53:22.515022   119 AdminTask.h:96] task(365, 0) finished, rc=[-37]
I0115 07:53:22.515108   119 AdminTaskManager.cpp:89] waiting for incoming task
I0115 07:57:35.738512    71 EventListner.h:18] Rocksdb start compaction column family: default because of LevelL0FilesNum, status: OK, compacted 8 files into 0, base level is 0, output level is 1
I0115 07:57:35.739439    71 CompactionFilter.h:66] Do default minor compaction!
I0115 07:57:41.635056    71 EventListner.h:30] Rocksdb compaction completed column family: default because of LevelL0FilesNum, status: OK, compacted 8 files into 4, base level is 0, output level is 1

看起来是子任务在寻找一个TAGindex没有找到，然后就failed了，寻求解决方案 :pray:

#3

问题2：在Person上建立TAG INDEX失败，报错[ERROR (-8)]: Request to storage failed, without failedCodes.，在新数据上建立TAG INDEX成功：

(root@nebula) [yago]> CREATE TAG INDEX person_only_index on Person();                                                                                           
[ERROR (-8)]: existed!                                                                                                                                          
                                                                                                                                                                
Fri, 15 Jan 2021 04:11:57 EST

(root@nebula) [yago]> CREATE TAG player(name string,age int);                                                                                                   
Execution succeeded (time spent 9069/10201 us)                                                                                                                  
                                                                                                                                                                
Fri, 15 Jan 2021 04:13:03 EST                                                                                                                                   
                                                                                                                                                                
(root@nebula) [yago]> CREATE TAG INDEX player_index on player();                                                                                                
Execution succeeded (time spent 7084/8058 us)                                                                                                                   
                                                                                                                                                                
Fri, 15 Jan 2021 04:13:15 EST                                                                                                                                   
                                                                                                                                                                
(root@nebula) [yago]> INSERT VERTEX player(name,age) VALUES "player100":("Tim Duncan", 42), "player101":("Tony Parker", 36);                                    
[ERROR (-8)]: Request to storage failed, without failedCodes.                                                                                                   
                                                                                                                                                                
Fri, 15 Jan 2021 04:13:30 EST                                                                                                                                   
                                                                                                                                                                
(root@nebula) [yago]> INSERT VERTEX player(name,age) VALUES "player100":("Tim Duncan", 42), "player101":("Tony Parker", 36);                                    
Execution succeeded (time spent 4744/5877 us)                                                                                                                   
                                                                                                                                                                
Fri, 15 Jan 2021 04:13:35 EST                                                                                                                                   
                                                                                                                                                                
(root@nebula) [yago]> LOOKUP ON player;                                                                                                                         
+-------------+                                                                                                                                                 
| VertexID    |                                                                                                                                                 
+-------------+                                                                                                                                                 
| "player101" |                                                                                                                                                 
+-------------+                                                                                                                                                 
| "player100" |                                                                                                                                                 
+-------------+                                                                                                                                                 
Got 2 rows (time spent 5451/6424 us)                                                                                                                            
                                                                                                                                                                
Fri, 15 Jan 2021 04:15:09 EST                                                                                                                                   
                                                                                                                                                                
(root@nebula) [yago]> CREATE TAG INDEX person_index_oo on Person();                                                                                             
[ERROR (-8)]: existed!

由于当前有一个Person(name(100))的TAG INDEX，初步猜想是，无法在拥有带属性的INDEX建立之后建立无属性INDEX。

#5

关于问题2索引的问题，你的猜想是正确的，nebula2.x版本，根据文档
假如一个tag有属性c1、c2、c3，那么创建该tag的索引应该符合如下规则：

  • 如果已经建立了c1的索引，那么就不能单独在c1上建立索引，重复建立索引是浪费空间的。
  • 同时，假如你已经基于c1、c2建立了复合索引，那么也不能单独在c1上再建索引，以为基于我们的设计原理，这也属于重复建立索引浪费了。
  • 所以同理，如果你已经建了一个属性索引，再建一个空属性索引是不允许的，因为浪费了。
    这里我让同事优化下文档描述，避免带来困扰:handshake:
#6

这个问题我让相关同事看一下，请稍等。

#7

关于问题2，如楼上Jerry同学所说，我们的index是左匹配index，如果tag关联的索引存在，则不允许创建无属性的索引，例如

image
image1236×916 42.5 KB

1赞
#8

这个问题猜测是storage和graph不是连的同一个meta，请验证。

1赞
#9

好的好的，谢谢，但是还是有三个问题：

  1. 如何查看storage和graph分别连接的是那个meta呢？
  2. 如果连接的不是一个meta如何修改？
  3. 之前建立TAG索引和重建都是成功的，到边索引这里如果出现这种问题有可能是我的什么操作导致的呢？
#10

同问，如何验证是不是连的同一个meta

#11
  1. 可以查看 etc/nebula-storage.conf 和 etc/nebula-graph.conf 的配置文件信息
  2. 重启一下服务，可以参考下我们的 常见问题
  3. 鉴于上面的问题，不太清除你指的索引是上面问题，可以再描述下。
#12

可以查看 etc/nebula-storage.conf 和 etc/nebula-graph.conf 的配置文件信息

浙ICP备20010487号