nebula-graph2.0，用nebula-import导入数据后，查询报错

问答 问题
1

提问参考模版：

nebula 版本：2.0 GA
部署方式（分布式 / 单机 / Docker / DBaaS）：单机
硬件信息
磁盘（ 推荐使用 SSD）
CPU、内存信息 8核64G
问题的具体描述 用nebula-import导入数据后，查询报错
相关的 meta / storage / graph info 日志信息

nebula-import日志：

[root@host-10-33-220-159 ~]# ./nebula-importer --config configEdgenew.yaml 
2021/04/01 16:50:05 --- START OF NEBULA IMPORTER ---
2021/04/01 16:50:05 [WARN] config.go:168: You have not configured whether to remove generated temporary files, reset to default value. removeTempFiles: false
2021/04/01 16:50:05 [INFO] connection_pool.go:74: [nebula-clients] connection pool is initialized successfully
2021/04/01 16:50:05 [INFO] clientmgr.go:28: Create 10 Nebula Graph clients
2021/04/01 16:50:05 [INFO] reader.go:64: Start to read file(0): /root/vertex.csv, schema: < :VID(string),question.subject:string,question.vertexType:string,question.listingStatus:int,question.reviewStatus:int >
2021/04/01 16:50:05 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 13112, error: record on line 13112: wrong number of fields
2021/04/01 16:50:05 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 13735, error: record on line 13735: wrong number of fields
2021/04/01 16:50:05 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 13787, error: record on line 13787: wrong number of fields
2021/04/01 16:50:05 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 13795, error: record on line 13795: wrong number of fields
2021/04/01 16:50:05 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 13953, error: record on line 13953: wrong number of fields
2021/04/01 16:50:05 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 13954, error: record on line 13954: wrong number of fields
2021/04/01 16:50:05 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 13980, error: record on line 13980: wrong number of fields
2021/04/01 16:50:05 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 14011, error: record on line 14011: wrong number of fields
2021/04/01 16:50:05 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 14052, error: record on line 14052: wrong number of fields
2021/04/01 16:50:05 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 14103, error: record on line 14103: wrong number of fields
2021/04/01 16:50:05 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 14531, error: record on line 14531: wrong number of fields
2021/04/01 16:50:05 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 14649, error: record on line 14649: wrong number of fields
2021/04/01 16:50:06 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 19931, error: record on line 19931: wrong number of fields
2021/04/01 16:50:06 [ERROR] reader.go:169: Fail to read file(/root/vertex.csv) line 36381, error: record on line 36381: wrong number of fields
2021/04/01 16:50:07 [INFO] reader.go:180: Total lines of file(/root/vertex.csv) is: 192011, error lines: 14
2021/04/01 16:50:07 [INFO] statsmgr.go:61: Done(/root/vertex.csv): Time(1.96s), Finished(191997), Failed(0), Latency AVG(694us), Batches Req AVG(994us), Rows AVG(98004.26/s)
2021/04/01 16:50:08 Finish import data, consume time: 2.46s
2021/04/01 16:50:09 --- END OF NEBULA IMPORTER ---

LookUp语句：LOOKUP ON question WHERE question.vertexType == 'question’and question.listingStatus == 0 YIELD question.vertexType,question.subject,question.reviewStatus ,question.listingStatus|order by VertexID asc| limit 157700,10

报错如下：

(root@nebula) [edu_graph]>  LOOKUP ON question WHERE question.vertexType == 'question'and question.listingStatus == 0 YIELD question.vertexType,question.subject,question.reviewStatus ,question.listingStatus|order by VertexID asc| limit 157700,10
2021/04/01 16:54:10 [ERROR] Error info: read tcp 10.33.220.159:58966->10.33.220.159:9669: i/o timeout
2021/04/01 16:54:10 Loop error, read tcp 10.33.220.159:58966->10.33.220.159:9669: i/o timeout

Bye root!
Thu, 01 Apr 2021 16:54:10 CST

panic: Loop error, read tcp 10.33.220.159:58966->10.33.220.159:9669: i/o timeout

goroutine 1 [running]:
log.Panicf(0x7906b8, 0xe, 0xc000117e78, 0x1, 0x1)
	/opt/hostedtoolcache/go/1.14.15/x64/src/log/log.go:358 +0xc0
main.main()
	/home/runner/work/nebula-console/nebula-console/main.go:386 +0x4e1
2

@jievince 快来，这里有个 importer 的问题

3

lookup语句执行了多久后报错的？报错后服务状态还正常吗？

4

问题1，数据格式和 schema 不一致了，贴一下 import 的配置，和第 13112 行的数据
sed -n ‘13112, 13113p’ /root/vertex.csv

问题2：跑了多久， lookup 的数据量有多大？可以在执行后，看一下 graph 的内存，是不是 oom 了。

1 个赞
5

执行之后立马报错了，进程status正常

6 
13112,chemistry,question,0,2
13113,math,question,13113,chemistry,question,0,2
13114,math,question,1,

这部分数据确实有问题 lookup语句是分页查询157700,10的数据，数据召回的量应该不大

7

import的配置文件如下：

version: v2
description: example
clientSettings:
  retry: 3
  concurrency: 10
  channelBufferSize: 128
  space: edu_graph
  connection:
    user: root
    password: nebula
    address: 127.0.0.1:9669
logPath: question.log
files:
  - path: /root/vertex.csv
    failDataPath: /root/question.csv
    batchSize: 10
    type: csv
    csv:
      withHeader: false
      withLabel: false
    schema:
      type: vertex
      vertex:
        tags:
          - name: question
            props:
              - name: subject
                type: string
              - name: vertexType
                type: string
              - name: listingStatus
                type: int
              - name: reviewStatus
                type: int
8

确认一下 importer 日志里报告的 vertex.csv 文件中出错行的格式是否有问题

9

csv出错行的问题都解决了，导入也没报错，但是在查询的时候还是报上面的错误。。。

10

导入日志如下

2021/04/01 18:36:31 [INFO] connection_pool.go:74: [nebula-clients] connection pool is initialized successfully
2021/04/01 18:36:31 [INFO] clientmgr.go:28: Create 10 Nebula Graph clients
2021/04/01 18:36:31 [INFO] reader.go:64: Start to read file(0): /root/vertex.csv, schema: < :VID(string),question.subject:string,question.vertexType:string,question.listingStatus:int,question.reviewStatus:int >
2021/04/01 18:36:34 [INFO] reader.go:180: Total lines of file(/root/vertex.csv) is: 192011, error lines: 0
2021/04/01 18:36:34 [INFO] statsmgr.go:61: Done(/root/vertex.csv): Time(2.28s), Finished(192011), Failed(0), Latency AVG(780us), Batches Req AVG(1156us), Rows AVG(84142.76/s)
11

检查是不是 graphd 发生了 OOM，如果是的话应该是查询的语句数据量比内存大导致

12

OK

13

顺便请教下，nebula2.0GA 是可以用nebula-importer导入数据的，对吧，我这里用的是nebula-importer 2.0版本

14

我发现，新建的点，LookUp就能查出来，但是用importer导入的数据就报错了。。。

15

用fetch语句查询正常吗

16

补充一个貌似有用的日志

I0402 09:51:57.395987 15138 RebuildIndexTask.cpp:88] Building index successful, space=25, part=1
I0402 09:51:57.396040 15138 RebuildIndexTask.cpp:91] Processing operation logs, space=25, part=1
I0402 09:51:57.396327 15138 RebuildIndexTask.cpp:100] RebuildIndexTask Finished, space=25, part=1
I0402 09:51:57.396353 15138 AdminTaskManager.cpp:177] subtask of task(29, 0) finished, unfinished task 0
I0402 09:51:57.396363 15138 AdminTask.h:96] task(29, 0) finished, rc=[0]
I0402 09:51:57.396368 15138 AdminTaskProcessor.cpp:41] reportTaskFinish(), job=29, task=0, rc=SUCCEEDED
I0402 09:51:57.397199 15138 AdminTaskProcessor.cpp:59] reportTaskFinish() has bad status:, job=29, task=0, rc=Store failure!
17 
(root@nebula) [edu_graph]> FETCH PROP ON question '190000'
+-----------------------------------------------------------------------------------------------------+
| vertices_                                                                                           |
+-----------------------------------------------------------------------------------------------------+
| ("190000" :question{listingStatus: 0, reviewStatus: 2, subject: "physics", vertexType: "question"}) |
+-----------------------------------------------------------------------------------------------------+
Got 1 rows (time spent 810/1110 us)

Fri, 02 Apr 2021 10:00:52 CST

fetch正常的

18

看这个日志，感觉是存储出问题了

E0401 16:44:10.083009 10163 JobManager.cpp:527] KVStore error: -3
E0401 16:44:10.083191 10163 JobManager.cpp:527] KVStore error: -3
E0401 16:51:49.916860 10165 JobManager.cpp:527] KVStore error: -3
E0401 16:51:49.917022 10165 JobManager.cpp:527] KVStore error: -3
E0401 18:35:15.686451 10163 JobManager.cpp:527] KVStore error: -3
E0401 18:35:15.686723 10163 JobManager.cpp:527] KVStore error: -3
E0401 18:37:03.020553 10161 JobManager.cpp:527] KVStore error: -3
E0401 18:37:03.020776 10161 JobManager.cpp:527] KVStore error: -3
E0402 09:29:34.329123 10165 JobManager.cpp:527] KVStore error: -3
E0402 09:29:34.329275 10165 JobManager.cpp:527] KVStore error: -3
E0402 09:51:57.396775 10168 JobManager.cpp:527] KVStore error: -3
E0402 09:51:57.396981 10168 JobManager.cpp:527] KVStore error: -3
19

看了看代码，这个问题是因为查询space元数据信息出错了，所查询的space不存在导致。

20

感觉还是我导入数据的姿势有问题