使用import导入csv文件数据报错，error: index out of range [2] with length

Flying · 2023 年4 月 13 日 08:30

报错内容：

2023/04/13 16:07:06 --- START OF NEBULA IMPORTER ---
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/wifi_type.csv/wifi_type.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/wifi_type.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/sb_mac.csv/sb_mac.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/sb_mac.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/imei.csv/imei.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/imei.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/imsi.csv/imsi.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/imsi.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/phone.csv/phone.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/phone.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/ajbh.csv/ajbh.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/ajbh.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/wifi_mac-sb_mac.csv/wifi_mac-sb_mac.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/wifi_mac-sb_mac.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/wifi_mac-phone.csv/wifi_mac-phone.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/wifi_mac-phone.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/wifi_mac-imei.csv/wifi_mac-imei.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/wifi_mac-imei.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/wifi_mac-imsi.csv/wifi_mac-imsi.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/wifi_mac-imsi.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/wifi_mac-ajbh.csv/wifi_mac-ajbh.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/wifi_mac-ajbh.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/sb_mac-phone.csv/sb_mac-phone.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/sb_mac-phone.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/sb_mac-imei.csv/sb_mac-imei.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/sb_mac-imei.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/sb_mac-imsi.csv/sb_mac-imsi.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/sb_mac-imsi.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/phone-imei.csv/phone-imei.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/phone-imei.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/phone-imsi.csv/phone-imsi.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/phone-imsi.csv
2023/04/13 16:07:06 [INFO] config.go:422: Failed data path: /data/nebula/err/phone-ajbh.csv/phone-ajbh.csv
2023/04/13 16:07:06 [INFO] config.go:428: find file: /data/nebula/jwwd/phone-ajbh.csv
2023/04/13 16:07:06 [INFO] config.go:500: files[0].schema.vertex is nil
2023/04/13 16:07:06 [INFO] config.go:500: files[1].schema.vertex is nil
2023/04/13 16:07:06 [INFO] config.go:500: files[2].schema.vertex is nil
2023/04/13 16:07:06 [INFO] config.go:500: files[3].schema.vertex is nil
2023/04/13 16:07:06 [INFO] config.go:500: files[4].schema.vertex is nil
2023/04/13 16:07:06 [INFO] config.go:500: files[5].schema.vertex is nil
2023/04/13 16:07:37 [INFO] clientmgr.go:31: Create 10 Nebula Graph clients
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/wifi_type.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/sb_mac.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/imei.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/imsi.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/phone.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/ajbh.csv
2023/04/13 16:07:37 [INFO] reader.go:49: The delimiter of /data/nebula/jwwd/imsi.csv is U+002C ','
2023/04/13 16:07:37 [INFO] reader.go:49: The delimiter of /data/nebula/jwwd/wifi_type.csv is U+002C ','
2023/04/13 16:07:37 [INFO] reader.go:49: The delimiter of /data/nebula/jwwd/ajbh.csv is U+002C ','
2023/04/13 16:07:37 [INFO] reader.go:63: Start to read file(0): /data/nebula/jwwd/wifi_type.csv, schema: < :VID(string),wifi_mac.wifi_mac:string,wifi_mac.ssid:string,wif
i_mac.wifi_type:string >2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/wifi_mac-sb_mac.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/wifi_mac-phone.csv
2023/04/13 16:07:37 [INFO] reader.go:179: Total lines of file(/data/nebula/jwwd/wifi_type.csv) is: 63, error lines: 0
2023/04/13 16:07:37 [INFO] runner.go:79: Finish to read /data/nebula/jwwd/wifi_type.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/wifi_mac-imei.csv
2023/04/13 16:07:37 [INFO] reader.go:49: The delimiter of /data/nebula/jwwd/imei.csv is U+002C ','
2023/04/13 16:07:37 [INFO] reader.go:63: Start to read file(2): /data/nebula/jwwd/imei.csv, schema: < :VID(string),imei.imei:string >
2023/04/13 16:07:37 [INFO] reader.go:63: Start to read file(3): /data/nebula/jwwd/imsi.csv, schema: < :VID(string),imsi.imsi:string >
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/wifi_mac-imsi.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/wifi_mac-ajbh.csv
2023/04/13 16:07:37 [INFO] reader.go:63: Start to read file(5): /data/nebula/jwwd/ajbh.csv, schema: < :VID(string),ajbh.ajbh:string >
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/sb_mac-phone.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/sb_mac-imei.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/sb_mac-imsi.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/phone-imei.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/phone-imsi.csv
2023/04/13 16:07:37 [INFO] runner.go:75: Start to read /data/nebula/jwwd/phone-ajbh.csv
2023/04/13 16:07:37 [INFO] runner.go:96: Waiting for stats manager done
2023/04/13 16:07:37 [INFO] reader.go:63: Start to read file(16): /data/nebula/jwwd/phone-ajbh.csv, schema: < :SRC_VID(string),:DST_VID(string),:RANK >
2023/04/13 16:07:37 [INFO] reader.go:49: The delimiter of /data/nebula/jwwd/sb_mac.csv is U+002C ','
2023/04/13 16:07:37 [INFO] reader.go:63: Start to read file(1): /data/nebula/jwwd/sb_mac.csv, schema: < :VID(string),sb_mac.sb_mac:string >
2023/04/13 16:07:37 [INFO] reader.go:49: The delimiter of /data/nebula/jwwd/phone.csv is U+002C ','
2023/04/13 16:07:37 [INFO] runner.go:79: Finish to read /data/nebula/jwwd/phone-ajbh.csv
panic: runtime error: index out of range [2] with length 2

goroutine 77 [running]:
github.com/vesoft-inc/nebula-importer/v3/pkg/config.(*Edge).FormatValues(0xc0000792c0, 0xc0001c38e0, 0x2, 0x2, 0x22, 0xc000619830, 0x0, 0x30)
	/home/runner/work/nebula-importer/nebula-importer/pkg/config/config.go:667 +0x698
github.com/vesoft-inc/nebula-importer/v3/pkg/reader.(*BatchMgr).makeEdgeInsertStmt(0xc000057500, 0xc000212000, 0x64, 0x64, 0x7fc8702d7900, 0x100000000000dfc, 0x0, 0x0)
	/home/runner/work/nebula-importer/nebula-importer/pkg/reader/batchmgr.go:309 +0x138
github.com/vesoft-inc/nebula-importer/v3/pkg/reader.(*BatchMgr).makeEdgeBatchStmt(0xc000057500, 0xc000212000, 0x64, 0x64, 0x44d60c, 0x4, 0x788e80, 0x40d201)
	/home/runner/work/nebula-importer/nebula-importer/pkg/reader/batchmgr.go:296 +0x16d
github.com/vesoft-inc/nebula-importer/v3/pkg/reader.makeStmt(0xc000212000, 0x64, 0x64, 0xc00021ac68, 0xc000613c50, 0xc00021ac98, 0x451aa5, 0xc000613c50)
	/home/runner/work/nebula-importer/nebula-importer/pkg/reader/batchmgr.go:232 +0x296
github.com/vesoft-inc/nebula-importer/v3/pkg/reader.(*BatchMgr).MakeEdgeStmt(...)
	/home/runner/work/nebula-importer/nebula-importer/pkg/reader/batchmgr.go:289
github.com/vesoft-inc/nebula-importer/v3/pkg/reader.(*Batch).requestClient(0xc0005d4e80)
	/home/runner/work/nebula-importer/nebula-importer/pkg/reader/batch.go:59 +0x299
github.com/vesoft-inc/nebula-importer/v3/pkg/reader.(*Batch).Add(0xc0005d4e80, 0x1, 0xc000617d20, 0x2, 0x2, 0x24)
	/home/runner/work/nebula-importer/nebula-importer/pkg/reader/batch.go:36 +0xb5
github.com/vesoft-inc/nebula-importer/v3/pkg/reader.(*BatchMgr).Add(0xc000057500, 0x1, 0xc000617d20, 0x2, 0x2, 0x24, 0xc00000e208, 0x0, 0x0)
	/home/runner/work/nebula-importer/nebula-importer/pkg/reader/batchmgr.go:197 +0x118
github.com/vesoft-inc/nebula-importer/v3/pkg/reader.(*FileReader).Read(0xc000061540, 0x0, 0x0, 0x0)
	/home/runner/work/nebula-importer/nebula-importer/pkg/reader/reader.go:158 +0x6bf
github.com/vesoft-inc/nebula-importer/v3/pkg/cmd.(*Runner).Run.func2(0xc00000e208, 0xc0000a0080, 0xc00017d3b0, 0xc000079780, 0xc000061540, 0xc000177740, 0x20)
	/home/runner/work/nebula-importer/nebula-importer/pkg/cmd/runner.go:82 +0xb4
created by github.com/vesoft-inc/nebula-importer/v3/pkg/cmd.(*Runner).Run
	/home/runner/work/nebula-importer/nebula-importer/pkg/cmd/runner.go:77 +0x857

配置文件如下所示：

 # 连接的 Nebula Graph 版本，连接 3.x 时设置为 v3。
version: v2

description: example

# 是否删除临时生成的日志和错误数据文件。
removeTempFiles: true

clientSettings:

  # nGQL 语句执行失败的重试次数。
  retry: 3

  # Nebula Graph 客户端并发数。
  concurrency: 10 

  # 每个 Nebula Graph 客户端的缓存队列大小。
  channelBufferSize: 128

  # 指定数据要导入的 Nebula Graph 图空间。
  space: jwwd

  # 连接信息。
  connection:
    user: root
    password: root
    address: 192.168.91.133:9669

  postStart:
    # 配置连接 Nebula Graph 服务器之后，在插入数据之前执行的一些操作。
    commands: |
      DROP SPACE IF EXISTS jwwd;
      CREATE SPACE IF NOT EXISTS jwwd(partition_num=5, replica_factor=1, vid_type=fixed_string(256));
      USE jwwd;
      CREATE TAG wifi_mac(wifi_mac string,ssid string,wifi_type string);
      CREATE TAG sb_mac(sb_mac string);
      CREATE TAG imei(imei string);
      CREATE TAG imsi(imsi string);
      CREATE TAG notel(phone string,brand string,phone_model string,phone_type string);
      CREATE TAG ajbh(ajbh string);
      CREATE TAG INDEX IF NOT EXISTS wifi_index_1 ON wifi_mac();
      CREATE TAG INDEX IF NOT EXISTS sb_index_1 ON sb_mac();
      CREATE TAG INDEX IF NOT EXISTS imei_index_1 ON imei();
      CREATE TAG INDEX IF NOT EXISTS imsi_index_1 ON imsi();
      CREATE TAG INDEX IF NOT EXISTS ajbh_index_1 ON ajbh();
      CREATE EDGE wifi_sb(wifi_type string default null,sb_type string default null);
      CREATE EDGE wifi_phone(wifi_mac string default null); 
      CREATE EDGE wifi_imei(wifi_mac string default null);
      CREATE EDGE wifi_imsi(wifi_mac string default null);
      CREATE EDGE wifi_ajxx(wifi_mac string default null,ajbh string default null,jyaq string default null);
      CREATE EDGE sb_phone(sb_mac string default null);
      CREATE EDGE sb_imei(sb_mac string default null);
      CREATE EDGE sb_imsi(sb_mac string default null);
      CREATE EDGE phone_imei(phone string default null);
      CREATE EDGE phone_imsi(phone string default null);
      CREATE EDGE phone_ajbh(phone string default null);
      
    # 执行上述命令后到执行插入数据命令之间的间隔。afterPeriod不要写错
    afterPeriod: 30s

  preStop:
    # 配置断开 Nebula Graph 服务器连接之前执行的一些操作。
    commands: |

# 错误等日志信息输出的文件路径。    
logPath: ./err/jwwd_test.log

# CSV 文件相关设置。 - path
 #一定要对齐！！！！！！一定要对齐！！！！！！！一定要对齐！！！！！！！！一定要对齐！！！！！！！！！！！一定要对齐！！！！！！！！！！！！
 #！！！！！！！！！！！！一定要对齐！！！！！！！！！！一定要对齐！！！！！！！！！！！！一定要对齐！！！！！！！！！！！！！！！！！！！！
files:

    # 数据文件的存放路径，如果使用相对路径，则会将路径和当前配置文件的目录拼接。本示例第一个数据文件为点的数据。
  - path: /data/nebula/jwwd/wifi_type.csv

    # 插入失败的数据文件存放路径，以便后面补写数据。
    failDataPath: /data/nebula/err/wifi_type.csv

    # 单批次插入数据的语句数量。
    batchSize: 100

    # 是否按顺序在文件中插入数据行。如果为 false，可以避免数据倾斜导致的导入速率降低。
    inOrder: true

    # 文件类型，当前仅支持 csv。
    type: csv
    csv:
      # 是否有表头。
      withHeader: true

      # 是否有 LABEL。
      withLabel: false

      # 指定 csv 文件的分隔符。只支持一个字符的字符串分隔符。
      delimiter: ","

    schema:
      # Schema 的类型，可选值为 vertex 和 edge。
      type: vertex

      
  - path: /data/nebula/jwwd/sb_mac.csv
    failDataPath: /data/nebula/err/sb_mac.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
      delimiter: ","
    schema:
      type: vertex	  
      
  - path: /data/nebula/jwwd/imei.csv
    failDataPath: /data/nebula/err/imei.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
      delimiter: ","
    schema:
      type: vertex	       
      
  - path: /data/nebula/jwwd/imsi.csv
    failDataPath: /data/nebula/err/imsi.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
      delimiter: ","
    schema:
      type: vertex	
      
  - path: /data/nebula/jwwd/phone.csv
    failDataPath: /data/nebula/err/phone.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
      delimiter: ","
    schema:
      type: vertex	
      
      
  - path: /data/nebula/jwwd/ajbh.csv
    failDataPath: /data/nebula/err/ajbh.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
      delimiter: ","
    schema:
      type: vertex	
      
      
##############边数据导入

  - path: /data/nebula/jwwd/wifi_mac-sb_mac.csv
    failDataPath: /data/nebula/err/wifi_mac-sb_mac.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
    schema:
      type: edge
      edge:
        name: wifi_sb
        withRanking: true   

  - path: /data/nebula/jwwd/wifi_mac-phone.csv
    failDataPath: /data/nebula/err/wifi_mac-phone.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
    schema:
      type: edge
      edge:
        name: wifi_phone
        withRanking: true  

  - path: /data/nebula/jwwd/wifi_mac-imei.csv
    failDataPath: /data/nebula/err/wifi_mac-imei.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
    schema:
      type: edge
      edge:
        name: wifi_imei
        withRanking: true  
        
  - path: /data/nebula/jwwd/wifi_mac-imsi.csv
    failDataPath: /data/nebula/err/wifi_mac-imsi.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
    schema:
      type: edge
      edge:
        name: wifi_imsi
        withRanking: true  

  - path: /data/nebula/jwwd/wifi_mac-ajbh.csv
    failDataPath: /data/nebula/err/wifi_mac-ajbh.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
    schema:
      type: edge
      edge:
        name: wifi_ajxx
        withRanking: true  
        
  - path: /data/nebula/jwwd/sb_mac-phone.csv
    failDataPath: /data/nebula/err/sb_mac-phone.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
    schema:
      type: edge
      edge:
        name: sb_phone
        withRanking: true  
        
  - path: /data/nebula/jwwd/sb_mac-imei.csv
    failDataPath: /data/nebula/err/sb_mac-imei.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
    schema:
      type: edge
      edge:
        name: sb_imei
        withRanking: true  
      
  - path: /data/nebula/jwwd/sb_mac-imsi.csv
    failDataPath: /data/nebula/err/sb_mac-imsi.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
    schema:
      type: edge
      edge:
        name: sb_imsi
        withRanking: true  
  
  - path: /data/nebula/jwwd/phone-imei.csv
    failDataPath: /data/nebula/err/phone-imei.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
    schema:
      type: edge
      edge:
        name: phone_imei
        withRanking: true  
 
  - path: /data/nebula/jwwd/phone-imsi.csv
    failDataPath: /data/nebula/err/phone-imsi.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
    schema:
      type: edge
      edge:
        name: phone_imsi
        withRanking: true  

  - path: /data/nebula/jwwd/phone-ajbh.csv
    failDataPath: /data/nebula/err/phone-ajbh.csv
    batchSize: 100
    inOrder: true
    type: csv
    csv:
      withHeader: true
      withLabel: false
    schema:
      type: edge
      edge:
        name: phone_ajbh
        withRanking: true

steam · 2023 年4 月 13 日 08:31

参考这个帖子。

Flying · 2023 年4 月 14 日 09:48

csv插入数据是报错：ErrMsg: SyntaxError: syntax error near Wi', ErrCode: -1004，是因为什么原因呐？以下是插入数据的样例： INSERT EDGE wifi_sb(wifi_type,sb_type`) VALUES “eeb97099ad79”->“ce8517968ff9”@Wi-Fi热点设备:("
Wi-Fi热点设备",“Wi-Fi终端设备”) , “eeb97099ad79”->“d2ccf303f323”@Wi-Fi热点设备:(“Wi-Fi热点设备”,“Intel Corporate”) , “eeb97099ad79”->“d46a6ac93631”@Wi-Fi热点设备:(“Wi-Fi热点设备”,“XIAMEN MEITU MOBILE TECHNOLOGY CO.LTD”) ,

steam · 2023 年4 月 14 日 09:58

你试试把所有的 Wi-Fi 热点设备，都用单引号括起来看看。

Flying · 2023 年4 月 17 日 06:30

我是用import导入的，这个是csv的数据，不能手动自己改insert语句吧，出现这种应该怎么解决呐？

steam · 2023 年4 月 17 日 06:40

这个 Wi-Fi热点设备你是不是变成了 rank 值。 nebulagraph 不支持用非 int 类型做 rank 的，你重新指定一列 int 类型的属性列，或者自己单独设置个 rank 列看看。

Flying · 2023 年4 月 18 日 08:30

好的，已解决，非常感谢！

system · 2023 年5 月 18 日 08:30

此话题已在最后回复的 30 天后被自动关闭。不再允许新回复。