importer导入边数据时使用星号（*）进行模糊匹配却只导入了最后一个边文件

xiaotaozi · 2021 年9 月 13 日 08:37

nebula 版本：（2.5.0）
部署方式（分布式）：
是否为线上版本：N
问题的具体描述
边文件一共有23个，不含表头，每个文件的格式一模一样。但是使用星号（*）进行模糊匹配却只导入了最后一个边文件

yaml文件内容（有删减）如下：

# 连接的Nebula Graph版本，连接2.x时设置为v2。
version: v2

description: import_uid_tid_rel_0628

# 是否删除临时生成的日志和错误数据文件。
removeTempFiles: false

clientSettings:

  # nGQL语句执行失败的重试次数。
  retry: 1

  # Nebula Graph客户端并发数。设置为 CPU 核数
  concurrency: 24

  # 每个Nebula Graph客户端的缓存队列大小。
  channelBufferSize: 2560

  # 指定数据要导入的Nebula Graph图空间。
  space: uid_sim_0628

  # 连接信息。
  connection:
    user: root
    password: ****
    address: 10.30.10.113:9669

  postStart:
    # 配置连接Nebula Graph服务器之后，在插入数据之前执行的一些操作。
    commands: |
      DROP SPACE IF EXISTS uid_sim_0628;
      CREATE SPACE IF NOT EXISTS uid_sim_0628(partition_num=15, replica_factor=1, vid_type=INT) ;
      USE uid_sim_0628;
      CREATE TAG uid();
      CREATE TAG mcn();
      CREATE TAG blk_uid(flag int);
      CREATE TAG wt_uid(flag int);
      CREATE TAG grey_uid(flag int);
      CREATE EDGE sim(tidList string,sim_cos_dacay1 float,sim_jac_dacay1 float,sim_jac float,uid_tid_node_cnt int,uid_tid_cnt_start int,uid_tid_cnt_end int);

    # 执行上述命令后到执行插入数据命令之间的间隔。
    afterPeriod: 15s

  preStop:
    # 配置断开Nebula Graph服务器连接之前执行的一些操作。
    commands: |

# 错误等日志信息输出的文件路径。    
logPath: ./err/test3output.log

# 1.CSV文件相关设置。
files:

    # 数据文件的存放路径，如果使用相对路径，则会将路径和当前配置文件的目录拼接。本示例第一个数据文件为点的数据。
  - path: ./data/data_uid.csv

    # 插入失败的数据文件存放路径，以便后面补写数据。
    failDataPath: ./err/data_uid.csv

    # 单批次插入数据的语句数量。
    batchSize: 1000

    # 读取数据的行数限制。
    #limit: 1000

    # 是否按顺序在文件中插入数据行。如果为false，可以避免数据倾斜导致的导入速率降低。
    inOrder: false

    # 文件类型，当前仅支持csv。
    type: csv

    csv:
      # 是否有表头。
      withHeader: true

      # 是否有LABEL。
      withLabel: false

      # 指定csv文件的分隔符。只支持一个字符的字符串分隔符。
      delimiter: ","

    schema:
      type: vertex
      vertex:
        tags:
          - name: uid

        

# 2.CSV文件相关设置。
    # 数据文件的存放路径，如果使用相对路径，则会将路径和当前配置文件的目录拼接。本示例第一个数据文件为点的数据。
  - path: ./data/del_rel_intersecTidCnt_equal2_*.csv

    # 插入失败的数据文件存放路径，以便后面补写数据。
    failDataPath: ./err/del_rel_intersecTidCnt_equal2_err.csv

    # 单批次插入数据的语句数量。
    batchSize: 1000

    # 读取数据的行数限制。
    #limit: 1000

    # 是否按顺序在文件中插入数据行。如果为false，可以避免数据倾斜导致的导入速率降低。
    inOrder: false

    # 文件类型，当前仅支持csv。
    type: csv

    csv:
      # 是否有表头。
      withHeader: false

      # 是否有LABEL。
      withLabel: false

      # 指定csv文件的分隔符。只支持一个字符的字符串分隔符。
      delimiter: ","


    schema:
      type: edge
      edge:
        name: sim
        withRanking: false
        srcVID:
          type: int
          index: 0
          # function: hash
        dstVID:
          type: int
          index: 1
          # function: uuid
        props:
          - name: tidList
            type: string
            index: 2
          - name: sim_cos_dacay1
            type: float
            index: 3
          - name: sim_jac_dacay1
            type: float
            index: 4  
          - name: sim_jac
            type: float
            index: 5 
          - name: uid_tid_node_cnt
            type: int
            index: 6
          - name: uid_tid_cnt_start
            type: int
            index: 7
          - name: uid_tid_cnt_end
            type: int
            index: 8

输出日志如下：

2021/09/13 15:26:58 — START OF NEBULA IMPORTER —
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/data_uid.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_00.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_01.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_02.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_03.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_04.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_05.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_06.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_07.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_08.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_09.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_10.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_11.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_12.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_13.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_14.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_15.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_16.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_17.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_18.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_19.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_20.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_21.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_22.csv
2021/09/13 15:26:58 [INFO] config.go:380: find file: /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv
2021/09/13 15:26:58 [INFO] connection_pool.go:79: [nebula-clients] connection pool is initialized successfully
2021/09/13 15:27:14 [INFO] clientmgr.go:28: Create 72 Nebula Graph clients
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/data_uid.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(0): /data1/idata/uid_tid_rel/data/data_uid.csv, schema: < :VID(int) >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(8): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(9): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(10): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >

2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(11): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(12): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(13): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(20): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(17): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(18): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(15): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(16): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(19): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(14): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(21): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(22): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(23): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(24): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(31): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(27): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(25): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(26): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(29): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(28): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:27:14 [INFO] reader.go:26: The delimiter of /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv is U+002C ‘,’
2021/09/13 15:27:14 [INFO] reader.go:64: Start to read file(30): /data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv, schema: < :SRC_VID(int),:DST_VID(int),sim.tidList:string,sim.sim_cos_dacay1:float,sim.sim_jac_dacay1:float,sim.sim_jac:float,sim.uid_tid_node_cnt:int,sim.uid_tid_cnt_start:int,sim.uid_tid_cnt_end:int >
2021/09/13 15:28:08 [INFO] statsmgr.go:61: Tick: Time(70.01s), Finished(23546739), Failed(0), Latency AVG(90975us), Batches Req AVG(154168us), Rows AVG(336343.10/s)
2021/09/13 15:28:13 [INFO] statsmgr.go:61: Tick: Time(75.00s), Finished(25786739), Failed(0), Latency AVG(91191us), Batches Req AVG(154563us), Rows AVG(343805.03/s)
2021/09/13 15:28:18 [INFO] statsmgr.go:61: Tick: Time(80.02s), Finished(27772739), Failed(0), Latency AVG(92452us), Batches Req AVG(155955us), Rows AVG(347069.47/s)
2021/09/13 15:28:23 [INFO] statsmgr.go:61: Tick: Time(85.01s), Finished(29728739), Failed(0), Latency AVG(93784us), Batches Req AVG(157273us), Rows AVG(349691.96/s)
2021/09/13 15:28:28 [INFO] statsmgr.go:61: Tick: Time(90.02s), Finished(31863739), Failed(0), Latency AVG(94123us), Batches Req AVG(157584us), Rows AVG(353961.81/s)
2021/09/13 15:28:33 [INFO] statsmgr.go:61: Tick: Time(95.01s), Finished(33956739), Failed(0), Latency AVG(94385us), Batches Req AVG(157671us), Rows AVG(357408.55/s)
2021/09/13 15:28:38 [INFO] statsmgr.go:61: Tick: Time(100.03s), Finished(36004739), Failed(0), Latency AVG(95091us), Batches Req AVG(158122us), Rows AVG(359949.92/s)
2021/09/13 15:28:43 [INFO] statsmgr.go:61: Tick: Time(105.02s), Finished(37910739), Failed(0), Latency AVG(95713us), Batches Req AVG(158948us), Rows AVG(360991.93/s)
2021/09/13 15:28:49 [INFO] statsmgr.go:61: Tick: Time(110.30s), Finished(39872739), Failed(0), Latency AVG(96929us), Batches Req AVG(160347us), Rows AVG(361502.36/s)
2021/09/13 15:28:53 [INFO] statsmgr.go:61: Tick: Time(115.01s), Finished(41838739), Failed(0), Latency AVG(97210us), Batches Req AVG(160521us), Rows AVG(363785.56/s)
2021/09/13 15:28:58 [INFO] statsmgr.go:61: Tick: Time(120.01s), Finished(43915739), Failed(0), Latency AVG(97008us), Batches Req AVG(160705us), Rows AVG(365941.49/s)
2021/09/13 15:29:03 [INFO] statsmgr.go:61: Tick: Time(125.00s), Finished(45844739), Failed(0), Latency AVG(98003us), Batches Req AVG(161542us), Rows AVG(366747.64/s)
2021/09/13 15:29:08 [INFO] statsmgr.go:61: Tick: Time(130.01s), Finished(47806739), Failed(0), Latency AVG(98482us), Batches Req AVG(161995us), Rows AVG(367719.56/s)
2021/09/13 15:29:13 [INFO] statsmgr.go:61: Tick: Time(135.02s), Finished(50099739), Failed(0), Latency AVG(98315us), Batches Req AVG(161722us), Rows AVG(371051.06/s)
2021/09/13 15:29:18 [INFO] statsmgr.go:61: Tick: Time(140.02s), Finished(52115739), Failed(0), Latency AVG(98364us), Batches Req AVG(162002us), Rows AVG(372214.30/s)
2021/09/13 15:29:20 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/data_uid.csv) is: 9433062, error lines: 0
2021/09/13 15:29:24 [INFO] statsmgr.go:61: Tick: Time(146.06s), Finished(54557739), Failed(0), Latency AVG(98663us), Batches Req AVG(162433us), Rows AVG(373529.39/s)
2021/09/13 15:29:28 [INFO] statsmgr.go:61: Tick: Time(150.03s), Finished(56146739), Failed(0), Latency AVG(98846us), Batches Req AVG(162532us), Rows AVG(374246.35/s)
2021/09/13 15:29:33 [INFO] statsmgr.go:61: Tick: Time(155.02s), Finished(58386122), Failed(0), Latency AVG(98383us), Batches Req AVG(161980us), Rows AVG(376624.31/s)
2021/09/13 15:29:38 [INFO] statsmgr.go:61: Tick: Time(160.01s), Finished(60665433), Failed(0), Latency AVG(97870us), Batches Req AVG(161503us), Rows AVG(379136.07/s)
2021/09/13 15:29:43 [INFO] statsmgr.go:61: Tick: Time(165.01s), Finished(62939433), Failed(0), Latency AVG(97389us), Batches Req AVG(161055us), Rows AVG(381431.20/s)
2021/09/13 15:29:48 [INFO] statsmgr.go:61: Tick: Time(170.01s), Finished(65302433), Failed(0), Latency AVG(96920us), Batches Req AVG(160489us), Rows AVG(384110.38/s)
2021/09/13 15:29:53 [INFO] statsmgr.go:61: Tick: Time(175.01s), Finished(67515668), Failed(0), Latency AVG(96156us), Batches Req AVG(160118us), Rows AVG(385792.00/s)
2021/09/13 15:29:55 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/data_uid.csv): Time(177.19s), Finished(68483800), Failed(0), Latency AVG(95980us), Batches Req AVG(159974us), Rows AVG(386508.71/s)
2021/09/13 15:29:59 [INFO] statsmgr.go:61: Tick: Time(180.73s), Finished(70014800), Failed(0), Latency AVG(95847us), Batches Req AVG(159903us), Rows AVG(387393.62/s)
2021/09/13 15:30:03 [INFO] statsmgr.go:61: Tick: Time(185.01s), Finished(71986800), Failed(0), Latency AVG(95575us), Batches Req AVG(159639us), Rows AVG(389098.82/s)
2021/09/13 15:30:08 [INFO] statsmgr.go:61: Tick: Time(190.00s), Finished(74188800), Failed(0), Latency AVG(95126us), Batches Req AVG(159388us), Rows AVG(390460.52/s)
2021/09/13 15:30:13 [INFO] statsmgr.go:61: Tick: Time(195.00s), Finished(76468800), Failed(0), Latency AVG(94528us), Batches Req AVG(158783us), Rows AVG(392140.86/s)
2021/09/13 15:30:18 [INFO] statsmgr.go:61: Tick: Time(200.01s), Finished(78490800), Failed(0), Latency AVG(94697us), Batches Req AVG(158904us), Rows AVG(392428.95/s)
2021/09/13 15:30:24 [INFO] statsmgr.go:61: Tick: Time(205.44s), Finished(80377800), Failed(0), Latency AVG(95787us), Batches Req AVG(159926us), Rows AVG(391245.02/s)
2021/09/13 15:30:28 [INFO] statsmgr.go:61: Tick: Time(210.00s), Finished(81811800), Failed(0), Latency AVG(96867us), Batches Req AVG(160938us), Rows AVG(389575.79/s)
2021/09/13 15:30:33 [INFO] statsmgr.go:61: Tick: Time(215.01s), Finished(83433800), Failed(0), Latency AVG(98133us), Batches Req AVG(161967us), Rows AVG(388049.21/s)
2021/09/13 15:30:36 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:37 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:38 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:38 [INFO] statsmgr.go:61: Tick: Time(220.03s), Finished(85325688), Failed(0), Latency AVG(98623us), Batches Req AVG(162355us), Rows AVG(387796.26/s)
2021/09/13 15:30:39 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:40 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:40 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:40 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:41 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:41 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:41 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:41 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:41 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:41 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:41 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:42 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:42 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:42 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:42 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:43 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:43 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:43 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:43 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:43 [INFO] statsmgr.go:61: Tick: Time(225.01s), Finished(87485388), Failed(0), Latency AVG(99272us), Batches Req AVG(162172us), Rows AVG(388806.45/s)
2021/09/13 15:30:43 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:43 [INFO] reader.go:180: Total lines of file(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv) is: 3346988, error lines: 0
2021/09/13 15:30:48 [INFO] statsmgr.go:61: Tick: Time(230.00s), Finished(90020960), Failed(0), Latency AVG(98422us), Batches Req AVG(159513us), Rows AVG(391394.77/s)
2021/09/13 15:30:53 [INFO] statsmgr.go:61: Tick: Time(235.00s), Finished(91880651), Failed(0), Latency AVG(97573us), Batches Req AVG(157643us), Rows AVG(390979.96/s)
2021/09/13 15:30:55 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(236.34s), Finished(92425308), Failed(0), Latency AVG(97223us), Batches Req AVG(156970us), Rows AVG(391074.02/s)
2021/09/13 15:30:55 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(236.67s), Finished(92551492), Failed(0), Latency AVG(97130us), Batches Req AVG(156794us), Rows AVG(391052.18/s)
2021/09/13 15:30:55 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.08s), Finished(92703141), Failed(0), Latency AVG(97007us), Batches Req AVG(156559us), Rows AVG(391022.94/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.32s), Finished(92791268), Failed(0), Latency AVG(96924us), Batches Req AVG(156412us), Rows AVG(390992.56/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.47s), Finished(92847992), Failed(0), Latency AVG(96865us), Batches Req AVG(156305us), Rows AVG(390984.00/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.63s), Finished(92901801), Failed(0), Latency AVG(96804us), Batches Req AVG(156198us), Rows AVG(390958.34/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.64s), Finished(92908659), Failed(0), Latency AVG(96794us), Batches Req AVG(156182us), Rows AVG(390966.47/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.72s), Finished(92931809), Failed(0), Latency AVG(96765us), Batches Req AVG(156132us), Rows AVG(390937.51/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.74s), Finished(92941696), Failed(0), Latency AVG(96754us), Batches Req AVG(156112us), Rows AVG(390939.01/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.85s), Finished(92983133), Failed(0), Latency AVG(96700us), Batches Req AVG(156022us), Rows AVG(390933.15/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.86s), Finished(92986563), Failed(0), Latency AVG(96695us), Batches Req AVG(156013us), Rows AVG(390926.75/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.87s), Finished(92991020), Failed(0), Latency AVG(96689us), Batches Req AVG(156003us), Rows AVG(390926.23/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.90s), Finished(92998879), Failed(0), Latency AVG(96677us), Batches Req AVG(155983us), Rows AVG(390915.55/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.92s), Finished(93002365), Failed(0), Latency AVG(96674us), Batches Req AVG(155978us), Rows AVG(390900.50/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.94s), Finished(93008766), Failed(0), Latency AVG(96665us), Batches Req AVG(155962us), Rows AVG(390888.38/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.97s), Finished(93012680), Failed(0), Latency AVG(96659us), Batches Req AVG(155952us), Rows AVG(390852.80/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(237.98s), Finished(93013166), Failed(0), Latency AVG(96658us), Batches Req AVG(155950us), Rows AVG(390844.34/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(238.00s), Finished(93013652), Failed(0), Latency AVG(96657us), Batches Req AVG(155949us), Rows AVG(390816.01/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(238.01s), Finished(93015138), Failed(0), Latency AVG(96655us), Batches Req AVG(155946us), Rows AVG(390809.27/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(238.03s), Finished(93018110), Failed(0), Latency AVG(96652us), Batches Req AVG(155940us), Rows AVG(390786.24/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(238.03s), Finished(93019568), Failed(0), Latency AVG(96649us), Batches Req AVG(155935us), Rows AVG(390781.60/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(238.06s), Finished(93022540), Failed(0), Latency AVG(96645us), Batches Req AVG(155929us), Rows AVG(390755.88/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(238.06s), Finished(93023026), Failed(0), Latency AVG(96644us), Batches Req AVG(155927us), Rows AVG(390748.38/s)
2021/09/13 15:30:56 [INFO] statsmgr.go:61: Done(/data1/idata/uid_tid_rel/data/del_rel_intersecTidCnt_equal2_23.csv): Time(238.07s), Finished(93023512), Failed(0), Latency AVG(96643us), Batches Req AVG(155925us), Rows AVG(390741.81/s)
2021/09/13 15:30:57 Finish import data, consume time: 238.57s
2021/09/13 15:30:58 — END OF NEBULA IMPORTER —

yee · 2021 年9 月 13 日 08:53

能否贴一下你的 yaml 配置？

具体使用的样例可以参看 https://github.com/vesoft-inc/nebula-importer/blob/master/examples/v2/example.yaml#L152-L166

xiaotaozi · 2021 年9 月 13 日 08:58

已经在问题描述里贴上了，麻烦再看看

yee · 2021 年9 月 13 日 09:32

好的，我本地先看一下，看上面的问题是文件列表获取到了，但是导入的时候好像都是读的同一个文件

yee · 2021 年9 月 13 日 12:12

问题已经定位是个 bug，fix 的 PR 已经提：https://github.com/vesoft-inc/nebula-importer/pull/161

谢谢你的反馈。后面会跟其他同学沟通保证测试质量。

xiaotaozi · 2021 年9 月 14 日 02:44

收到，如果修正好了，烦请再回复一下

luciaark · 2021 年9 月 17 日 02:51

我用最新mater分支编译，运行出错啊
[snsoadmin@dorisbe1 DD]$ ./nebula-importer --config aikg_goodstheme.yaml
2021/09/17 10:47:51 — START OF NEBULA IMPORTER —
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00000-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00000-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00001-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00001-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00002-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00002-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00003-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00003-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00004-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00004-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00005-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00005-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00006-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00006-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00007-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00007-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00008-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00008-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00009-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00009-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00010-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00010-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00011-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00011-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00012-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00012-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00013-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00013-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00014-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00014-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00015-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00015-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00016-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00016-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00017-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00017-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00018-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00018-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00019-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00019-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00020-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00020-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00021-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00021-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00022-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00022-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00023-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00023-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00024-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00024-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00025-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00025-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00026-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00026-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00027-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00027-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00028-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00028-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00029-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00029-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00030-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00030-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00031-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00031-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00032-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00032-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00033-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00033-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00034-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00034-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00035-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00035-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00036-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00036-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00037-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00037-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00038-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00038-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00039-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00039-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00040-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00040-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00041-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00041-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00042-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00042-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00043-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00043-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00044-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00044-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00045-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00045-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00046-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00046-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00047-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00047-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00048-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00048-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:387: Failed data path: err/Brand.csv/download?filename=part-00049-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] config.go:400: find file: /opt/nebula_graph/DD/data/Brand/download?filename=part-00049-4fa93e4a-b75c-47b9-8c69-983f4f406456-c000.csv
2021/09/17 10:47:51 [INFO] connection_pool.go:79: [nebula-clients] connection pool is initialized successfully
2021/09/17 10:47:51 [INFO] clientmgr.go:28: Create 15 Nebula Graph clients
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(0) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(1) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(2) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(10) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(3) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(4) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(5) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(8) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] session.go:72: [nebula-clients] Error info: read tcp :34404->:9669: use of closed network connection
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(11) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(6) fails to execute commands (USE aikg_goodstheme;), error: read tcp ->: use of closed network connection
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(9) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(13) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(14) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(12) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:51 [ERROR] clientpool.go:156: Client(7) fails to execute commands (USE aikg_goodstheme;), error: failed to execute: Session has been released
2021/09/17 10:47:52 mkdir /opt/nebula_graph/DD/err/Brand.csv: not a directory
2021/09/17 10:47:53 — END OF NEBULA IMPORTER —

luciaark · 2021 年9 月 17 日 03:07

用最新matser分支编译的程序运行出错，fix的PR提交的是否存在什么问题呢？

luciaark · 2021 年9 月 18 日 07:52

1、文档和实际使用中failDataPath是带文件名的路径
files.failDataPath - 是插入失败的数据文件存放路径，以便后面补写数据。
failDataPath: ./err/studenterr.csv
2、可是你们提交的PR中failDataPath变成了不带文件名的路径，所以导致导入失败
var failedDataPath *string = nil
if f.FailDataPath != nil {
base := filepath.Base(fileNames[i])
tmp := filepath.Join(*f.FailDataPath, base)
failedDataPath = &tmp
logger.Infof(“Failed data path: %v”, *failedDataPath)
}

yee · 2021 年9 月 22 日 03:24

能否把出错的文件的相关 YAML 配置贴一下，同时贴一下通配符适配到的文件名列表。

看给的报错信息，出错的文件路径中含有 ? = 等非法字符。

上面的代码是处理的路径中含有通配符的情况，不含有通配符是在上面的逻辑中已经返回。因为通配符会匹配多个文件，所以如果只指定一个 failedDataPath 会造成不同的文件同时写一个文件，所以就以指定的 failedDataPath 为目录存放所有通配符匹配到的文件的失败数据。

system · 2021 年10 月 22 日 03:24

此话题已在最后回复的 30 天后被自动关闭。不再允许新回复。