nebula graph v2.0 使用nebula-importer 2.0导入LDBC数据集失败

  • nebula 版本:v2.0.0
  • 部署方式(分布式 / 单机 / Docker / DBaaS):单机

问题:生成LDBC数据集,采用了官方给出的data generator(生成参数:branch=stable, version=0.3.3, scale=1),使用nebula importer 2.0部分数据导入失败。

yaml文件的配置我参考了nebula-bench中给出的例子,具体如下:

version: v2
description: ldbc
removeTempFiles: false
clientSettings:
  retry: 3
  concurrency: 1 # number of graph clients
  channelBufferSize: 1
  space: ldbc_snb_sf100_vid_int
  connection:
    user: root
    password: nebula
    address: 127.0.0.1:9669
  postStart:
    commands: |
      CREATE SPACE IF NOT EXISTS ldbc_snb_sf100_vid_int(PARTITION_NUM = 5, REPLICA_FACTOR = 1, vid_type = int64);
      USE ldbc_snb_sf100_vid_int;
      CREATE TAG IF NOT EXISTS person(first_name string, last_name string, gender string, birthday string, ip string, browser string);
      CREATE TAG IF NOT EXISTS place(name string, type string, url string);
      CREATE TAG IF NOT EXISTS organization(name string, type string, url string);
      CREATE TAG IF NOT EXISTS post(`time` string, image string, ip string, browser string, language string, content string, length int);
      CREATE TAG IF NOT EXISTS comment(`time` string, ip string, browser string, content string, length int);
      CREATE TAG IF NOT EXISTS forum(`time` string, title string,type string);
      CREATE TAG IF NOT EXISTS hastag(name string, url string);
      CREATE TAG IF NOT EXISTS tagclass(name string, url string);
      CREATE EDGE IF NOT EXISTS knows(`time` string);
      CREATE EDGE IF NOT EXISTS is_part_of();
      CREATE EDGE IF NOT EXISTS person_is_located_in();
      CREATE EDGE IF NOT EXISTS post_is_located_in();
      CREATE EDGE IF NOT EXISTS comment_is_located_in();
      CREATE EDGE IF NOT EXISTS org_is_located_in();
      CREATE EDGE IF NOT EXISTS forum_has_tag(`time` string);
      CREATE EDGE IF NOT EXISTS comment_has_tag(`time` string);
      CREATE EDGE IF NOT EXISTS post_has_tag(`time` string);
      CREATE EDGE IF NOT EXISTS person_has_tag(`time` string);
      CREATE EDGE IF NOT EXISTS study_at(year string);
      CREATE EDGE IF NOT EXISTS work_at(year string);
      CREATE EDGE IF NOT EXISTS has_post(`time` string);
      CREATE EDGE IF NOT EXISTS has_comment(`time` string);
      CREATE EDGE IF NOT EXISTS likes_post(`time` string);
      CREATE EDGE IF NOT EXISTS likes_comment(`time` string);
      CREATE EDGE IF NOT EXISTS is_reply_of_post(`time` string);
      CREATE EDGE IF NOT EXISTS is_reply_of_comment(`time` string);
      CREATE EDGE IF NOT EXISTS has_member(`time` string,type string);
      CREATE EDGE IF NOT EXISTS contains_post(`time` string);
      CREATE EDGE IF NOT EXISTS has_moderator(`time` string);
      CREATE EDGE IF NOT EXISTS is_tagclass();
      CREATE EDGE IF NOT EXISTS is_sub_of_tagclass();
    afterPeriod: 8s
logPath: ./err/test.log
files:
  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/person.csv
    failDataPath: ./err/data/person.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 1
          type: int
        tags:
          - name: person
            props:
              - name: first_name
                type: string
                index: 2
              - name: last_name
                type: string
                index: 3
              - name: gender
                type: string
                index: 4
              - name: birthday
                type: string
                index: 5
              - name: ip
                type: string
                index: 6
              - name: browser
                type: string
                index: 7

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/static/place.csv
    failDataPath: ./err/data/place.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 0
          type: int
        tags:
          - name: place
            props:
              - name: name
                type: string
                index: 1
              - name: type
                type: string
                index: 3
              - name: url
                type: string
                index: 2

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/static/organisation.csv
    failDataPath: ./err/data/organisation.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 0
          type: int
        tags:
          - name: organization
            props:
              - name: name
                type: string
                index: 2
              - name: type
                type: string
                index: 1
              - name: url
                type: string
                index: 3

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/post.csv
    failDataPath: ./err/data/post.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 1
          type: int
        tags:
          - name: post
            props:
              - name: time
                type: string
                index: 0
              - name: image
                type: string
                index: 2
              - name: ip
                type: string
                index: 3
              - name: browser
                type: string
                index: 4
              - name: language
                type: string
                index: 5
              - name: content
                type: string
                index: 6
              - name: length
                type: int
                index: 7

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/comment.csv
    failDataPath: ./err/data/comment.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 1
          type: int
        tags:
          - name: comment
            props:
              - name: time
                type: string
                index: 0
              - name: ip
                type: string
                index: 2
              - name: browser
                type: string
                index: 3
              - name: content
                type: string
                index: 4
              - name: length
                type: int
                index: 5

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/forum.csv
    failDataPath: ./err/data/forum.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 1
          type: int
        tags:
          - name: forum
            props:
              - name: time
                type: string
                index: 0
              - name: title
                type: string
                index: 2
              - name: type
                type: string
                index: 3

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/static/tag.csv
    failDataPath: ./err/data/tag.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 0
          type: int
        tags:
          - name: hastag
            props:
              - name: name
                type: string
                index: 1
              - name: url
                type: string
                index: 2

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/static/tagclass.csv
    failDataPath: ./err/data/tagclass.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          type: int
          index: 0
        tags:
          - name: tagclass
            props:
              - name: name
                type: string
                index: 1
              - name: url
                type: string
                index: 2

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/person_knows_person.csv
    failDataPath: ./err/data/person_knows_person.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: knows
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: time
            type: string
            index: 0

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/static/place_isPartOf_place.csv
    failDataPath: ./err/data/place_isPartOf_place.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: is_part_of
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/person_isLocatedIn_place.csv
    failDataPath: ./err/data/person_isLocatedIn_place.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: person_is_located_in
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/comment_isLocatedIn_place.csv
    failDataPath: ./err/data/comment_isLocatedIn_place.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: comment_is_located_in
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/static/organisation_isLocatedIn_place.csv
    failDataPath: ./err/data/organisation_isLocatedIn_place.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: org_is_located_in
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int


  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/forum_hasTag_tag.csv
    failDataPath: ./err/data/forum_hasTag_tag.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: forum_has_tag
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: time
            type: string
            index: 0
  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/comment_hasTag_tag.csv
    failDataPath: ./err/data/comment_hasTag_tag.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: comment_has_tag
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: time
            type: string
            index: 0


  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/post_hasTag_tag.csv
    failDataPath: ./err/data/post_hasTag_tag.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: post_has_tag
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: time
            type: string
            index: 0

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/person_hasInterest_tag.csv
    failDataPath: ./err/data/person_hasInterest_tag.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: person_has_tag
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: time
            type: string
            index: 0


  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/person_studyAt_organisation.csv
    failDataPath: ./err/data/person_studyAt_organisation.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: study_at
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: year
            type: string
            index: 3

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/person_workAt_organisation.csv
    failDataPath: ./err/data/person_workAt_organisation.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: work_at
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: year
            type: string
            index: 3


  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/post_hasCreator_person.csv
    failDataPath: ./err/data/post_hasCreator_person.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: has_post
        withRanking: false
        srcVID:
          index: 2
          type: int
        dstVID:
          index: 1
          type: int
        props:
          - name: time
            type: string
            index: 0

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/comment_hasCreator_person.csv
    failDataPath: ./err/data/comment_hasCreator_person.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: has_comment
        withRanking: false
        srcVID:
          index: 2
          type: int
        dstVID:
          index: 1
          type: int
        props:
          - name: time
            type: string
            index: 0

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/person_likes_post.csv
    failDataPath: ./err/data/person_likes_post.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: likes_post
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: time
            type: string
            index: 0

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/person_likes_comment.csv
    failDataPath: ./err/data/person_likes_comment.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: likes_comment
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: time
            type: string
            index: 0


  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/comment_replyOf_post.csv
    failDataPath: ./err/data/comment_replyOf_post.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: is_reply_of_post
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: time
            type: string
            index: 0


  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/comment_replyOf_comment.csv
    failDataPath: ./err/data/comment_replyOf_comment.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: is_reply_of_comment
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: time
            type: string
            index: 0


  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/forum_hasMember_person.csv
    failDataPath: ./err/data/forum_hasMember_person.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: has_member
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: time
            type: string
            index: 0
          - name: type
            type: string
            index: 3

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/forum_containerOf_post.csv
    failDataPath: ./err/data/forum_containerOf_post.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: contains_post
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: time
            type: string
            index: 0


  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/forum_hasModerator_person.csv
    failDataPath: ./err/data/forum_hasModerator_person.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: has_moderator
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int
        props:
          - name: time
            type: string
            index: 0


  - path: /home/skye/ldbc_snb_datagen-stable/social_network/static/tag_hasType_tagclass.csv
    failDataPath: ./err/data/tag_hasType_tagclass.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: is_tagclass
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/static/tagclass_isSubclassOf_tagclass.csv
    failDataPath: ./err/data/tagclass_isSubclassOf_tagclass.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: is_sub_of_tagclass
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int

  - path: /home/skye/ldbc_snb_datagen-stable/social_network/dynamic/post_isLocatedIn_place.csv
    failDataPath: ./err/data/post_isLocatedIn_place.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: post_is_located_in
        withRanking: false
        srcVID:
          index: 1
          type: int
        dstVID:
          index: 2
          type: int

以下是报错截图:

这种报错基本上是数据列和配置文件的格式没对齐,你可以看看 ./err/test.log 日志文件中的 INSERT 语句,就知道为什么报错语法错了

1 个赞

forum.csv 的配置错了,给一个 data generator v0.3.3 的配置文件做参考。

version: v2
description: ldbc
removeTempFiles: false
clientSettings:
  retry: 3
  concurrency: 30 # number of graph clients
  channelBufferSize: 128
  space: sf30
  connection:
    user: root
    password: nebula
    address: 192.168.8.145:9669
  postStart:
    commands: |
      CREATE SPACE IF NOT EXISTS sf30(PARTITION_NUM = 24, REPLICA_FACTOR = 1, vid_type = int64);
      USE sf30;
      CREATE TAG IF NOT EXISTS `Comment`(`creationDate` string,`locationIP` string,`browserUsed` string,`content` string,`length` int);
      CREATE TAG IF NOT EXISTS `Place`(`name` string,`url` string,`type` string);
      CREATE TAG IF NOT EXISTS `Post`(`imageFile` string,`creationDate` string,`locationIP` string,`browserUsed` string,`language` string,`content` string,`length` int);
      CREATE TAG IF NOT EXISTS `Tagclass`(`name` string,`url` string);
      CREATE TAG IF NOT EXISTS `Person`(`firstName` string,`lastName` string,`gender` string,`birthday` string,`creationDate` string,`locationIP` string,`browserUsed` string);
      CREATE TAG IF NOT EXISTS `Forum`(`title` string,`creationDate` string);
      CREATE TAG IF NOT EXISTS `Tag`(`name` string,`url` string);
      CREATE TAG IF NOT EXISTS `Organisation`(`type` string,`name` string,`url` string);
      CREATE EDGE IF NOT EXISTS `HAS_INTEREST`();
      CREATE EDGE IF NOT EXISTS `REPLY_OF`();
      CREATE EDGE IF NOT EXISTS `HAS_MODERATOR`();
      CREATE EDGE IF NOT EXISTS `LIKES`(`creationDate` string);
      CREATE EDGE IF NOT EXISTS `STUDY_AT`(`classYear` int);
      CREATE EDGE IF NOT EXISTS `WORK_AT`(`workFrom` int);
      CREATE EDGE IF NOT EXISTS `IS_PART_OF`();
      CREATE EDGE IF NOT EXISTS `CONTAINER_OF`();
      CREATE EDGE IF NOT EXISTS `HAS_MEMBER`(`joinDate` string);
      CREATE EDGE IF NOT EXISTS `KNOWS`(`creationDate` string);
      CREATE EDGE IF NOT EXISTS `IS_LOCATED_IN`();
      CREATE EDGE IF NOT EXISTS `IS_SUBCLASS_OF`();
      CREATE EDGE IF NOT EXISTS `HAS_TAG`();
      CREATE EDGE IF NOT EXISTS `HAS_CREATOR`();
      CREATE EDGE IF NOT EXISTS `HAS_TYPE`();
      

    afterPeriod: 8s
logPath: ./err/test.log
files:
  - path: /home/vesoft/sf30/social_network/dynamic/post.csv
    failDataPath: ./err/data/Post.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 0
          type: int
        tags:
          - name: Post
            props:
              - name: imageFile
                type: string
                index: 1
              - name: creationDate
                type: string
                index: 2
              - name: locationIP
                type: string
                index: 3
              - name: browserUsed
                type: string
                index: 4
              - name: language
                type: string
                index: 5
              - name: content
                type: string
                index: 6
              - name: length
                type: int
                index: 7

  - path: /home/vesoft/sf30/social_network/dynamic/comment.csv
    failDataPath: ./err/data/Comment.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 0
          type: int
        tags:
          - name: Comment
            props:
              - name: creationDate
                type: string
                index: 1
              - name: locationIP
                type: string
                index: 2
              - name: browserUsed
                type: string
                index: 3
              - name: content
                type: string
                index: 4
              - name: length
                type: int
                index: 5

  - path: /home/vesoft/sf30/social_network/dynamic/forum.csv
    failDataPath: ./err/data/Forum.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 0
          type: int
        tags:
          - name: Forum
            props:
              - name: title
                type: string
                index: 1
              - name: creationDate
                type: string
                index: 2

  - path: /home/vesoft/sf30/social_network/dynamic/person.csv
    failDataPath: ./err/data/Person.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 0
          type: int
        tags:
          - name: Person
            props:
              - name: firstName
                type: string
                index: 1
              - name: lastName
                type: string
                index: 2
              - name: gender
                type: string
                index: 3
              - name: birthday
                type: string
                index: 4
              - name: creationDate
                type: string
                index: 5
              - name: locationIP
                type: string
                index: 6
              - name: browserUsed
                type: string
                index: 7

  - path: /home/vesoft/sf30/social_network/static/organisation.csv
    failDataPath: ./err/data/Organisation.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 0
          type: int
        tags:
          - name: Organisation
            props:
              - name: type
                type: string
                index: 1
              - name: name
                type: string
                index: 2
              - name: url
                type: string
                index: 3

  - path: /home/vesoft/sf30/social_network/static/place.csv
    failDataPath: ./err/data/Place.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 0
          type: int
        tags:
          - name: Place
            props:
              - name: name
                type: string
                index: 1
              - name: url
                type: string
                index: 2
              - name: type
                type: string
                index: 3

  - path: /home/vesoft/sf30/social_network/static/tag.csv
    failDataPath: ./err/data/Tag.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 0
          type: int
        tags:
          - name: Tag
            props:
              - name: name
                type: string
                index: 1
              - name: url
                type: string
                index: 2

  - path: /home/vesoft/sf30/social_network/static/tagclass.csv
    failDataPath: ./err/data/Tagclass.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: vertex
      vertex:
        vid:
          index: 0
          type: int
        tags:
          - name: Tagclass
            props:
              - name: name
                type: string
                index: 1
              - name: url
                type: string
                index: 2

  - path: /home/vesoft/sf30/social_network/dynamic/person_isLocatedIn_place.csv
    failDataPath: ./err/data/IS_LOCATED_IN.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: IS_LOCATED_IN
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/post_hasCreator_person.csv
    failDataPath: ./err/data/HAS_CREATOR.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: HAS_CREATOR
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/post_hasTag_tag.csv
    failDataPath: ./err/data/HAS_TAG.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: HAS_TAG
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/post_isLocatedIn_place.csv
    failDataPath: ./err/data/IS_LOCATED_IN.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: IS_LOCATED_IN
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/comment_hasCreator_person.csv
    failDataPath: ./err/data/HAS_CREATOR.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: HAS_CREATOR
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/comment_hasTag_tag.csv
    failDataPath: ./err/data/HAS_TAG.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: HAS_TAG
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/comment_isLocatedIn_place.csv
    failDataPath: ./err/data/IS_LOCATED_IN.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: IS_LOCATED_IN
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/comment_replyOf_comment.csv
    failDataPath: ./err/data/REPLY_OF.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: REPLY_OF
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/comment_replyOf_post.csv
    failDataPath: ./err/data/REPLY_OF.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: REPLY_OF
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/forum_containerOf_post.csv
    failDataPath: ./err/data/CONTAINER_OF.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: CONTAINER_OF
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/forum_hasMember_person.csv
    failDataPath: ./err/data/HAS_MEMBER.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: HAS_MEMBER
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:
          - name: joinDate
            type: string
            index: 2

  - path: /home/vesoft/sf30/social_network/dynamic/forum_hasModerator_person.csv
    failDataPath: ./err/data/HAS_MODERATOR.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: HAS_MODERATOR
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/forum_hasTag_tag.csv
    failDataPath: ./err/data/HAS_TAG.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: HAS_TAG
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/person_hasInterest_tag.csv
    failDataPath: ./err/data/HAS_INTEREST.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: HAS_INTEREST
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/dynamic/person_knows_person.csv
    failDataPath: ./err/data/KNOWS.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: KNOWS
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:
          - name: creationDate
            type: string
            index: 2

  - path: /home/vesoft/sf30/social_network/dynamic/person_likes_comment.csv
    failDataPath: ./err/data/LIKES.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: LIKES
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:
          - name: creationDate
            type: string
            index: 2

  - path: /home/vesoft/sf30/social_network/dynamic/person_likes_post.csv
    failDataPath: ./err/data/LIKES.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: LIKES
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:
          - name: creationDate
            type: string
            index: 2

  - path: /home/vesoft/sf30/social_network/dynamic/person_studyAt_organisation.csv
    failDataPath: ./err/data/STUDY_AT.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: STUDY_AT
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:
          - name: classYear
            type: int
            index: 2

  - path: /home/vesoft/sf30/social_network/dynamic/person_workAt_organisation.csv
    failDataPath: ./err/data/WORK_AT.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: WORK_AT
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:
          - name: workFrom
            type: int
            index: 2

  - path: /home/vesoft/sf30/social_network/static/organisation_isLocatedIn_place.csv
    failDataPath: ./err/data/IS_LOCATED_IN.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: IS_LOCATED_IN
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/static/place_isPartOf_place.csv
    failDataPath: ./err/data/IS_PART_OF.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: IS_PART_OF
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/static/tagclass_isSubclassOf_tagclass.csv
    failDataPath: ./err/data/IS_SUBCLASS_OF.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: IS_SUBCLASS_OF
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

  - path: /home/vesoft/sf30/social_network/static/tag_hasType_tagclass.csv
    failDataPath: ./err/data/HAS_TYPE.csv
    batchSize: 100
    type: csv
    csv:
      withHeader: false
      withLabel: false
      delimiter: "|"
    schema:
      type: edge
      edge:
        name: HAS_TYPE
        withRanking: false
        srcVID:
          index: 0
          type: int
        dstVID:
          index: 1
          type: int
        props:

3 个赞

您好,感谢您的回复,我又检查了一遍自己yaml文件的配置,的确发现了其与生成的LDBC数据集不匹配的地方,现在已经按照您给的示例逐一修改,但是再次导入数据一开始的时候遇到了下面的错误

即importer不能执行(CREATE SPACE IF NOT EXISTS ldbc(PARTITION_NUM = 5, REPLICA_FACTOR = 1, vid_type = int64);语句,我尝试直接在console中执行这条语句是没有问题的,不知道是哪里出现了问题。

你看下最后报错, 'M = '语法错误,检查下是不是有非法字符在

应该不是非法字符的问题,我直接复制这条语句到console是可以执行的

贴上我的部分yaml文件信息

version: v2
description: ldbc
removeTempFiles: false
clientSettings:
  retry: 3
  concurrency: 12 # number of graph clients
  channelBufferSize: 1
  space: ldbc
  connection:
    user: root
    password: nebula
    address: 127.0.0.1:9669
  postStart:
    commands: |
      CREATE SPACE IF NOT EXISTS ldbc(PARTITION_NUM = 5, REPLICA_FACTOR = 1, vid_type = int64);
      USE ldbc;
      CREATE TAG IF NOT EXISTS person(first_name string, last_name string, gender string, birthday string, creationDate string, ip string, browser string);
      CREATE TAG IF NOT EXISTS place(name string, url string, type string);
      CREATE TAG IF NOT EXISTS organization(type string, name string, url string);
      CREATE TAG IF NOT EXISTS post(image string, time string, ip string, browser string, language string, content string, length int);
      CREATE TAG IF NOT EXISTS comment(time string, ip string, browser string, content string, length int);
      CREATE TAG IF NOT EXISTS forum(title string, time string);
      CREATE TAG IF NOT EXISTS hastag(name string, url string);
      CREATE TAG IF NOT EXISTS tagclass(name string, url string);
CREATE TAG IF NOT EXISTS post(image string, time string, ip string, browser string, language string, content string, length int);

time 是关键字,需要转义,用 `time`

importer 是把 postStart 作为一个批,一起发到 graph,所以错误会有一点难分辨。遇到这种情况,可以一条一条在 console 里执行来排查。

3 个赞

好的 数据已经可以成功导入了 感谢您一直的支持!
另外,想再询问一下,一般来说下面的concurrency取什么值比较合理呢

version: v2
description: ldbc
removeTempFiles: false
clientSettings:
  retry: 3
  concurrency: 12 # number of graph clients

concurrency 是连接 nebula-graph 的 client 数量,channelBufferSize 是每个 client 中 channel 的 buffer 大小。
如果你服务端性能比较好,磁盘读写快的话,可以适当调大,提高导入速度。
因为是和机器环境有关,不确定你那边的推荐值,默认的话,concurrency 是 10,channelBufferSize 是 128

2 个赞

好的 了解了 感谢:heart: