nebula-bench import失败

提问参考模版:

  • nebula 版本:v2.0.1

  • 部署方式(分布式 / 单机 / Docker / DBaaS):分布式部署

  • 是否为线上版本:Y

  • 硬件信息

    • 磁盘: SSD
    • CPU、内存信息 8C / 32G 500G SSD
  • 问题的具体描述

执行python3 run.py nebula importer的时候提示response error code: -8, message: Unsupported!
求大佬帮忙

2021/08/17 19:26:17 Client(0) fails to execute commands (CREATE SPACE IF NOT EXISTS stress_test_0817(PARTITION_NUM = 24, REPLICA_FACTOR = 3, vid_type = int64);
USE stress_test_0817;
CREATE TAG IF NOT EXISTS `Place`(`name` string,`url` string,`type` string);
CREATE TAG IF NOT EXISTS `Organisation`(`type` string,`name` string,`url` string);
CREATE TAG IF NOT EXISTS `Tag`(`name` string,`url` string);
CREATE TAG IF NOT EXISTS `Forum`(`title` string,`creationDate` string);
CREATE TAG IF NOT EXISTS `Post`(`imageFile` string,`creationDate` string,`locationIP` string,`browserUsed` string,`language` string,`content` string,`length` int);
CREATE TAG IF NOT EXISTS `Person`(`firstName` string,`lastName` string,`gender` string,`birthday` string,`creationDate` string,`locationIP` string,`browserUsed` string);
CREATE TAG IF NOT EXISTS `Tagclass`(`name` string,`url` string);
CREATE TAG IF NOT EXISTS `Comment`(`creationDate` string,`locationIP` string,`browserUsed` string,`content` string,`length` int);
CREATE EDGE IF NOT EXISTS `HAS_CREATOR`();
CREATE EDGE IF NOT EXISTS `HAS_TAG`();
CREATE EDGE IF NOT EXISTS `IS_SUBCLASS_OF`();
CREATE EDGE IF NOT EXISTS `LIKES`(`creationDate` string);
CREATE EDGE IF NOT EXISTS `IS_PART_OF`();
CREATE EDGE IF NOT EXISTS `HAS_INTEREST`();
CREATE EDGE IF NOT EXISTS `STUDY_AT`(`classYear` int);
CREATE EDGE IF NOT EXISTS `HAS_MODERATOR`();
CREATE EDGE IF NOT EXISTS `HAS_TYPE`();
CREATE EDGE IF NOT EXISTS `IS_LOCATED_IN`();
CREATE EDGE IF NOT EXISTS `KNOWS`(`creationDate` string);
CREATE EDGE IF NOT EXISTS `HAS_MEMBER`(`joinDate` string);
CREATE EDGE IF NOT EXISTS `REPLY_OF`();
CREATE EDGE IF NOT EXISTS `CONTAINER_OF`();
CREATE EDGE IF NOT EXISTS `WORK_AT`(`workFrom` int);
), response error code: -8, message: Unsupported!
2021/08/17 19:26:18 --- END OF NEBULA IMPORTER ---

配置文件如下

version: v2
description: ldbc
removeTempFiles: false
clientSettings:
  retry: 3
  concurrency: 30 # number of graph clients
  channelBufferSize: 128
  space: stress_test_0817
  connection:
    user: root
    password: nebula
    address: 172.18.245.12:9669
  postStart:
    commands: |
      CREATE SPACE IF NOT EXISTS stress_test_0817(PARTITION_NUM = 24, REPLICA_FACTOR = 3, vid_type = int64);
      USE stress_test_0817;
      CREATE TAG IF NOT EXISTS `Organisation`(`type` string,`name` string,`url` string);
      CREATE TAG IF NOT EXISTS `Post`(`imageFile` string,`creationDate` string,`locationIP` string,`browserUsed` string,`language` string,`content` string,`length` int);
      CREATE TAG IF NOT EXISTS `Forum`(`title` string,`creationDate` string);
      CREATE TAG IF NOT EXISTS `Place`(`name` string,`url` string,`type` string);
      CREATE TAG IF NOT EXISTS `Person`(`firstName` string,`lastName` string,`gender` string,`birthday` string,`creationDate` string,`locationIP` string,`browserUsed` string);
      CREATE TAG IF NOT EXISTS `Comment`(`creationDate` string,`locationIP` string,`browserUsed` string,`content` string,`length` int);
      CREATE TAG IF NOT EXISTS `Tagclass`(`name` string,`url` string);
      CREATE TAG IF NOT EXISTS `Tag`(`name` string,`url` string);
      CREATE EDGE IF NOT EXISTS `REPLY_OF`();
      CREATE EDGE IF NOT EXISTS `HAS_CREATOR`();
      CREATE EDGE IF NOT EXISTS `HAS_TAG`();
      CREATE EDGE IF NOT EXISTS `IS_PART_OF`();
      CREATE EDGE IF NOT EXISTS `HAS_TYPE`();
      CREATE EDGE IF NOT EXISTS `IS_LOCATED_IN`();
      CREATE EDGE IF NOT EXISTS `HAS_MODERATOR`();
      CREATE EDGE IF NOT EXISTS `IS_SUBCLASS_OF`();
      CREATE EDGE IF NOT EXISTS `KNOWS`(`creationDate` string);
      CREATE EDGE IF NOT EXISTS `CONTAINER_OF`();
      CREATE EDGE IF NOT EXISTS `HAS_INTEREST`();
      CREATE EDGE IF NOT EXISTS `HAS_MEMBER`(`joinDate` string);
      CREATE EDGE IF NOT EXISTS `LIKES`(`creationDate` string);
      CREATE EDGE IF NOT EXISTS `WORK_AT`(`workFrom` int);
      CREATE EDGE IF NOT EXISTS `STUDY_AT`(`classYear` int);


你执行下 SHOW HOSTS 操作,看下服务起来了没有呢?你的语句提醒是说 Space 创建没成功

部署了几个节点啊,我碰到这个错误通常是space定义的副本数(replica)大于存储的节点数,你检查下。

1 个赞

nebula-bench 生成的 importer 配置文件,默认的副本数是 3,然后你的集群应该只有 1 个实例。

可以增加实例,或者手动改一下 importer_config.yaml,用 importer 导入。

1 个赞

问题修复了,主要是storage的locaip配置成了0.0.0.0,导致虽然我有三个节点,但是meta哪里因为host是一样的,实际上只有一个节点join成功了

现在是一个新问题:
python3 run.py nebula importer -s mytest -a 172.18.244.41:9669

storage的日志显示以下错误
Log file created at: 2021/08/24 10:41:41
Running on machine: prod-nebulua-storage-e003
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0824 10:41:41.636945 41789 RocksEngineConfig.cpp:152] Unsupported compression type: "no
F0824 10:41:41.637230 41789 RocksEngine.cpp:103] Check failed: status.ok()

root@prod-nebulua-storage-e003:/usr/local/nebula/etc# cat nebula-storaged.conf.yingmi | grep compress

–rocksdb_compression=lz4hc

–rocksdb_compression_per_level=“no:no:lz4hc:lz4hc:snappy:snappy:zstd”

这个问题,暂时通过删除rocksdb_compression_per_level配置解决了,但是不知道上面的配置错在哪里?求解

新问题:
执行一下压测命令的时候,没有任何报错,但是结果显示全部都失败了,求解?
python3 run.py stress run -scenario go.Go1Step -a 172.18.245.12:9669

INFO[0062] 2021/08/24 11:27:31 [INFO] begin init the nebula pool
INFO[0062] 2021/08/24 11:27:31 [INFO] connection pool is initialized successfully
INFO[0062] 2021/08/24 11:27:31 [INFO] finish init the pool

 ✗ IsSucceed
  ↳  0% — ✓ 0 / ✗ 1623128

 █ setup

 █ teardown

 checks...............: 0.00%   ✓ 0            ✗ 1623128
 data_received........: 0 B     0 B/s
 data_sent............: 0 B     0 B/s
 iteration_duration...: min=1.25ms avg=3.69ms      med=3ms  max=1.03s p(90)=5.16ms p(95)=8.07ms p(99)=16.41ms
 iterations...........: 1623128 26583.892777/s
 latency..............: min=20     avg=49.800268   med=39   max=15363 p(90)=86     p(95)=109    p(99)=170
 responseTime.........: min=1181   avg=3566.705877 med=2912 max=39309 p(90)=5005   p(95)=7738   p(99)=15876
 vus..................: 100     min=0          max=100
 vus_max..............: 100     min=100        max=100

看一下 output 文件夹里的 csv,最后一列是错误

此问题也解决了,space名称错了

新问题:

如何解读iteration_duration和latency,responseTime的关系

我看了一下k6的说明,没有这几个变量,尤其是后面两个,不知道如何解释

 ✗ IsSucceed
  ↳  99% — ✓ 1365895 / ✗ 27

 █ setup

 █ teardown

 checks...............: 99.99%  ✓ 1365895      ✗ 27
 data_received........: 0 B     0 B/s
 data_sent............: 0 B     0 B/s
 iteration_duration...: min=1.2ms avg=4.38ms     med=3.54ms max=1s    p(90)=6.56ms p(95)=10.01ms p(99)=18.47ms
 iterations...........: 1365922 22384.404314/s
 latency..............: min=303   avg=471.519116 med=454    max=21872 p(90)=563    p(95)=612     p(99)=747
 responseTime.........: min=1348  avg=4265.01119 med=3449   max=45633 p(90)=6366   p(95)=9723    p(99)=18052
 vus..................: 100     min=0          max=100
 vus_max..............: 100     min=100        max=100

iteration_duration 是 k6 每个 vu 跑一次的时间

后面两个是我们自定义的

latency 是服务端返回的耗时
responseTime 是客户端从执行 execute 到收到数据的耗时。

大体上,responseTime = latency + 网路传输的耗时 + 客户端从网络 io 解码的耗时

1 个赞

这个问题,要么新开一个帖子,然后让存储的同学来看一下。

该话题在最后一个回复创建后7天后自动关闭。不再允许新的回复。