查询性能问题,数据量才80多W,查询速度完全没法接受

提问参考模版:

  • nebula 版本:3.2.0
  • 部署方式:单机
  • 安装方式:RPM
  • 是否为线上版本:N
  • 硬件信息
    • 磁盘 40T
    • CPU、内存信息 48核,256G
  • 问题的具体描述
    总数据量:

执行语句:match (v:device)-[c]->(b) where v.device.ipAddress != b.device.ipAddress return v,c,b limit 5

机器是一台非常好物理机,目前就部署了nebula graph相关服务,现在才导入了80W数据,查询要11秒,这查询性能完全没法接受,同样的场景,ES数据有5000W的时候 查询速度都是1秒内,我现在怀疑是我哪里搞错了什么吗,这跟理想的情况差太远了,求大佬们 指点指点

企业版性能会更好

可能是limit没下推

1 个赞

:joy:
没有啥优化建议吗

1 个赞

可以用多步 match,用 with,第一步 limit 一下,第二步再取数据 不行,要用 LOOKUP 才行

抱歉,我试了下发现 像这样 看起来这么写,第二步并不是串行的从上一步的 LIMIT 边 c 作为起点 @Shylock-Hg 咱们有可能增加优化把这个变成 第二个 MATCH 起点从第一个 LIMIT 结果里往下走吗?

MATCH ()-[c:log]->()
WITH c LIMIT 1000
MATCH  (v:device)-[c:log]-(b:device) WHERE v.device.ipAddress != b.device.ipAddress
RETURN v, c, b LIMIT 5

@zou79189747 你可以用 LOOKUP LIMIT去做

LOOKUP on log YIELD id(vertex) AS v | LIMIT 100 | \
  GO FROM $-.v OVER log WHERE $^.device.ipAddress != $$.device.ipAddress YIELD $^ AS v, $$ AS b | LIMIT 5
(root@nebula) [basketballplayer]> explain LOOKUP on player YIELD id(vertex) AS v | LIMIT 100 | GO FROM $-.v OVER follow YIELD $^ AS v, $$ AS b | LIMIT 5
Execution succeeded (time spent 846/58085 us)

Execution Plan (optimize time 328 us)

-----+------------------+--------------+----------------+------------------------------------
| id | name             | dependencies | profiling data | operator info                     |
-----+------------------+--------------+----------------+------------------------------------
| 22 | Project          | 20           |                | outputVar: {                      |
|    |                  |              |                |   "colNames": [                   |
|    |                  |              |                |     "v",                          |
|    |                  |              |                |     "b"                           |
|    |                  |              |                |   ],                              |
|    |                  |              |                |   "type": "DATASET",              |
|    |                  |              |                |   "name": "__DataCollect_14"      |
|    |                  |              |                | }                                 |
|    |                  |              |                | inputVar: __Limit_20              |
|    |                  |              |                | columns: [                        |
|    |                  |              |                |   "$^ AS v",                      |
|    |                  |              |                |   "$$ AS b"                       |
|    |                  |              |                | ]                                 |
-----+------------------+--------------+----------------+------------------------------------
| 20 | Limit            | 11           |                | outputVar: {                      |
|    |                  |              |                |   "colNames": [                   |
|    |                  |              |                |     "$^",                         |
|    |                  |              |                |     "_vid",                       |
|    |                  |              |                |     "JOIN_DST_VID",               |
|    |                  |              |                |     "$$",                         |
|    |                  |              |                |     "DST_VID",                    |
|    |                  |              |                |     "v"                           |
|    |                  |              |                |   ],                              |
|    |                  |              |                |   "type": "DATASET",              |
|    |                  |              |                |   "name": "__Limit_20"            |
|    |                  |              |                | }                                 |
|    |                  |              |                | inputVar: __InnerJoin_11          |
|    |                  |              |                | offset: 0                         |
|    |                  |              |                | count: 5                          |
-----+------------------+--------------+----------------+------------------------------------
| 11 | InnerJoin        | 10           |                | outputVar: {                      |
|    |                  |              |                |   "colNames": [                   |
|    |                  |              |                |     "$^",                         |
|    |                  |              |                |     "_vid",                       |
|    |                  |              |                |     "JOIN_DST_VID",               |
|    |                  |              |                |     "$$",                         |
|    |                  |              |                |     "DST_VID",                    |
|    |                  |              |                |     "v"                           |
|    |                  |              |                |   ],                              |
|    |                  |              |                |   "type": "DATASET",              |
|    |                  |              |                |   "name": "__InnerJoin_11"        |
|    |                  |              |                | }                                 |
|    |                  |              |                | inputVar: {                       |
|    |                  |              |                |   "rightVar": {                   |
|    |                  |              |                |     "__Limit_3": "0"              |
|    |                  |              |                |   },                              |
|    |                  |              |                |   "leftVar": {                    |
|    |                  |              |                |     "__LeftJoin_10": "0"          |
|    |                  |              |                |   }                               |
|    |                  |              |                | }                                 |
|    |                  |              |                | hashKeys: [                       |
|    |                  |              |                |   "$__LeftJoin_10._vid"           |
|    |                  |              |                | ]                                 |
|    |                  |              |                | probeKeys: [                      |
|    |                  |              |                |   "$-.v"                          |
|    |                  |              |                | ]                                 |
|    |                  |              |                | kind: InnerJoin                   |
-----+------------------+--------------+----------------+------------------------------------
| 10 | LeftJoin         | 9            |                | outputVar: {                      |
|    |                  |              |                |   "colNames": [                   |
|    |                  |              |                |     "$^",                         |
|    |                  |              |                |     "_vid",                       |
|    |                  |              |                |     "JOIN_DST_VID",               |
|    |                  |              |                |     "$$",                         |
|    |                  |              |                |     "DST_VID"                     |
|    |                  |              |                |   ],                              |
|    |                  |              |                |   "type": "DATASET",              |
|    |                  |              |                |   "name": "__LeftJoin_10"         |
|    |                  |              |                | }                                 |
|    |                  |              |                | inputVar: {                       |
|    |                  |              |                |   "rightVar": {                   |
|    |                  |              |                |     "__Project_9": "0"            |
|    |                  |              |                |   },                              |
|    |                  |              |                |   "leftVar": {                    |
|    |                  |              |                |     "__Project_7": "0"            |
|    |                  |              |                |   }                               |
|    |                  |              |                | }                                 |
|    |                  |              |                | hashKeys: [                       |
|    |                  |              |                |   "COLUMN[-1]"                    |
|    |                  |              |                | ]                                 |
|    |                  |              |                | probeKeys: [                      |
|    |                  |              |                |   "COLUMN[-1]"                    |
|    |                  |              |                | ]                                 |
|    |                  |              |                | kind: LeftJoin                    |
-----+------------------+--------------+----------------+------------------------------------
|  9 | Project          | 8            |                | outputVar: {                      |
|    |                  |              |                |   "colNames": [                   |
|    |                  |              |                |     "$$",                         |
|    |                  |              |                |     "DST_VID"                     |
|    |                  |              |                |   ],                              |
|    |                  |              |                |   "type": "DATASET",              |
|    |                  |              |                |   "name": "__Project_9"           |
|    |                  |              |                | }                                 |
|    |                  |              |                | inputVar: __GetVertices_8         |
|    |                  |              |                | columns: [                        |
|    |                  |              |                |   "$$ AS $$",                     |
|    |                  |              |                |   "COLUMN[0] AS DST_VID"          |
|    |                  |              |                | ]                                 |
-----+------------------+--------------+----------------+------------------------------------
|  8 | GetVertices      | 7            |                | outputVar: {                      |
|    |                  |              |                |   "colNames": [],                 |
|    |                  |              |                |   "type": "DATASET",              |
|    |                  |              |                |   "name": "__GetVertices_8"       |
|    |                  |              |                | }                                 |
|    |                  |              |                | inputVar: __Project_7             |
|    |                  |              |                | space: 2                          |
|    |                  |              |                | dedup: true                       |
|    |                  |              |                | limit: 9223372036854775807        |
|    |                  |              |                | filter:                           |
|    |                  |              |                | orderBy: []                       |
|    |                  |              |                | src: COLUMN[-1]                   |
|    |                  |              |                | props: [                          |
|    |                  |              |                |   {                               |
|    |                  |              |                |     "props": [                    |
|    |                  |              |                |       "_tag",                     |
|    |                  |              |                |       "name"                      |
|    |                  |              |                |     ],                            |
|    |                  |              |                |     "tagId": 4                    |
|    |                  |              |                |   },                              |
|    |                  |              |                |   {                               |
|    |                  |              |                |     "props": [                    |
|    |                  |              |                |       "_tag",                     |
|    |                  |              |                |       "age",                      |
|    |                  |              |                |       "name"                      |
|    |                  |              |                |     ],                            |
|    |                  |              |                |     "tagId": 3                    |
|    |                  |              |                |   }                               |
|    |                  |              |                | ]                                 |
|    |                  |              |                | exprs:                            |
-----+------------------+--------------+----------------+------------------------------------
|  7 | Project          | 17           |                | outputVar: {                      |
|    |                  |              |                |   "colNames": [                   |
|    |                  |              |                |     "$^",                         |
|    |                  |              |                |     "_vid",                       |
|    |                  |              |                |     "JOIN_DST_VID"                |
|    |                  |              |                |   ],                              |
|    |                  |              |                |   "type": "DATASET",              |
|    |                  |              |                |   "name": "__Project_7"           |
|    |                  |              |                | }                                 |
|    |                  |              |                | inputVar: __GetNeighbors_6        |
|    |                  |              |                | columns: [                        |
|    |                  |              |                |   "$^ AS $^",                     |
|    |                  |              |                |   "COLUMN[0] AS _vid",            |
|    |                  |              |                |   "*._dst AS JOIN_DST_VID"        |
|    |                  |              |                | ]                                 |
-----+------------------+--------------+----------------+------------------------------------
| 17 | GetNeighbors     | 19           |                | outputVar: {                      |
|    |                  |              |                |   "colNames": [],                 |
|    |                  |              |                |   "type": "DATASET",              |
|    |                  |              |                |   "name": "__GetNeighbors_6"      |
|    |                  |              |                | }                                 |
|    |                  |              |                | inputVar: __Limit_3               |
|    |                  |              |                | space: 2                          |
|    |                  |              |                | dedup: true                       |
|    |                  |              |                | limit: -1                         |
|    |                  |              |                | filter:                           |
|    |                  |              |                | orderBy: []                       |
|    |                  |              |                | src: $-.v                         |
|    |                  |              |                | edgeTypes: []                     |
|    |                  |              |                | edgeDirection: OUT_EDGE           |
|    |                  |              |                | vertexProps: [                    |
|    |                  |              |                |   {                               |
|    |                  |              |                |     "props": [                    |
|    |                  |              |                |       "_tag",                     |
|    |                  |              |                |       "name"                      |
|    |                  |              |                |     ],                            |
|    |                  |              |                |     "tagId": 4                    |
|    |                  |              |                |   },                              |
|    |                  |              |                |   {                               |
|    |                  |              |                |     "props": [                    |
|    |                  |              |                |       "_tag",                     |
|    |                  |              |                |       "age",                      |
|    |                  |              |                |       "name"                      |
|    |                  |              |                |     ],                            |
|    |                  |              |                |     "tagId": 3                    |
|    |                  |              |                |   }                               |
|    |                  |              |                | ]                                 |
|    |                  |              |                | edgeProps: [                      |
|    |                  |              |                |   {                               |
|    |                  |              |                |     "props": [                    |
|    |                  |              |                |       "_dst"                      |
|    |                  |              |                |     ],                            |
|    |                  |              |                |     "type": "6"                   |
|    |                  |              |                |   }                               |
|    |                  |              |                | ]                                 |
|    |                  |              |                | statProps:                        |
|    |                  |              |                | exprs:                            |
|    |                  |              |                | random: false                     |
-----+------------------+--------------+----------------+------------------------------------
| 19 | Project          | 23           |                | outputVar: {                      |
|    |                  |              |                |   "colNames": [                   |
|    |                  |              |                |     "v"                           |
|    |                  |              |                |   ],                              |
|    |                  |              |                |   "type": "DATASET",              |
|    |                  |              |                |   "name": "__Limit_3"             |
|    |                  |              |                | }                                 |
|    |                  |              |                | inputVar: __Limit_18              |
|    |                  |              |                | columns: [                        |
|    |                  |              |                |   "id(VERTEX) AS v"               |
|    |                  |              |                | ]                                 |
-----+------------------+--------------+----------------+------------------------------------
| 23 | Limit            | 24           |                | outputVar: {                      |
|    |                  |              |                |   "colNames": [                   |
|    |                  |              |                |     "_vid",                       |
|    |                  |              |                |     "player._tag",                |
|    |                  |              |                |     "player.age",                 |
|    |                  |              |                |     "player.name"                 |
|    |                  |              |                |   ],                              |
|    |                  |              |                |   "type": "DATASET",              |
|    |                  |              |                |   "name": "__Limit_18"            |
|    |                  |              |                | }                                 |
|    |                  |              |                | inputVar: __TagIndexFullScan_24   |
|    |                  |              |                | offset: 0                         |
|    |                  |              |                | count: 100                        |
-----+------------------+--------------+----------------+------------------------------------
| 24 | TagIndexFullScan | 0            |                | outputVar: {                      | # 这里有 limit 下推,不用数据全扫描
|    |                  |              |                |   "colNames": [                   |
|    |                  |              |                |     "_vid",                       |
|    |                  |              |                |     "player._tag",                |
|    |                  |              |                |     "player.age",                 |
|    |                  |              |                |     "player.name"                 |
|    |                  |              |                |   ],                              |
|    |                  |              |                |   "type": "DATASET",              |
|    |                  |              |                |   "name": "__TagIndexFullScan_24" |
|    |                  |              |                | }                                 |
|    |                  |              |                | inputVar:                         |
|    |                  |              |                | space: 2                          |
|    |                  |              |                | dedup: false                      |
|    |                  |              |                | limit: 100                        |
|    |                  |              |                | filter:                           |
|    |                  |              |                | orderBy: []                       |
|    |                  |              |                | schemaId: 3                       |
|    |                  |              |                | isEdge: false                     |
|    |                  |              |                | returnCols: [                     |
|    |                  |              |                |   "_vid",                         |
|    |                  |              |                |   "_tag",                         |
|    |                  |              |                |   "age",                          |
|    |                  |              |                |   "name"                          |
|    |                  |              |                | ]                                 |
|    |                  |              |                | indexCtx: [                       |
|    |                  |              |                |   {                               |
|    |                  |              |                |     "columnHints": [],            |
|    |                  |              |                |     "filter": "",                 |
|    |                  |              |                |     "index_id": 7                 |
|    |                  |              |                |   }                               |
|    |                  |              |                | ]                                 |
-----+------------------+--------------+----------------+------------------------------------
|  0 | Start            |              |                | outputVar: {                      |
|    |                  |              |                |   "colNames": [],                 |
|    |                  |              |                |   "type": "DATASET",              |
|    |                  |              |                |   "name": "__Start_0"             |
|    |                  |              |                | }                                 |
-----+------------------+--------------+----------------+------------------------------------

可以尝试使用 ngql 的subgraph的语法,速度会快

浙ICP备20010487号