limit作用在match语句上对于性能没有影响

提问参考模版:

  • nebula 版本:v3.4.0
  • 部署方式:单机
  • 安装方式:Docker
  • 是否上生产环境:N

对于match语句,有没有limit并没有性能上的提升,两者的执行效率几乎相同。例如

match (v1)-[e:attack]-(v2) where id(v1)=="172.16.210.1" return e limit 10

Got 10 rows (time spent 14.724429s/14.732450258s)

profile如下:

-----+----------------+--------------+---------------------------------------------+---------------------------------
| id | name           | dependencies | profiling data                              | operator info                  |
-----+----------------+--------------+---------------------------------------------+---------------------------------
| 11 | Project        | 10           | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "36(us)",                     |   "colNames": [                |
|    |                |              |   "rows": 10,                               |     "e"                        |
|    |                |              |   "totalTime": "45(us)",                    |   ],                           |
|    |                |              |   "version": 0                              |   "type": "DATASET",           |
|    |                |              | }                                           |   "name": "__Limit_8"          |
|    |                |              |                                             | }                              |
|    |                |              |                                             | inputVar: __Limit_10           |
|    |                |              |                                             | columns: [                     |
|    |                |              |                                             |   "$-.e[0]"                    |
|    |                |              |                                             | ]                              |
-----+----------------+--------------+---------------------------------------------+---------------------------------
| 10 | Limit          | 5            | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "27710(us)",                  |   "colNames": [                |
|    |                |              |   "rows": 10,                               |     "v1",                      |
|    |                |              |   "totalTime": "27720(us)",                 |     "e",                       |
|    |                |              |   "version": 0                              |     "v2"                       |
|    |                |              | }                                           |   ],                           |
|    |                |              |                                             |   "type": "DATASET",           |
|    |                |              |                                             |   "name": "__Limit_10"         |
|    |                |              |                                             | }                              |
|    |                |              |                                             | inputVar: __AppendVertices_5   |
|    |                |              |                                             | offset: 0                      |
|    |                |              |                                             | count: 10                      |
-----+----------------+--------------+---------------------------------------------+---------------------------------
|  5 | AppendVertices | 4            | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "128950(us)",                 |   "colNames": [                |
|    |                |              |   "resp[0]": {                              |     "v1",                      |
|    |                |              |     "exec": "3807(us)",                     |     "e",                       |
|    |                |              |     "host": "storaged:9779",                |     "v2"                       |
|    |                |              |     "total": "4466(us)"                     |   ],                           |
|    |                |              |   },                                        |   "type": "DATASET",           |
|    |                |              |   "rows": 123077,                           |   "name": "__AppendVertices_5" |
|    |                |              |   "totalTime": "133382(us)",                | }                              |
|    |                |              |   "total_rpc": "4531(us)",                  | inputVar: __Traverse_4         |
|    |                |              |   "version": 0                              | space: 89                      |
|    |                |              | }                                           | dedup: true                    |
|    |                |              |                                             | limit: -1                      |
|    |                |              |                                             | filter:                        |
|    |                |              |                                             | orderBy: []                    |
|    |                |              |                                             | src: none_direct_dst($-.e)     |
|    |                |              |                                             | props: [                       |
|    |                |              |                                             |   {                            |
|    |                |              |                                             |     "props": [                 |
|    |                |              |                                             |       "_tag"                   |
|    |                |              |                                             |     ],                         |
|    |                |              |                                             |     "tagId": 90                |
|    |                |              |                                             |   }                            |
|    |                |              |                                             | ]                              |
|    |                |              |                                             | exprs:                         |
|    |                |              |                                             | vertex_filter:                 |
|    |                |              |                                             | if_track_previous_path: true   |
-----+----------------+--------------+---------------------------------------------+---------------------------------
|  4 | Traverse       | 2            | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "785491(us)",                 |   "colNames": [                |
|    |                |              |   "rows": 123077,                           |     "v1",                      |
|    |                |              |   "step[1]": [                              |     "e"                        |
|    |                |              |     {                                       |   ],                           |
|    |                |              |       "exec": "13325848(us)",               |   "type": "DATASET",           |
|    |                |              |       "host": "storaged:9779",              |   "name": "__Traverse_4"       |
|    |                |              |       "storage_detail": {                   | }                              |
|    |                |              |         "GetNeighborsNode": "13323867(us)", | inputVar: __Dedup_2            |
|    |                |              |         "HashJoinNode": "219(us)",          | space: 89                      |
|    |                |              |         "RelNode": "13323868(us)",          | dedup: true                    |
|    |                |              |         "SingleEdgeNode": "215(us)"         | limit: -1                      |
|    |                |              |       },                                    | filter:                        |
|    |                |              |       "total": "13772842(us)",              | orderBy: []                    |
|    |                |              |       "total_rpc_time": "13772942(us)",     | src: $-._vid                   |
|    |                |              |       "vertices": 1                         | edgeTypes: []                  |
|    |                |              |     }                                       | edgeDirection: BOTH            |
|    |                |              |   ],                                        | vertexProps:                   |
|    |                |              |   "totalTime": "14558480(us)",              | edgeProps: [                   |
|    |                |              |   "version": 0                              |   {                            |
|    |                |              | }                                           |     "props": [                 |
|    |                |              |                                             |       "_src",                  |
|    |                |              |                                             |       "_type",                 |
|    |                |              |                                             |       "_rank",                 |
|    |                |              |                                             |       "_dst",                  |
|    |                |              |                                             |       "name",                  |
|    |                |              |                                             |       "proto",                 |
|    |                |              |                                             |       "app_proto",             |
|    |                |              |                                             |       "tool",                  |
|    |                |              |                                             |       "attacker_port",         |
|    |                |              |                                             |       "victim_port",           |
|    |                |              |                                             |       "phase",                 |
|    |                |              |                                             |       "result",                |
|    |                |              |                                             |       "tag",                   |
|    |                |              |                                             |       "severity",              |
|    |                |              |                                             |       "event_time",            |
|    |                |              |                                             |       "flow_id"                |
|    |                |              |                                             |     ],                         |
|    |                |              |                                             |     "type": -91                |
|    |                |              |                                             |   },                           |
|    |                |              |                                             |   {                            |
|    |                |              |                                             |     "props": [                 |
|    |                |              |                                             |       "_src",                  |
|    |                |              |                                             |       "_type",                 |
|    |                |              |                                             |       "_rank",                 |
|    |                |              |                                             |       "_dst",                  |
|    |                |              |                                             |       "name",                  |
|    |                |              |                                             |       "proto",                 |
|    |                |              |                                             |       "app_proto",             |
|    |                |              |                                             |       "tool",                  |
|    |                |              |                                             |       "attacker_port",         |
|    |                |              |                                             |       "victim_port",           |
|    |                |              |                                             |       "phase",                 |
|    |                |              |                                             |       "result",                |
|    |                |              |                                             |       "tag",                   |
|    |                |              |                                             |       "severity",              |
|    |                |              |                                             |       "event_time",            |
|    |                |              |                                             |       "flow_id"                |
|    |                |              |                                             |     ],                         |
|    |                |              |                                             |     "type": 91                 |
|    |                |              |                                             |   }                            |
|    |                |              |                                             | ]                              |
|    |                |              |                                             | statProps:                     |
|    |                |              |                                             | exprs:                         |
|    |                |              |                                             | random: false                  |
|    |                |              |                                             | steps: 1..1                    |
|    |                |              |                                             | vertex filter:                 |
|    |                |              |                                             | edge filter:                   |
|    |                |              |                                             | if_track_previous_path: false  |
|    |                |              |                                             | first step filter:             |
|    |                |              |                                             | tag filter:                    |
-----+----------------+--------------+---------------------------------------------+---------------------------------
|  2 | Dedup          | 1            | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "8(us)",                      |   "colNames": [                |
|    |                |              |   "rows": 1,                                |     "_vid"                     |
|    |                |              |   "totalTime": "10(us)",                    |   ],                           |
|    |                |              |   "version": 0                              |   "type": "DATASET",           |
|    |                |              | }                                           |   "name": "__Dedup_2"          |
|    |                |              |                                             | }                              |
|    |                |              |                                             | inputVar: __VAR_0              |
-----+----------------+--------------+---------------------------------------------+---------------------------------
|  1 | PassThrough    | 3            | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "6(us)",                      |   "colNames": [                |
|    |                |              |   "rows": 0,                                |     "_vid"                     |
|    |                |              |   "totalTime": "10(us)",                    |   ],                           |
|    |                |              |   "version": 0                              |   "type": "DATASET",           |
|    |                |              | }                                           |   "name": "__VAR_0"            |
|    |                |              |                                             | }                              |
|    |                |              |                                             | inputVar:                      |
-----+----------------+--------------+---------------------------------------------+---------------------------------
|  3 | Start          |              | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "0(us)",                      |   "colNames": [],              |
|    |                |              |   "rows": 0,                                |   "type": "DATASET",           |
|    |                |              |   "totalTime": "34(us)",                    |   "name": "__Start_3"          |
|    |                |              |   "version": 0                              | }                              |
|    |                |              | }                                           |                                |
-----+----------------+--------------+---------------------------------------------+---------------------------------

match (v1)-[e:attack]-(v2) where id(v1)=="172.16.210.1" return e

Got 123077 rows (time spent 15.296167s/33.908918198s)

profile如下:

-----+----------------+--------------+---------------------------------------------+---------------------------------
| id | name           | dependencies | profiling data                              | operator info                  |
-----+----------------+--------------+---------------------------------------------+---------------------------------
|  8 | Project        | 5            | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "52809(us)",                  |   "colNames": [                |
|    |                |              |   "rows": 123077,                           |     "e"                        |
|    |                |              |   "totalTime": "52822(us)",                 |   ],                           |
|    |                |              |   "version": 0                              |   "type": "DATASET",           |
|    |                |              | }                                           |   "name": "__Project_7"        |
|    |                |              |                                             | }                              |
|    |                |              |                                             | inputVar: __AppendVertices_5   |
|    |                |              |                                             | columns: [                     |
|    |                |              |                                             |   "$-.e[0]"                    |
|    |                |              |                                             | ]                              |
-----+----------------+--------------+---------------------------------------------+---------------------------------
|  5 | AppendVertices | 4            | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "176464(us)",                 |   "colNames": [                |
|    |                |              |   "resp[0]": {                              |     "v1",                      |
|    |                |              |     "exec": "554(us)",                      |     "e",                       |
|    |                |              |     "host": "storaged:9779",                |     "v2"                       |
|    |                |              |     "total": "1083(us)"                     |   ],                           |
|    |                |              |   },                                        |   "type": "DATASET",           |
|    |                |              |   "rows": 123077,                           |   "name": "__AppendVertices_5" |
|    |                |              |   "totalTime": "177537(us)",                | }                              |
|    |                |              |   "total_rpc": "1154(us)",                  | inputVar: __Traverse_4         |
|    |                |              |   "version": 0                              | space: 89                      |
|    |                |              | }                                           | dedup: true                    |
|    |                |              |                                             | limit: -1                      |
|    |                |              |                                             | filter:                        |
|    |                |              |                                             | orderBy: []                    |
|    |                |              |                                             | src: none_direct_dst($-.e)     |
|    |                |              |                                             | props: [                       |
|    |                |              |                                             |   {                            |
|    |                |              |                                             |     "props": [                 |
|    |                |              |                                             |       "_tag"                   |
|    |                |              |                                             |     ],                         |
|    |                |              |                                             |     "tagId": 90                |
|    |                |              |                                             |   }                            |
|    |                |              |                                             | ]                              |
|    |                |              |                                             | exprs:                         |
|    |                |              |                                             | vertex_filter:                 |
|    |                |              |                                             | if_track_previous_path: true   |
-----+----------------+--------------+---------------------------------------------+---------------------------------
|  4 | Traverse       | 2            | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "791901(us)",                 |   "colNames": [                |
|    |                |              |   "rows": 123077,                           |     "v1",                      |
|    |                |              |   "step[1]": [                              |     "e"                        |
|    |                |              |     {                                       |   ],                           |
|    |                |              |       "exec": "13895094(us)",               |   "type": "DATASET",           |
|    |                |              |       "host": "storaged:9779",              |   "name": "__Traverse_4"       |
|    |                |              |       "storage_detail": {                   | }                              |
|    |                |              |         "GetNeighborsNode": "13894766(us)", | inputVar: __Dedup_2            |
|    |                |              |         "HashJoinNode": "230(us)",          | space: 89                      |
|    |                |              |         "RelNode": "13894767(us)",          | dedup: true                    |
|    |                |              |         "SingleEdgeNode": "227(us)"         | limit: -1                      |
|    |                |              |       },                                    | filter:                        |
|    |                |              |       "total": "14272535(us)",              | orderBy: []                    |
|    |                |              |       "total_rpc_time": "14272646(us)",     | src: $-._vid                   |
|    |                |              |       "vertices": 1                         | edgeTypes: []                  |
|    |                |              |     }                                       | edgeDirection: BOTH            |
|    |                |              |   ],                                        | vertexProps:                   |
|    |                |              |   "totalTime": "15064605(us)",              | edgeProps: [                   |
|    |                |              |   "version": 0                              |   {                            |
|    |                |              | }                                           |     "props": [                 |
|    |                |              |                                             |       "_src",                  |
|    |                |              |                                             |       "_type",                 |
|    |                |              |                                             |       "_rank",                 |
|    |                |              |                                             |       "_dst",                  |
|    |                |              |                                             |       "name",                  |
|    |                |              |                                             |       "proto",                 |
|    |                |              |                                             |       "app_proto",             |
|    |                |              |                                             |       "tool",                  |
|    |                |              |                                             |       "attacker_port",         |
|    |                |              |                                             |       "victim_port",           |
|    |                |              |                                             |       "phase",                 |
|    |                |              |                                             |       "result",                |
|    |                |              |                                             |       "tag",                   |
|    |                |              |                                             |       "severity",              |
|    |                |              |                                             |       "event_time",            |
|    |                |              |                                             |       "flow_id"                |
|    |                |              |                                             |     ],                         |
|    |                |              |                                             |     "type": -91                |
|    |                |              |                                             |   },                           |
|    |                |              |                                             |   {                            |
|    |                |              |                                             |     "props": [                 |
|    |                |              |                                             |       "_src",                  |
|    |                |              |                                             |       "_type",                 |
|    |                |              |                                             |       "_rank",                 |
|    |                |              |                                             |       "_dst",                  |
|    |                |              |                                             |       "name",                  |
|    |                |              |                                             |       "proto",                 |
|    |                |              |                                             |       "app_proto",             |
|    |                |              |                                             |       "tool",                  |
|    |                |              |                                             |       "attacker_port",         |
|    |                |              |                                             |       "victim_port",           |
|    |                |              |                                             |       "phase",                 |
|    |                |              |                                             |       "result",                |
|    |                |              |                                             |       "tag",                   |
|    |                |              |                                             |       "severity",              |
|    |                |              |                                             |       "event_time",            |
|    |                |              |                                             |       "flow_id"                |
|    |                |              |                                             |     ],                         |
|    |                |              |                                             |     "type": 91                 |
|    |                |              |                                             |   }                            |
|    |                |              |                                             | ]                              |
|    |                |              |                                             | statProps:                     |
|    |                |              |                                             | exprs:                         |
|    |                |              |                                             | random: false                  |
|    |                |              |                                             | steps: 1..1                    |
|    |                |              |                                             | vertex filter:                 |
|    |                |              |                                             | edge filter:                   |
|    |                |              |                                             | if_track_previous_path: false  |
|    |                |              |                                             | first step filter:             |
|    |                |              |                                             | tag filter:                    |
-----+----------------+--------------+---------------------------------------------+---------------------------------
|  2 | Dedup          | 1            | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "8(us)",                      |   "colNames": [                |
|    |                |              |   "rows": 1,                                |     "_vid"                     |
|    |                |              |   "totalTime": "11(us)",                    |   ],                           |
|    |                |              |   "version": 0                              |   "type": "DATASET",           |
|    |                |              | }                                           |   "name": "__Dedup_2"          |
|    |                |              |                                             | }                              |
|    |                |              |                                             | inputVar: __VAR_0              |
-----+----------------+--------------+---------------------------------------------+---------------------------------
|  1 | PassThrough    | 3            | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "9(us)",                      |   "colNames": [                |
|    |                |              |   "rows": 0,                                |     "_vid"                     |
|    |                |              |   "totalTime": "14(us)",                    |   ],                           |
|    |                |              |   "version": 0                              |   "type": "DATASET",           |
|    |                |              | }                                           |   "name": "__VAR_0"            |
|    |                |              |                                             | }                              |
|    |                |              |                                             | inputVar:                      |
-----+----------------+--------------+---------------------------------------------+---------------------------------
|  3 | Start          |              | {                                           | outputVar: {                   |
|    |                |              |   "execTime": "0(us)",                      |   "colNames": [],              |
|    |                |              |   "rows": 0,                                |   "type": "DATASET",           |
|    |                |              |   "totalTime": "20(us)",                    |   "name": "__Start_3"          |
|    |                |              |   "version": 0                              | }                              |
|    |                |              | }                                           |                                |
-----+----------------+--------------+---------------------------------------------+---------------------------------

看起来二者都是扫描全表

  1. 因为 match 匹配在 nebula 里,需要判断 v2 是不是悬挂边,所以 limit 推不到 Traverse 里。
    只能先拿到 e 所有边,然后 append vertex 的时候去 limit。
  2. 当然,我们也可以优化 append vertex 10 个就返回,但是从你的 profile 上,Traverse 比较慢。

总共返回的边的数量是12万,十几秒的性能是符合预期的吗

此话题已在最后回复的 30 天后被自动关闭。不再允许新回复。