关于find noloop path 的profile问题

maslke · 2024 年10 月 11 日 05:44

nebula 版本：v3.8.0
部署方式：单机
安装方式：RPM
是否上生产环境：N

对于find noloop path查询，针对不同的起始点，在相同的steps和limits条件下，查询性能差异巨大。执行profile分析之后，发现buildAdjTime时间相差无几，主要的差异是build path time的时间。从profile中可以发现，查询关联的节点的个数差别也不大。

除了以上的因素外，还有哪些导致了请求的执行效率呢。

谢谢指导。

下面附上两个profile结果。

查询时间长的请求。

{
    "buildAdjTime reverse step[10]": "1175(us)",
    "buildAdjTime reverse step[1]": "19(us)",
    "buildAdjTime reverse step[2]": "324(us)",
    "buildAdjTime reverse step[3]": "572(us)",
    "buildAdjTime reverse step[4]": "554(us)",
    "buildAdjTime reverse step[5]": "821(us)",
    "buildAdjTime reverse step[6]": "856(us)",
    "buildAdjTime reverse step[7]": "1625(us)",
    "buildAdjTime reverse step[8]": "1139(us)",
    "buildAdjTime reverse step[9]": "1146(us)",
    "buildAdjTime step[10]": "1413(us)",
    "buildAdjTime step[1]": "13(us)",
    "buildAdjTime step[2]": "40(us)",
    "buildAdjTime step[3]": "153(us)",
    "buildAdjTime step[4]": "343(us)",
    "buildAdjTime step[5]": "591(us)",
    "buildAdjTime step[6]": "721(us)",
    "buildAdjTime step[7]": "1064(us)",
    "buildAdjTime step[8]": "1374(us)",
    "buildAdjTime step[9]": "1040(us)",
    "build_path_time": "123761391(us)",
    "conjunct_path_time": "123761346(us)",
    "execTime": "101(us)",
    "resp[0]": {
      "exec": "1349(us)",
      "host": "127.0.0.1:9779",
      "total": "3431(us)"
    },
    "reverse step[10]": [
      {
        "exec": "11291(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "29115(us)",
          "HashJoinNode": "27553(us)",
          "RelNode": "29133(us)",
          "SingleEdgeNode": "27404(us)"
        },
        "total": "13742(us)",
        "vertices": 196
      }
    ],
    "reverse step[1]": [
      {
        "exec": "488(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "329(us)",
          "HashJoinNode": "317(us)",
          "RelNode": "329(us)",
          "SingleEdgeNode": "315(us)"
        },
        "total": "1196(us)",
        "vertices": 1
      }
    ],
    "reverse step[2]": [
      {
        "exec": "699(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "1233(us)",
          "HashJoinNode": "1000(us)",
          "RelNode": "1234(us)",
          "SingleEdgeNode": "995(us)"
        },
        "total": "1260(us)",
        "vertices": 4
      }
    ],
    "reverse step[3]": [
      {
        "exec": "1176(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "2751(us)",
          "HashJoinNode": "2490(us)",
          "RelNode": "2753(us)",
          "SingleEdgeNode": "2476(us)"
        },
        "total": "2392(us)",
        "vertices": 17
      }
    ],
    "reverse step[4]": [
      {
        "exec": "3153(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "7096(us)",
          "HashJoinNode": "6473(us)",
          "RelNode": "7100(us)",
          "SingleEdgeNode": "6435(us)"
        },
        "total": "5049(us)",
        "vertices": 51
      }
    ],
    "reverse step[5]": [
      {
        "exec": "4245(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "9986(us)",
          "HashJoinNode": "9218(us)",
          "RelNode": "10002(us)",
          "SingleEdgeNode": "9150(us)"
        },
        "total": "6451(us)",
        "vertices": 97
      }
    ],
    "reverse step[6]": [
      {
        "exec": "6094(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "15481(us)",
          "HashJoinNode": "14225(us)",
          "RelNode": "15491(us)",
          "SingleEdgeNode": "14138(us)"
        },
        "total": "8330(us)",
        "vertices": 121
      }
    ],
    "reverse step[7]": [
      {
        "exec": "6684(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "16748(us)",
          "HashJoinNode": "15459(us)",
          "RelNode": "16760(us)",
          "SingleEdgeNode": "15362(us)"
        },
        "total": "9142(us)",
        "vertices": 149
      }
    ],
    "reverse step[8]": [
      {
        "exec": "7969(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "21457(us)",
          "HashJoinNode": "20260(us)",
          "RelNode": "21472(us)",
          "SingleEdgeNode": "20126(us)"
        },
        "total": "10921(us)",
        "vertices": 178
      }
    ],
    "reverse step[9]": [
      {
        "exec": "11294(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "29863(us)",
          "HashJoinNode": "28505(us)",
          "RelNode": "29879(us)",
          "SingleEdgeNode": "28371(us)"
        },
        "total": "13612(us)",
        "vertices": 173
      }
    ],
    "rows": 10,
    "step[10]": [
      {
        "exec": "9163(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "25722(us)",
          "HashJoinNode": "24420(us)",
          "RelNode": "25737(us)",
          "SingleEdgeNode": "24294(us)"
        },
        "total": "11299(us)",
        "vertices": 174
      }
    ],
    "step[1]": [
      {
        "exec": "482(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "330(us)",
          "HashJoinNode": "320(us)",
          "RelNode": "330(us)",
          "SingleEdgeNode": "319(us)"
        },
        "total": "1138(us)",
        "vertices": 1
      }
    ],
    "step[2]": [
      {
        "exec": "165(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "90(us)",
          "HashJoinNode": "61(us)",
          "RelNode": "90(us)",
          "SingleEdgeNode": "59(us)"
        },
        "total": "686(us)",
        "vertices": 2
      }
    ],
    "step[3]": [
      {
        "exec": "815(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "1136(us)",
          "HashJoinNode": "1012(us)",
          "RelNode": "1137(us)",
          "SingleEdgeNode": "1005(us)"
        },
        "total": "1651(us)",
        "vertices": 6
      }
    ],
    "step[4]": [
      {
        "exec": "1564(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "3112(us)",
          "HashJoinNode": "2783(us)",
          "RelNode": "3114(us)",
          "SingleEdgeNode": "2764(us)"
        },
        "total": "3057(us)",
        "vertices": 24
      }
    ],
    "step[5]": [
      {
        "exec": "1841(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "3521(us)",
          "HashJoinNode": "2963(us)",
          "RelNode": "3525(us)",
          "SingleEdgeNode": "2933(us)"
        },
        "total": "3599(us)",
        "vertices": 55
      }
    ],
    "step[6]": [
      {
        "exec": "3772(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "9559(us)",
          "HashJoinNode": "8726(us)",
          "RelNode": "9565(us)",
          "SingleEdgeNode": "8664(us)"
        },
        "total": "6082(us)",
        "vertices": 94
      }
    ],
    "step[7]": [
      {
        "exec": "4663(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "10003(us)",
          "HashJoinNode": "8964(us)",
          "RelNode": "10011(us)",
          "SingleEdgeNode": "8895(us)"
        },
        "total": "6851(us)",
        "vertices": 128
      }
    ],
    "step[8]": [
      {
        "exec": "8646(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "21374(us)",
          "HashJoinNode": "20165(us)",
          "RelNode": "21388(us)",
          "SingleEdgeNode": "20038(us)"
        },
        "total": "10971(us)",
        "vertices": 171
      }
    ],
    "step[9]": [
      {
        "exec": "7407(us)",
        "host": "127.0.0.1:9779",
        "storage_detail": {
          "GetNeighborsNode": "20452(us)",
          "HashJoinNode": "19279(us)",
          "RelNode": "20466(us)",
          "SingleEdgeNode": "19143(us)"
        },
        "total": "9893(us)",
        "vertices": 168
      }
    ],
    "totalTime": "123849832(us)",
    "version": 0
  }

查询时间短的请求。

{

    "buildAdjTime reverse step[10]": "908(us)",

    "buildAdjTime reverse step[11]": "1122(us)",

    "buildAdjTime reverse step[12]": "1295(us)",

    "buildAdjTime reverse step[1]": "11(us)",

    "buildAdjTime reverse step[2]": "11(us)",

    "buildAdjTime reverse step[3]": "73(us)",

    "buildAdjTime reverse step[4]": "68(us)",

    "buildAdjTime reverse step[5]": "81(us)",

    "buildAdjTime reverse step[6]": "123(us)",

    "buildAdjTime reverse step[7]": "338(us)",

    "buildAdjTime reverse step[8]": "599(us)",

    "buildAdjTime reverse step[9]": "762(us)",

    "buildAdjTime step[10]": "759(us)",

    "buildAdjTime step[11]": "815(us)",

    "buildAdjTime step[12]": "910(us)",

    "buildAdjTime step[13]": "1111(us)",

    "buildAdjTime step[1]": "13(us)",

    "buildAdjTime step[2]": "36(us)",

    "buildAdjTime step[3]": "76(us)",

    "buildAdjTime step[4]": "65(us)",

    "buildAdjTime step[5]": "110(us)",

    "buildAdjTime step[6]": "148(us)",

    "buildAdjTime step[7]": "238(us)",

    "buildAdjTime step[8]": "299(us)",

    "buildAdjTime step[9]": "463(us)",

    "build_path_time": "52843(us)",

    "conjunct_path_time": "52805(us)",

    "execTime": "84(us)",

    "resp[0]": {

      "exec": "990(us)",

      "host": "127.0.0.1:9779",

      "total": "2199(us)"

    },

    "reverse step[10]": [

      {

        "exec": "10332(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "24775(us)",

          "HashJoinNode": "23449(us)",

          "RelNode": "24788(us)",

          "SingleEdgeNode": "23330(us)"

        },

        "total": "12336(us)",

        "vertices": 152

      }

    ],

    "reverse step[11]": [

      {

        "exec": "11916(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "27932(us)",

          "HashJoinNode": "26626(us)",

          "RelNode": "27948(us)",

          "SingleEdgeNode": "26486(us)"

        },

        "total": "14465(us)",

        "vertices": 182

      }

    ],

    "reverse step[12]": [

      {

        "exec": "14880(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "37499(us)",

          "HashJoinNode": "35382(us)",

          "RelNode": "37518(us)",

          "SingleEdgeNode": "35076(us)"

        },

        "total": "17520(us)",

        "vertices": 199

      }

    ],

    "reverse step[1]": [

      {

        "exec": "219(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "70(us)",

          "HashJoinNode": "61(us)",

          "RelNode": "70(us)",

          "SingleEdgeNode": "59(us)"

        },

        "total": "851(us)",

        "vertices": 1

      }

    ],

    "reverse step[2]": [

      {

        "exec": "452(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "669(us)",

          "HashJoinNode": "652(us)",

          "RelNode": "669(us)",

          "SingleEdgeNode": "648(us)"

        },

        "total": "929(us)",

        "vertices": 2

      }

    ],

    "reverse step[3]": [

      {

        "exec": "997(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "1597(us)",

          "HashJoinNode": "1566(us)",

          "RelNode": "1597(us)",

          "SingleEdgeNode": "1562(us)"

        },

        "total": "1794(us)",

        "vertices": 2

      }

    ],

    "reverse step[4]": [

      {

        "exec": "1069(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "1285(us)",

          "HashJoinNode": "1211(us)",

          "RelNode": "1285(us)",

          "SingleEdgeNode": "1203(us)"

        },

        "total": "1672(us)",

        "vertices": 7

      }

    ],

    "reverse step[5]": [

      {

        "exec": "1069(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "1898(us)",

          "HashJoinNode": "1807(us)",

          "RelNode": "1901(us)",

          "SingleEdgeNode": "1796(us)"

        },

        "total": "1630(us)",

        "vertices": 15

      }

    ],

    "reverse step[6]": [

      {

        "exec": "813(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "1656(us)",

          "HashJoinNode": "1536(us)",

          "RelNode": "1658(us)",

          "SingleEdgeNode": "1524(us)"

        },

        "total": "1457(us)",

        "vertices": 16

      }

    ],

    "reverse step[7]": [

      {

        "exec": "2596(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "5778(us)",

          "HashJoinNode": "5043(us)",

          "RelNode": "5780(us)",

          "SingleEdgeNode": "5015(us)"

        },

        "total": "3872(us)",

        "vertices": 32

      }

    ],

    "reverse step[8]": [

      {

        "exec": "5161(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "12228(us)",

          "HashJoinNode": "11189(us)",

          "RelNode": "12236(us)",

          "SingleEdgeNode": "11129(us)"

        },

        "total": "7211(us)",

        "vertices": 76

      }

    ],

    "reverse step[9]": [

      {

        "exec": "7334(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "17010(us)",

          "HashJoinNode": "15499(us)",

          "RelNode": "17020(us)",

          "SingleEdgeNode": "15406(us)"

        },

        "total": "9088(us)",

        "vertices": 109

      }

    ],

    "rows": 10,

    "step[10]": [

      {

        "exec": "7282(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "16937(us)",

          "HashJoinNode": "16097(us)",

          "RelNode": "16948(us)",

          "SingleEdgeNode": "16013(us)"

        },

        "total": "8986(us)",

        "vertices": 96

      }

    ],

    "step[11]": [

      {

        "exec": "6114(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "14322(us)",

          "HashJoinNode": "13408(us)",

          "RelNode": "14332(us)",

          "SingleEdgeNode": "13333(us)"

        },

        "total": "7889(us)",

        "vertices": 119

      }

    ],

    "step[12]": [

      {

        "exec": "9821(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "27080(us)",

          "HashJoinNode": "25864(us)",

          "RelNode": "27094(us)",

          "SingleEdgeNode": "25737(us)"

        },

        "total": "12194(us)",

        "vertices": 144

      }

    ],

    "step[13]": [

      {

        "exec": "9984(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "24282(us)",

          "HashJoinNode": "22834(us)",

          "RelNode": "24299(us)",

          "SingleEdgeNode": "22697(us)"

        },

        "total": "12411(us)",

        "vertices": 170

      }

    ],

    "step[1]": [

      {

        "exec": "202(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "73(us)",

          "HashJoinNode": "60(us)",

          "RelNode": "73(us)",

          "SingleEdgeNode": "59(us)"

        },

        "total": "810(us)",

        "vertices": 1

      }

    ],

    "step[2]": [

      {

        "exec": "801(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "1016(us)",

          "HashJoinNode": "983(us)",

          "RelNode": "1016(us)",

          "SingleEdgeNode": "978(us)"

        },

        "total": "1277(us)",

        "vertices": 4

      }

    ],

    "step[3]": [

      {

        "exec": "1238(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "1930(us)",

          "HashJoinNode": "1854(us)",

          "RelNode": "1930(us)",

          "SingleEdgeNode": "1845(us)"

        },

        "total": "1919(us)",

        "vertices": 7

      }

    ],

    "step[4]": [

      {

        "exec": "822(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "2094(us)",

          "HashJoinNode": "2018(us)",

          "RelNode": "2096(us)",

          "SingleEdgeNode": "2009(us)"

        },

        "total": "1396(us)",

        "vertices": 8

      }

    ],

    "step[5]": [

      {

        "exec": "1051(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "1664(us)",

          "HashJoinNode": "1409(us)",

          "RelNode": "1664(us)",

          "SingleEdgeNode": "1398(us)"

        },

        "total": "1632(us)",

        "vertices": 15

      }

    ],

    "step[6]": [

      {

        "exec": "1042(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "2282(us)",

          "HashJoinNode": "2122(us)",

          "RelNode": "2283(us)",

          "SingleEdgeNode": "2106(us)"

        },

        "total": "1657(us)",

        "vertices": 21

      }

    ],

    "step[7]": [

      {

        "exec": "2631(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "4338(us)",

          "HashJoinNode": "4090(us)",

          "RelNode": "4339(us)",

          "SingleEdgeNode": "4066(us)"

        },

        "total": "3527(us)",

        "vertices": 31

      }

    ],

    "step[8]": [

      {

        "exec": "1981(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "4789(us)",

          "HashJoinNode": "4501(us)",

          "RelNode": "4792(us)",

          "SingleEdgeNode": "4472(us)"

        },

        "total": "2889(us)",

        "vertices": 42

      }

    ],

    "step[9]": [

      {

        "exec": "3660(us)",

        "host": "127.0.0.1:9779",

        "storage_detail": {

          "GetNeighborsNode": "9210(us)",

          "HashJoinNode": "8467(us)",

          "RelNode": "9217(us)",

          "SingleEdgeNode": "8416(us)"

        },

        "total": "5063(us)",

        "vertices": 65

      }

    ],

    "totalTime": "151326(us)",

    "version": 0

  }

MuYi-方扬 · 2024 年10 月 14 日 08:10

一般涉及到的因素：

是否有超级节点；
是否带属性；
路径深度是否太大
配置
FIND PATH - NebulaGraph Database 手册

maslke · 2024 年10 月 14 日 09:13

谢谢你的回复。

是否有超级节点：没有超级节点。度最大的节点，也不超过5.
是否带属性：带有属性。但两个查询，响应时间差异巨大。并且不带有属性进行测试，也对测试结果没有影响。
路径深度是否太大：深度是比较大，为25。但上述的两个查询，路径深度是一样的。

所以，比较迷惑，不知道两个查询的差异在哪里。

MuYi-方扬 · 2024 年10 月 14 日 10:34

方便把语句发下嘛？

maslke · 2024 年10 月 15 日 14:42

好的，查询语句是下面这种。

FIND NOLOOP PATH WITH PROP FROM "321121070000000000022750" TO "321120000000000937704205" OVER route BIDIRECT  UPTO 25 STEPS YIELD path as p | limit 10;

MuYi-方扬 · 2024 年10 月 16 日 00:57

看起来没有太特殊的。看你的描述应该是构建路径的时候花的时间比较长，那就主要是不同的点构建出的路径有区别。
我猜如果你把语句的 limit 10 去掉，结果应该不太一样

maslke · 2024 年10 月 16 日 07:10

实际测试下来，limit 10对最后的结果没有太大影响。

前面也咨询过，在find noloop path的时候，没有实现limit下推，实际也是这样。比如，设置limit 1，对最后的查询时间影响基本没有。

测试后发现路径的区别在于：

查询耗时很长的时候，路径少，并且路径的节点只有3-4个。
查询耗时短的时候，路径多，路径节点在10个以上。

是不是在双向BFS实现的时候，在给定的最大深度情况下，最终结果路径长短会对算法的实现有影响？

MuYi-方扬 · 2024 年10 月 16 日 07:28

嗯，limit10 没有下推对单个语句是没有太大影响的。

你上面不是说两个语句时间差别比较大么？把 limit 去掉，返回全量路径，你就知道两边的路径个数是否一样了

maslke · 2024 年10 月 16 日 08:11

测试结果：

结果有1000+，耗时很短。
结果只有3条，耗时特别长。

system · 2024 年11 月 15 日 08:12

此话题已在最后回复的 30 天后被自动关闭。不再允许新回复。