find all path edge filter 新功能尝鲜

[Reid00]

提问参考模版：

• nebula 版本：最新master (commit id: 0dfa699 )
• 部署方式（分布式 / 单机 / Docker / DBaaS）：单机
• 是否为线上版本：N
• 硬件信息
  • 磁盘（ 推荐使用 SSD）
  • CPU、内存信息
• 问题的具体描述

1. FIND ALL PATH 支持属性过滤，但是我在测试是发现似乎没用，是哪里用的不对吗？

find noloop path with prop from "player102" to "team201" OVER * BIDIRECT

结果:

或者文字版:

(root@nebula) [Test]> find noloop path with prop from "player102" to "team201" OVER * BIDIRECT
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| path
                                                                                                  |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| <("player102" :player{age: 33, name: "LaMarcus Aldridge"})-[:follow@0 {degree: 75}]->("player101" :player{age: 36, name: "Tony Parker"})-[:serve@0 {end_year: 2018, start_year: 1999}]->("team201" :team{name: "Nuggets"})>                                                                              |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| <("player102" :player{age: 33, name: "LaMarcus Aldridge"})<-[:follow@0 {degree: 90}]-("player100" :player{age: 42, name: "Tim Duncan"})-[:follow@0 {degree: 95}]->("player101" :player{age: 36, name: "Tony Parker"})-[:serve@0 {end_year: 2018, start_year: 1999}]->("team201" :team{name: "Nuggets"})> |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

但是当我执行，filter 功能时没有结果

(root@nebula) [Test]> find noloop path with prop from "player102" to "team201" OVER * BIDIRECT where follow.degree > 70
Empty set (time spent 5947/6984 us)

Mon, 19 Jul 2021 11:44:02 CST

2. 看到PR 里面 query_concurrently = true 已经commit， 但是我该怎么测试，配置中的这个功能已经起作用了呢？Profile find all path 语句吗？

########## basics ##########
# Whether to run as a daemon process
--daemonize=true
# The file to host the process id
--pid_file=pids/nebula-storaged.pid
# Whether to use the configuration obtained from the configuration file
--local_config=true
--query_concurrently=true

[wey]

您好，感谢您的细心关注

1. edge filter merged 了，但是还么有正式 release，所以您需要用 master 的 nebula graph 才可以（2.0.1 不行），请问您是用的 master while condition 之后之后得到空结果么？

2. https://github.com/vesoft-inc/nebula-storage/pull/503 也是需要 master，更多信息请 @critical27 帮忙补充哈，多谢！

[Reid00]

hello 我用的就是master 分支的code 哈，今天早晨pull 了一遍代码，重新编译之后做的测试，我看一看到commit id 是最新的:

[INFO] nebula-metad(**0dfa699**): Running as 19843, Listening on 9559
[INFO] nebula-graphd(**0dfa699**): Running as 19906, Listening on 9669
[INFO] nebula-storaged(**0dfa699**): Running as 19931, Listening on 9779

[wey]

@Reid00

find noloop path with prop from "player102" to "team201" OVER * BIDIRECT where follow.degree > 70

看起来第一个问题是因为 follow.degree > 70 意味着 path 的每一个 edge都是 follow。

比如会过滤掉 serve 的边，这么写就出结果了。

$ find noloop path with prop from "player102" to "team201" OVER * BIDIRECT where follow.degree > 70 or serve.start_year > 1900

+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| path                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| <("player102" :player{age: 33, name: "LaMarcus Aldridge"})-[:follow@0 {degree: 75}]->("player100" :player{age: 42, name: "Tim Duncan"})<-[:follow@0 {degree: 80}]-("player144" :player{age: 47, name: "Shaquille O'Neal"})-[:follow@0 {degree: 100}]->("player145" :player{age: 31, name: "JaVale McGee"})-[:serve@0 {end_year: 2015, start_year: 2012}]->("team201" :team{name: "Nuggets"})>                                                                                                                                      |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| <("player102" :player{age: 33, name: "LaMarcus Aldridge"})-[:serve@0 {end_year: 2019, start_year: 2015}]->("team204" :team{name: "Spurs"})<-[:serve@0 {end_year: 2018, start_year: 2010}]-("player105" :player{age: 31, name: "Danny Green"})-[:follow@0 {degree: 80}]->("player116" :player{age: 34, name: "LeBron James"})<-[:follow@0 {degree: 90}]-("player128" :player{age: 34, name: "Carmelo Anthony"})-[:serve@0 {end_year: 2011, start_year: 2003}]->("team201" :team{name: "Nuggets"})>                                  |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| <("player102" :player{age: 33, name: "LaMarcus Aldridge"})-[:follow@0 {degree: 75}]->("player100" :player{age: 42, name: "Tim Duncan"})<-[:follow@0 {degree: 99}]-("player113" :player{age: 29, name: "Dejounte Murray"})-[:follow@0 {degree: 99}]->("player116" :player{age: 34, name: "LeBron James"})<-[:follow@0 {degree: 90}]-("player128" :player{age: 34, name: "Carmelo Anthony"})-[:serve@0 {end_year: 2011, start_year: 2003}]->("team201" :team{name: "Nuggets"})>                                                      |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| <("player102" :player{age: 33, name: "LaMarcus Aldridge"})<-[:follow@0 {degree: 90}]-("player101" :player{age: 36, name: "Tony Parker"})<-[:follow@0 {degree: 99}]-("player113" :player{age: 29, name: "Dejounte Murray"})-[:follow@0 {degree: 99}]->("player116" :player{age: 34, name: "LeBron James"})<-[:follow@0 {degree: 90}]-("player128" :player{age: 34, name: "Carmelo Anthony"})-[:serve@0 {end_year: 2011, start_year: 2003}]->("team201" :team{name: "Nuggets"})>                                                     |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
...

[Reid00]

谢谢，理解了。这个角度理解确实这样。
现在等第二个问题 query_concurrently= true feature 如何测试呢？

[critical27]

这该咋回答 看代码 → 看cpu → 看时间

[wey]

@Reid00

现在等第二个问题 query_concurrently= true feature 如何测试呢？

看起来 processor 里都是 doProcess 之后才有判断区分:

void GetNeighborsProcessor::doProcess(const cpp2::GetNeighborsRequest& req) {
...
    if (!FLAGS_query_concurrently) {
        runInSingleThread(req, limit, random);
    } else {
        runInMultipleThread(req, limit, random);
    }
}

...


void GetNeighborsProcessor::runInSingleThread(const cpp2::GetNeighborsRequest& req,
                                              int64_t limit,
                                              bool random) {
    contexts_.emplace_back(RunTimeContext(planContext_.get()));
    expCtxs_.emplace_back(StorageExpressionContext(spaceVidLen_, isIntId_));
    auto plan = buildPlan(&contexts_.front(), &expCtxs_.front(), &resultDataSet_, limit, random);
    std::unordered_set<PartitionID> failedParts;
    for (const auto& partEntry : req.get_parts()) {
        auto partId = partEntry.first;
        for (const auto& row : partEntry.second) {
            CHECK_GE(row.values.size(), 1);
            auto vId = row.values[0].getStr();

            if (!NebulaKeyUtils::isValidVidLen(spaceVidLen_, vId)) {
                LOG(ERROR) << "Space " << spaceId_ << ", vertex length invalid, "
                           << " space vid len: " << spaceVidLen_ << ",  vid is " << vId;
                pushResultCode(nebula::cpp2::ErrorCode::E_INVALID_VID, partId);
                onFinished();
                return;
            }

            // the first column of each row would be the vertex id
            auto ret = plan.go(partId, vId);
            if (ret != nebula::cpp2::ErrorCode::SUCCEEDED) {
                if (failedParts.find(partId) == failedParts.end()) {
                    failedParts.emplace(partId);
                    handleErrorCode(ret, spaceId_, partId);
                }
            }
        }
    }
    onProcessFinished();
    onFinished();
}

void GetNeighborsProcessor::runInMultipleThread(const cpp2::GetNeighborsRequest& req,
                                                int64_t limit,
                                                bool random) {
    for (size_t i = 0; i < req.get_parts().size(); i++) {
        nebula::DataSet result = resultDataSet_;
        results_.emplace_back(std::move(result));
        contexts_.emplace_back(RunTimeContext(planContext_.get()));
        expCtxs_.emplace_back(StorageExpressionContext(spaceVidLen_, isIntId_));
    }
    size_t i = 0;
    std::vector<folly::Future<std::pair<nebula::cpp2::ErrorCode, PartitionID>>> futures;
    for (const auto& [partId, rows] : req.get_parts()) {
        futures.emplace_back(
            runInExecutor(&contexts_[i], &expCtxs_[i], &results_[i], partId, rows, limit, random));
        i++;
    }

    folly::collectAll(futures).via(executor_).thenTry([this] (auto&& t) mutable {
        CHECK(!t.hasException());
        const auto& tries = t.value();
        for (size_t j = 0; j < tries.size(); j++) {
            CHECK(!tries[j].hasException());
            const auto& [code, partId] = tries[j].value();
            if (code != nebula::cpp2::ErrorCode::SUCCEEDED) {
                handleErrorCode(code, spaceId_, partId);
            } else {
                resultDataSet_.append(std::move(results_[j]));
            }
        }
        this->onProcessFinished();
        this->onFinished();
    });
}

所以从 profile 的name 那一列，应该看不出来区别，不过如果有两个环境，像 @critical27 说的，可以比较一下中间的时间

vesoft-inc/nebula-storage/blob/0dfa6994390def2250337bb2b2aa891a944fd47a/src/storage/query/GetNeighborsProcessor.cpp#L65

[Reid00]

@wey

较一下中间的时

好的，收到，谢谢 @critical27 @wey .