find all path edge filter 新功能尝鲜

提问参考模版:

  • nebula 版本:最新master (commit id: 0dfa699 )
  • 部署方式(分布式 / 单机 / Docker / DBaaS):单机
  • 是否为线上版本:N
  • 硬件信息
    • 磁盘( 推荐使用 SSD)
    • CPU、内存信息
  • 问题的具体描述
  1. FIND ALL PATH 支持属性过滤,但是我在测试是发现似乎没用,是哪里用的不对吗?
find noloop path with prop from "player102" to "team201" OVER * BIDIRECT

结果:


或者文字版:

(root@nebula) [Test]> find noloop path with prop from "player102" to "team201" OVER * BIDIRECT
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| path
                                                                                                  |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| <("player102" :player{age: 33, name: "LaMarcus Aldridge"})-[:follow@0 {degree: 75}]->("player101" :player{age: 36, name: "Tony Parker"})-[:serve@0 {end_year: 2018, start_year: 1999}]->("team201" :team{name: "Nuggets"})>                                                                              |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| <("player102" :player{age: 33, name: "LaMarcus Aldridge"})<-[:follow@0 {degree: 90}]-("player100" :player{age: 42, name: "Tim Duncan"})-[:follow@0 {degree: 95}]->("player101" :player{age: 36, name: "Tony Parker"})-[:serve@0 {end_year: 2018, start_year: 1999}]->("team201" :team{name: "Nuggets"})> |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

但是当我执行,filter 功能时没有结果

(root@nebula) [Test]> find noloop path with prop from "player102" to "team201" OVER * BIDIRECT where follow.degree > 70
Empty set (time spent 5947/6984 us)

Mon, 19 Jul 2021 11:44:02 CST
  1. 看到PR 里面 query_concurrently = true 已经commit, 但是我该怎么测试,配置中的这个功能已经起作用了呢?Profile find all path 语句吗?
########## basics ##########
# Whether to run as a daemon process
--daemonize=true
# The file to host the process id
--pid_file=pids/nebula-storaged.pid
# Whether to use the configuration obtained from the configuration file
--local_config=true
--query_concurrently=true

您好,感谢您的细心关注:+1:t2:

  1. edge filter merged 了,但是还么有正式 release,所以您需要用 master 的 nebula graph 才可以(2.0.1 不行),请问您是用的 master while condition 之后之后得到空结果么?

  2. https://github.com/vesoft-inc/nebula-storage/pull/503 也是需要 master,更多信息请 @critical27 帮忙补充哈,多谢!

hello 我用的就是master 分支的code 哈,今天早晨pull 了一遍代码,重新编译之后做的测试,我看一看到commit id 是最新的:

[INFO] nebula-metad(**0dfa699**): Running as 19843, Listening on 9559
[INFO] nebula-graphd(**0dfa699**): Running as 19906, Listening on 9669
[INFO] nebula-storaged(**0dfa699**): Running as 19931, Listening on 9779

看起来第一个问题是因为 follow.degree > 70 意味着 path 的每一个 edge都是 follow

比如会过滤掉 serve 的边,这么写就出结果了。

$ find noloop path with prop from "player102" to "team201" OVER * BIDIRECT where follow.degree > 70 or serve.start_year > 1900

+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| path                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| <("player102" :player{age: 33, name: "LaMarcus Aldridge"})-[:follow@0 {degree: 75}]->("player100" :player{age: 42, name: "Tim Duncan"})<-[:follow@0 {degree: 80}]-("player144" :player{age: 47, name: "Shaquille O'Neal"})-[:follow@0 {degree: 100}]->("player145" :player{age: 31, name: "JaVale McGee"})-[:serve@0 {end_year: 2015, start_year: 2012}]->("team201" :team{name: "Nuggets"})>                                                                                                                                      |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| <("player102" :player{age: 33, name: "LaMarcus Aldridge"})-[:serve@0 {end_year: 2019, start_year: 2015}]->("team204" :team{name: "Spurs"})<-[:serve@0 {end_year: 2018, start_year: 2010}]-("player105" :player{age: 31, name: "Danny Green"})-[:follow@0 {degree: 80}]->("player116" :player{age: 34, name: "LeBron James"})<-[:follow@0 {degree: 90}]-("player128" :player{age: 34, name: "Carmelo Anthony"})-[:serve@0 {end_year: 2011, start_year: 2003}]->("team201" :team{name: "Nuggets"})>                                  |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| <("player102" :player{age: 33, name: "LaMarcus Aldridge"})-[:follow@0 {degree: 75}]->("player100" :player{age: 42, name: "Tim Duncan"})<-[:follow@0 {degree: 99}]-("player113" :player{age: 29, name: "Dejounte Murray"})-[:follow@0 {degree: 99}]->("player116" :player{age: 34, name: "LeBron James"})<-[:follow@0 {degree: 90}]-("player128" :player{age: 34, name: "Carmelo Anthony"})-[:serve@0 {end_year: 2011, start_year: 2003}]->("team201" :team{name: "Nuggets"})>                                                      |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| <("player102" :player{age: 33, name: "LaMarcus Aldridge"})<-[:follow@0 {degree: 90}]-("player101" :player{age: 36, name: "Tony Parker"})<-[:follow@0 {degree: 99}]-("player113" :player{age: 29, name: "Dejounte Murray"})-[:follow@0 {degree: 99}]->("player116" :player{age: 34, name: "LeBron James"})<-[:follow@0 {degree: 90}]-("player128" :player{age: 34, name: "Carmelo Anthony"})-[:serve@0 {end_year: 2011, start_year: 2003}]->("team201" :team{name: "Nuggets"})>                                                     |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
...

谢谢,理解了。这个角度理解确实这样。
现在等第二个问题 query_concurrently= true feature 如何测试呢?

这该咋回答 :sweat_smile: 看代码 → 看cpu → 看时间

1 个赞

看起来 processor 里都是 doProcess 之后才有判断区分:

void GetNeighborsProcessor::doProcess(const cpp2::GetNeighborsRequest& req) {
...
    if (!FLAGS_query_concurrently) {
        runInSingleThread(req, limit, random);
    } else {
        runInMultipleThread(req, limit, random);
    }
}

...


void GetNeighborsProcessor::runInSingleThread(const cpp2::GetNeighborsRequest& req,
                                              int64_t limit,
                                              bool random) {
    contexts_.emplace_back(RunTimeContext(planContext_.get()));
    expCtxs_.emplace_back(StorageExpressionContext(spaceVidLen_, isIntId_));
    auto plan = buildPlan(&contexts_.front(), &expCtxs_.front(), &resultDataSet_, limit, random);
    std::unordered_set<PartitionID> failedParts;
    for (const auto& partEntry : req.get_parts()) {
        auto partId = partEntry.first;
        for (const auto& row : partEntry.second) {
            CHECK_GE(row.values.size(), 1);
            auto vId = row.values[0].getStr();

            if (!NebulaKeyUtils::isValidVidLen(spaceVidLen_, vId)) {
                LOG(ERROR) << "Space " << spaceId_ << ", vertex length invalid, "
                           << " space vid len: " << spaceVidLen_ << ",  vid is " << vId;
                pushResultCode(nebula::cpp2::ErrorCode::E_INVALID_VID, partId);
                onFinished();
                return;
            }

            // the first column of each row would be the vertex id
            auto ret = plan.go(partId, vId);
            if (ret != nebula::cpp2::ErrorCode::SUCCEEDED) {
                if (failedParts.find(partId) == failedParts.end()) {
                    failedParts.emplace(partId);
                    handleErrorCode(ret, spaceId_, partId);
                }
            }
        }
    }
    onProcessFinished();
    onFinished();
}

void GetNeighborsProcessor::runInMultipleThread(const cpp2::GetNeighborsRequest& req,
                                                int64_t limit,
                                                bool random) {
    for (size_t i = 0; i < req.get_parts().size(); i++) {
        nebula::DataSet result = resultDataSet_;
        results_.emplace_back(std::move(result));
        contexts_.emplace_back(RunTimeContext(planContext_.get()));
        expCtxs_.emplace_back(StorageExpressionContext(spaceVidLen_, isIntId_));
    }
    size_t i = 0;
    std::vector<folly::Future<std::pair<nebula::cpp2::ErrorCode, PartitionID>>> futures;
    for (const auto& [partId, rows] : req.get_parts()) {
        futures.emplace_back(
            runInExecutor(&contexts_[i], &expCtxs_[i], &results_[i], partId, rows, limit, random));
        i++;
    }

    folly::collectAll(futures).via(executor_).thenTry([this] (auto&& t) mutable {
        CHECK(!t.hasException());
        const auto& tries = t.value();
        for (size_t j = 0; j < tries.size(); j++) {
            CHECK(!tries[j].hasException());
            const auto& [code, partId] = tries[j].value();
            if (code != nebula::cpp2::ErrorCode::SUCCEEDED) {
                handleErrorCode(code, spaceId_, partId);
            } else {
                resultDataSet_.append(std::move(results_[j]));
            }
        }
        this->onProcessFinished();
        this->onFinished();
    });
}



所以从 profile 的name 那一列,应该看不出来区别,不过如果有两个环境,像 @critical27 说的,可以比较一下中间的时间 :smiley:

好的,收到,谢谢 @critical27 @wey .

1 个赞

浙ICP备20010487号