版本号:V2.0.0
问题:rebuild index对于新的prop失败。
issue的复现步骤:
创建一个tag:player
(root@nebula) [nba]> desc tag player
+--------+----------+-------+---------+
| Field | Type | Null | Default |
+--------+----------+-------+---------+
| "name" | "string" | "YES" | |
+--------+----------+-------+---------+
| "age" | "int64" | "YES" | |
插入一些数据:
(root@nebula) [nba]> lookup on player yield player.name, player.age
+---------------------+---------------------+------------+
| VertexID | player.name | player.age |
+---------------------+---------------------+------------+
| "Aron Baynes" | "Aron Baynes" | 32 |
+---------------------+---------------------+------------+
| "Ben Simmons" | "Ben Simmons" | 22 |
+---------------------+---------------------+------------+
| "Blake Griffin" | "Blake Griffin" | 30 |
+---------------------+---------------------+------------+
执行alter tag增加一个属性:
(root@nebula) [nba]> ALTER TAG player ADD (new_name string);
Execution succeeded (time spent 5773/6190 us)
(root@nebula) [nba]> lookup on player yield player.name, player.age, player.new_name
+---------------------+---------------------+------------+-----------------+
| VertexID | player.name | player.age | player.new_name |
+---------------------+---------------------+------------+-----------------+
| "Aron Baynes" | "Aron Baynes" | 32 | __NULL__ |
+---------------------+---------------------+------------+-----------------+
| "Ben Simmons" | "Ben Simmons" | 22 | __NULL__ |
+---------------------+---------------------+------------+-----------------+
| "Blake Griffin" | "Blake Griffin" | 30 | __NULL__ |
创建一个新的组合索引:player_new_name
(root@nebula) [nba]> create tag index player_new_name on player(name(10), new_name(10))
Execution succeeded (time spent 7067/7389 us)
Thu, 04 Nov 2021 20:43:32 CST
重新rebuild这个新的index:player_new_name
(root@nebula) [nba]> rebuild tag index player_new_name
+------------+
| New Job Id |
+------------+
| 9168 |
+------------+
(root@nebula) [nba]> show job 9168
+----------------+---------------------+------------+-------------------------+-------------------------+
| Job Id(TaskId) | Command(Dest) | Status | Start Time | Stop Time |
+----------------+---------------------+------------+-------------------------+-------------------------+
| 9168 | "REBUILD_TAG_INDEX" | "FINISHED" | 2021-11-04T12:43:58.000 | 2021-11-04T12:43:58.000 |
+----------------+---------------------+------------+-------------------------+-------------------------+
| 0 | "xxx.xxx.xxx.xxx" | "FINISHED" | 2021-11-04T12:43:58.000 | 2021-11-04T12:45:50.000 |
+----------------+---------------------+------------+-------------------------+-------------------------+
| 1 | "xxx.xxx.xxx.xxx" | "FINISHED" | 2021-11-04T12:43:58.000 | 2021-11-04T12:45:50.000 |
+----------------+---------------------+------------+-------------------------+-------------------------+
| 2 | "xxx.xxx.xxx.xxx" | "FINISHED" | 2021-11-04T12:43:58.000 | 2021-11-04T12:45:50.000 |
+----------------+---------------------+------------+-------------------------+-------------------------+
但其实看nebula-storaged的日志会发现很多错误,实际对于已经插入的数据,rebuild index是失败的。
E1104 20:43:58.509814 18879 IndexKeyUtils.cpp:112] prop error by : new_name. status : Unknown prop
W1104 20:43:58.509853 18879 RebuildTagIndexTask.cpp:128] Collect index value failed
E1104 20:43:58.509876 18879 IndexKeyUtils.cpp:112] prop error by : new_name. status : Unknown prop
W1104 20:43:58.509896 18879 RebuildTagIndexTask.cpp:128] Collect index value failed
E1104 20:43:58.509945 18879 IndexKeyUtils.cpp:112] prop error by : new_name. status : Unknown prop
W1104 20:43:58.509963 18879 RebuildTagIndexTask.cpp:128] Collect index value failed
E1104 20:43:58.509982 18879 IndexKeyUtils.cpp:112] prop error by : new_name. status : Unknown prop
W1104 20:43:58.510000 18879 RebuildTagIndexTask.cpp:128] Collect index value failed
E1104 20:43:58.510020 18879 IndexKeyUtils.cpp:112] prop error by : new_name. status : Unknown prop
W1104 20:43:58.510036 18879 RebuildTagIndexTask.cpp:128] Collect index value failed
这个问题的原因是,老的数据里面的val使用的是老的schema_ver=0,schema_ver=0的schema里面还没有这个新的prop(new_name),在IndexKeyUtils::collectIndexValues中会找不到player_new_name这个index需要的属性(new_name),所以才会报错。
IndexKeyUtils::collectIndexValues(RowReader* reader,
const std::vector<nebula::meta::cpp2::ColumnDef>& cols) {
if (reader == nullptr) {
return Status::Error("Invalid row reader");
}
std::vector<Value> values;
for (const auto& col : cols) {
auto v = reader->getValueByName(col.get_name());
auto isNullable = col.__isset.nullable && *(col.get_nullable());
auto ret = checkValue(v, isNullable);
if (!ret.ok()) {
LOG(ERROR) << "prop error by : " << col.get_name()
<< ". status : " << ret;
return ret;
}
values.emplace_back(std::move(v));
}
return encodeValues(std::move(values), cols);
}
问题就是对于新的prop创建的index,在rebuild包含这个prop的index时,老的val里面找不到这个prop,导致rebuild index失败。从用户角度来说这种情况应该按照null处理,而不是按照prop error处理。