边 Schema 无故丢失

yulijun · 2023 年12 月 11 日 03:20

nebula 版本：（3.6.0）
部署方式：单机
安装方式： RPM
是否上生产环境： N
硬件信息
- 磁盘（ 50G）
- CPU 48核内存信息（32G）
问题的具体描述
当我创建完一条边后，另外一条边在studio里面不可见了，我接着创建这条丢失的边，报边已存在，而且都是了其他的一些数据，感觉是元数据问题，目前已经遇到了两次这种问题，以下是日志

Log file created at: 2023/12/11 10:44:35
Log file created at: 2023/12/11 10:44:35
Running on machine: iZuf68j5ei93s92673e81gZ
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
E20231211 10:44:35.777556  5541 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 10:44:37.530948  5540 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `catalogs_item_ref'.
E20231211 10:44:41.746284  5540 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `catalogs_item_ref'.
E20231211 10:44:42.263425  5541 Serializer.h:43] Thrift serialization is only defined for structs and unions, not containers thereof. Attemping to serialize a value of type `nebula::Value`.
E20231211 10:44:43.406517  5540 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `catalogs_item_ref'.
E20231211 10:45:03.021620  5540 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `catalogs_item_ref'.
E20231211 10:45:23.406958  5540 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `catalogs_item_ref'.
E20231211 10:45:29.565722  5541 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `catalogs_item_ref'.
E20231211 10:45:36.898085  5540 OptimizerUtils.cpp:683] No valid index found
E20231211 10:45:36.904408  5540 OptimizerUtils.cpp:683] No valid index found
E20231211 10:45:36.927130  5540 MutateValidator.cpp:85] Unknown column `ecu_name' in schema
E20231211 10:45:36.927188  5540 QueryInstance.cpp:151] SemanticError: Unknown column `ecu_name' in schema, query: USE `KGDI-UAT`;INSERT VERTEX IF NOT EXISTS `fault`(ecu_name,fault_code,fault_index,fault_des,fault_level,trigger_condition,om_type,om_frequency,setting_condition,clear_condition,fault_effect,light_msg,typical_case_key,perceived_name) VALUES 'fault_key_00000000000000000007':($ecu_name,$fault_code,$fault_index,$fault_des,$fault_level,$trigger_condition,$om_type,$om_frequency,$setting_condition,$clear_condition,$fault_effect,$light_msg,$typical_case_key,$perceived_name);
E20231211 10:45:59.098986  5541 OptimizerUtils.cpp:683] No valid index found
E20231211 10:45:59.105798  5540 OptimizerUtils.cpp:683] No valid index found
E20231211 10:45:59.124886  5540 MutateValidator.cpp:85] Unknown column `ecu_name' in schema
E20231211 10:45:59.124922  5540 QueryInstance.cpp:151] SemanticError: Unknown column `ecu_name' in schema, query: USE `KGDI-UAT`;INSERT VERTEX IF NOT EXISTS `fault`(ecu_name,fault_code,fault_index,fault_des,fault_level,trigger_condition,om_type,om_frequency,setting_condition,clear_condition,fault_effect,light_msg,typical_case_key,perceived_name) VALUES 'fault_key_00000000000000000008':($ecu_name,$fault_code,$fault_index,$fault_des,$fault_level,$trigger_condition,$om_type,$om_frequency,$setting_condition,$clear_condition,$fault_effect,$light_msg,$typical_case_key,$perceived_name);
E20231211 10:46:37.151978  5540 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `catalogs_item_ref'.
E20231211 10:47:04.433965  5540 OptimizerUtils.cpp:683] No valid index found
E20231211 10:47:04.440739  5540 OptimizerUtils.cpp:683] No valid index found
E20231211 10:48:53.536698  5540 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `dtc_info_config_ref_fault'.
E20231211 10:48:53.546321  5540 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `dtc_info_config_ref_fault'.
E20231211 10:48:56.711081  5541 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `dtc_info_config_ref_fault'.
E20231211 10:48:56.716691  5540 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `dtc_info_config_ref_fault'.
E20231211 10:49:05.257179  5540 QueryInstance.cpp:151] EdgeNotFound: EdgeName `dtc_info_config_ref_tag`, query: USE `KGDI-UAT`;INSERT EDGE IF NOT EXISTS `dtc_info_config_ref_tag`() VALUES 'dtc_info_config_00000000000000000004'->'tag_00000000000000000055':();
E20231211 10:49:11.330824  5540 QueryInstance.cpp:151] EdgeNotFound: EdgeName `dtc_info_config_ref_tag`, query: USE `KGDI-UAT`;INSERT EDGE IF NOT EXISTS `dtc_info_config_ref_tag`() VALUES 'dtc_info_config_00000000000000000005'->'tag_00000000000000000055':();
E20231211 10:49:35.782634  6420 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 10:49:42.245604  5541 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `catalogs_item_ref'.
E20231211 10:49:57.494071  5540 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `dtc_info_config_ref_fault'.
E20231211 10:49:57.502046  5540 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `dtc_info_config_ref_fault'.
E20231211 10:50:03.811309  5540 QueryInstance.cpp:151] SemanticError: `catalogs_item_ref': Unknown edge type, query: USE `KGDI-UAT`;MATCH ()-[e:catalogs_item_ref]->()RETURN e LIMIT 2147483647;
E20231211 10:50:20.053815  5541 QueryInstance.cpp:151] SemanticError: `catalogs_item_ref': Unknown edge type, query: USE `KGDI-UAT`;MATCH ()-[e:catalogs_item_ref]->()RETURN e LIMIT 2147483647;
E20231211 10:50:22.873802  5541 QueryInstance.cpp:151] SemanticError: `catalogs_item_ref': Unknown edge type, query: USE `KGDI-UAT`;MATCH ()-[e:catalogs_item_ref]->()RETURN e LIMIT 2147483647




Running on machine: iZuf68j5ei93s92673e81gZ
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
E20231128 10:31:44.654757 52430 FileUtils.cpp:377] Failed to read the directory "/usr/local/nebula/data/meta/nebula" (2): No such file or directory
E20231128 10:58:28.245502 52627 PeekingManager.h:115] Received SSL connection on non SSL port
E20231128 10:58:31.750223 52626 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 10:58:31.750401 52626 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 10:58:31.750425 52626 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231128 13:18:37.112341 52502 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 13:18:37.112521 52502 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 13:18:37.112543 52502 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231128 13:18:47.286175 52500 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 13:18:47.286337 52500 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 13:18:47.286360 52500 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231128 13:59:43.586961 52495 PeekingManager.h:115] Received SSL connection on non SSL port
E20231128 13:59:46.927001 52494 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 13:59:46.927173 52494 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 13:59:46.927196 52494 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231128 14:08:22.672634 52626 PeekingManager.h:115] Received SSL connection on non SSL port
E20231128 14:08:26.135637 52627 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 14:08:26.135787 52627 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 14:08:26.135807 52627 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231128 20:59:56.426636 52493 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 20:59:56.426779 52493 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 20:59:56.426798 52493 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231128 21:00:06.494736 52505 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 21:00:06.494894 52505 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 21:00:06.494935 52505 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231129 01:31:58.972949 52626 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231129 01:31:58.973028 52626 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231129 01:31:58.973080 52626 GeneratedCodeHelper.cpp:93] invalid message from client in function process


Log file created at: 2023/12/11 10:44:42
Running on machine: iZuf68j5ei93s92673e81gZ
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
E20231211 10:44:42.265820  6106 Serializer.h:43] Thrift serialization is only defined for structs and unions, not containers thereof. Attemping to deserialize a value of type `nebula::Value`.

yulijun · 2023 年12 月 11 日 03:38

已经可以稳定重现了，还好我做了快照。。。

yulijun · 2023 年12 月 11 日 03:44

再次重现的日志：

Log file created at: 2023/12/11 11:32:20
Running on machine: iZuf68j5ei93s92673e81gZ
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
E20231211 11:32:20.672996  7338 QueryInstance.cpp:151] SemanticError: `catalogs_item_ref': Unknown edge type, query: USE `KGDI-UAT`;MATCH ()-[e:catalogs_item_ref]->()RETURN e LIMIT 2147483647;




Log file created at: 2023/11/28 10:31:44
Running on machine: iZuf68j5ei93s92673e81gZ
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
E20231128 10:31:44.654757 52430 FileUtils.cpp:377] Failed to read the directory "/usr/local/nebula/data/meta/nebula" (2): No such file or directory
E20231128 10:58:28.245502 52627 PeekingManager.h:115] Received SSL connection on non SSL port
E20231128 10:58:31.750223 52626 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 10:58:31.750401 52626 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 10:58:31.750425 52626 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231128 13:18:37.112341 52502 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 13:18:37.112521 52502 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 13:18:37.112543 52502 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231128 13:18:47.286175 52500 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 13:18:47.286337 52500 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 13:18:47.286360 52500 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231128 13:59:43.586961 52495 PeekingManager.h:115] Received SSL connection on non SSL port
E20231128 13:59:46.927001 52494 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 13:59:46.927173 52494 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 13:59:46.927196 52494 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231128 14:08:22.672634 52626 PeekingManager.h:115] Received SSL connection on non SSL port
E20231128 14:08:26.135637 52627 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 14:08:26.135787 52627 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 14:08:26.135807 52627 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231128 20:59:56.426636 52493 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 20:59:56.426779 52493 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 20:59:56.426798 52493 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231128 21:00:06.494736 52505 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 21:00:06.494894 52505 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231128 21:00:06.494935 52505 GeneratedCodeHelper.cpp:93] invalid message from client in function process
E20231129 01:31:58.972949 52626 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231129 01:31:58.973028 52626 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231129 01:31:58.973080 52626 GeneratedCodeHelper.cpp:93] invalid message from client in function process







Log file created at: 2023/12/11 11:01:16
Running on machine: iZuf68j5ei93s92673e81gZ
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
E20231211 11:01:16.801924  6024 Serializer.h:43] Thrift serialization is only defined for structs and unions, not containers thereof. Attemping to deserialize a value of type `nebula::Value`.
E20231211 11:19:45.987038  5557 GeneratedCodeHelper.cpp:201] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231211 11:19:45.987154  5557 GeneratedCodeHelper.cpp:134] received invalid message from client: No version identifier... old protocol client in strict mode? sz=1195725856
E20231211 11:19:45.987185  5557 GeneratedCodeHelper.cpp:93] invalid message from client in function process

steam · 2023 年12 月 11 日 04:07

Studio 的版本号是多少，你是如何判断边数据丢失的？

在 studio 端执行

USE \`KGDI-UAT`;MATCH ()-[e:catalogs_item_ref]->()RETURN e LIMIT 2147483647

这个语句么？

yulijun · 2023 年12 月 11 日 05:55

studio的版本是3.5
在studio的网页上，点schema->相应的space->Edge Type页面上随便新增一个边，原来的边就没了，目前是38个边
目前的感觉是哪里限制了边的数量或者什么其他数量

steam · 2023 年12 月 11 日 06:12

3.5 的 studio 和 3.6 的 nebula 内核是不匹配的，你可以把 studio 升级下，3.6 内核对应的 stuido 是 3.8 版本的。

等你版本对齐之后，看下是不是还有这个问题。

yulijun · 2023 年12 月 11 日 06:12

好的，我这边再测试下

yulijun · 2023 年12 月 11 日 06:58

安装了studio3.8版本的，随便新增了一个边，第一个边catalogs_item_ref就没了,
以下是日志

Log file created at: 2023/12/11 12:10:03
Running on machine: iZuf68j5ei93s92673e81gZ
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
E20231211 12:10:03.128371  1814 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 12:10:20.542353  1813 Serializer.h:43] Thrift serialization is only defined for structs and unions, not containers thereof. Attemping to serialize a value of type `nebula::Value`.
E20231211 12:15:03.132891  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 12:20:03.133082  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 12:25:03.133255  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 12:30:03.133419  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 12:35:03.133587  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 12:40:03.133754  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 12:45:03.133919  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 12:50:03.134085  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 12:55:03.134241  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 13:00:03.134403  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 13:05:03.134567  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 13:10:03.134724  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 13:15:03.134891  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 13:20:03.135071  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 13:25:03.135227  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 13:30:03.135380  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 13:35:03.135535  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 13:40:03.135704  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 13:45:03.135890  2534 FunctionUdfManager.cpp:41] UDF Folder doesn't Exist!
E20231211 13:48:18.131498  1814 LabelIndexSeek.cpp:56] SemanticError: No valid index for label `catalogs_item_ref'.

steam · 2023 年12 月 11 日 07:04

你去 nebula-studio 的 console 里执行下

分别执行 SHOW SPACES、SHOW EDGES 和 SHOW EDGE INDEXES，看下结果。

yulijun · 2023 年12 月 11 日 07:11

新增边之前
$ SHOW SPACES;

Table

Name
KGDI-TEST
KGDI-UAT

KGDI-UAT下SHOW EDGES
show_edges.csv (861 字节)

KGDI-UAT下SHOW EDGE INDEXES
空的

KGDI-UAT下随意新增一个testedge121111边后
HOW SPACES;

[KGDI-UAT]> SHOW SPACES;

Table

Name
KGDI-TEST
KGDI-UAT

KGDI-UAT下SHOW EDGES
show_edges2.csv (858 字节)

KGDI-UAT下SHOW EDGE INDEXES
空的

steam · 2023 年12 月 11 日 07:13

按照这个设定，先把 udf 报错解决掉。

然后你插入边的话，找到这个边对应的图空间，先 use，到这个图空间里再创建边。如果你要看边的数据是否插入成功，你可以用 show stats 命令（它有一个使用前提，你可以看下文档：https://docs.nebula-graph.com.cn/3.6.0/3.ngql-guide/7.general-query-statements/6.show/14.show-stats/

yulijun · 2023 年12 月 11 日 07:47

已经将–enable_udf=true更改为false，并且重启了数据库，请问“先 use”是在哪里执行，我没有安装console

steam · 2023 年12 月 11 日 07:48

nebula-studio 左上角有个图空间，你选下你要用的图空间就可以了（如果我没记错的话

yulijun · 2023 年12 月 11 日 07:53

还是不行，下面是我新增边testedge202312112222前的stats，
show_stats.csv (2.2 KB)

新增后的
show_stats2.csv (2.2 KB)

steam · 2023 年12 月 11 日 07:56

论坛可以传视频的，你录个视频，让我瞅瞅。

yulijun · 2023 年12 月 11 日 07:56

还需要什么日志什么的吗？

steam · 2023 年12 月 11 日 07:56

不用，应该是操作问题。你先操作下。

yulijun · 2023 年12 月 11 日 08:41

步骤1：
链接：百度网盘请输入提取码
提取码：2ptg

步骤2：
链接：百度网盘请输入提取码
提取码：jgm0

steam · 2023 年12 月 11 日 09:14

step 1 和 2 我都看完了，不是创建成功了边类型么。还没到边插入 insert 这步呢。

yulijun · 2023 年12 月 11 日 09:24

是的，创建完边类型后第一个边类型catalogs_item_ref不见了