Nebula 3.5.0 Storage 无法启动

  • nebula 版本:3.5.0
  • 部署方式:分布式
  • 安装方式:RPM
  • 是否上生产环境:Y
  • 硬件信息
    • 磁盘 7T * 12 (多目录部署)
    • CPU 32 核
  • 问题的具体描述

数据入库方式为Exchange方式,当数据正在执行Ingest的时候,出现To many Open Files。。。。导致Storage进程挂掉。手动重启Stoage的时候,其中一块盘Scan的时候报错:报错信息如下:

2023/09/21-14:05:47.702176 139701153380096 [ERROR] [table/block_based/block_based_table_reader.cc:894] Corruption: An external sst file with version 2 have global seqno property with value ü€‡, while largest seqno in the file is 8880037
2023/09/21-14:05:47.702329 139701163869952 [ERROR] [table/block_based/block_based_table_reader.cc:894] Corruption: An external sst file with version 2 have global seqno property with value ö€‡, while largest seqno in the file is 8880031
2023/09/21-14:05:47.702374 139701174359808 [ERROR] [table/block_based/block_based_table_reader.cc:894] Corruption: An external sst file with version 2 have global seqno property with value ô€‡, while largest seqno in the file is 8880029
2023/09/21-14:05:47.709183 139702580945152 [WARN] [db/db_impl/db_impl_open.cc:2112] DB::Open() **failed: Corruption: An external sst file with version 2 have global seqno property with value 퀇, while largest seqno in the file is 8880022 in file /datag/nebula/storage/nebula/1/data/MANIFEST-126962**
**2023/09/21-14:05:47.587323 139702580945152 **RocksDB version: 7.5.3**
2023/09/21-14:05:47.587389 139702580945152 Git sha 0
2023/09/21-14:05:47.587394 139702580945152 Compile date 2023-01-28 06:47:39
2023/09/21-14:05:47.587397 139702580945152 DB SUMMARY
2023/09/21-14:05:47.587400 139702580945152 DB Session ID:  9WT9FN934NFP89DLAFCE
2023/09/21-14:05:47.615167 139702580945152 CURRENT file:  CURRENT
2023/09/21-14:05:47.615173 139702580945152 IDENTITY file:  IDENTITY
2023/09/21-14:05:47.615180 139702580945152 MANIFEST file:  MANIFEST-126962 size: 5871260 Bytes
2023/09/21-14:05:47.615184 139702580945152 SST files in /datag/nebula/storage/nebula/1/data dir, Total Num: 19181, files: 091649.sst 091650.sst 091651.sst 091652.sst 091653.sst 091654.sst 091655.sst 091656.sst 091657.sst**

先增加ulimit -n的设置

此话题已在最后回复的 30 天后被自动关闭。不再允许新回复。