nebula data目录下的数据不能直接用rocksdb解析

问答 问题
1

nebula data目录下的数据不能直接用rocksdb解析。
nebula版本 2.0.1
rocksdb 6.4.6

访问 /data/nebula/data/meta/nebula/0/data/
Failed: Corruption: Snappy not supported or corrupted Snappy compressed block contents

访问 /usr/local/nebula/data/storage/nebula/1/data/
Failed: Corruption: LZ4 not supported or corrupted LZ4 compressed block contents

2
  1. 你要访问的data目录，是跟wal目录同级的吗？
  2. 看看rocksdb的版本是否匹配
3

是的，和wal同级。
nebulagraph 的 rocksdb版本在哪里看？

4

当前版本用的rocksdb-6.15.5.tar.gz.
不过之前刚升级到6.15.5的。

5

我查了一下，2.0.1的版本是6.7.3，我编译一个这个版本的rocksdb试试看

1 个赞
6

两个地方修正一下：
1.通过 docker 镜像 vesoft/nebula-dev:centos7 下的目录 /opt/vesoft/third-party/include/rocksdb/version.h 找到使用的rocksdb版本号是6.7.3

cat /opt/vesoft/third-party/include/rocksdb/version.h
// Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
//  This source code is licensed under both the GPLv2 (found in the
//  COPYING file in the root directory) and Apache 2.0 License
//  (found in the LICENSE.Apache file in the root directory).
#pragma once

#define ROCKSDB_MAJOR 6
#define ROCKSDB_MINOR 7
#define ROCKSDB_PATCH 3

// Do not use these. We made the mistake of declaring macros starting with
// double underscore. Now we have to live with our choice. We'll deprecate these
// at some point
#define __ROCKSDB_MAJOR__ ROCKSDB_MAJOR
#define __ROCKSDB_MINOR__ ROCKSDB_MINOR
#define __ROCKSDB_PATCH__ ROCKSDB_PATCH

2.编译rocksdb时注意使用 -DWITH_LZ4=ON -DWITH_ZLIB=ON -DWITH_SNAPPY=ON 开启几个压缩算法的支持。因为它默认是关闭的。
cmake -DCMAKE_INSTALL_PREFIX=/usr/local/rocksdb -DWITH_LZ4=ON -DWITH_ZLIB=ON -DWITH_SNAPPY=ON

...
option(WITH_JEMALLOC "build with JeMalloc" OFF)
option(WITH_SNAPPY "build with SNAPPY" OFF)
option(WITH_LZ4 "build with lz4" OFF)
option(WITH_ZLIB "build with zlib" OFF)
option(WITH_ZSTD "build with zstd" OFF)
...
1 个赞