提问参考模版:
-
nebula 版本:(为节省回复者核对版本信息的时间,首次发帖的版本信息记得以截图形式展示)
v3.2.1 基线 自己编译的
-
部署方式:分布式
-
安装方式:源码编译
-
是否上生产环境:Y
- OS:
uname -a
Linux ncn4a-wisemlopsdppservice-32-203-100 4.18.0-147.5.1.6.h934.eulerosv2r9.x86_64 #1 SMP Sat Feb 4 09:00:27 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
- Compiler:
g++ --version
orclang++ --version
g++ (Ubuntu 10.5.0-1ubuntu1~20.04) 10.5.0
- CPU:
lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 42 bits physical, 48 bits virtual
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Gold 6266C CPU @ 3.
00GHz
Stepping: 7
CPU MHz: 3000.000
BogoMIPS: 6000.00
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 256 KiB
L1i cache: 256 KiB
L2 cache: 8 MiB
L3 cache: 30.3 MiB
NUMA node0 CPU(s): 0-15
Vulnerability Itlb multihit: Processor vulnerable
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Vulnerable: Clear CPU buffers attemp
ted, no microcode; SMT Host state un
known
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass
disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers
and __user pointer sanitization
Vulnerability Spectre v2: Vulnerable, IBPB: disabled, STIBP: d
isabled, PBRSB-eIBRS: Vulnerable
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Vulnerable: Clear CPU buffers attemp
ted, no microcode; SMT Host state un
known
Flags: fpu vme de pse tsc msr pae mce cx8 a
pic sep mtrr pge mca cmov pat pse36
clflush mmx fxsr sse sse2 ss ht sysc
all nx pdpe1gb rdtscp lm constant_ts
c rep_good nopl xtopology nonstop_ts
c cpuid tsc_known_freq pni pclmulqdq
ssse3 fma cx16 pcid sse4_1 sse4_2 x
2apic movbe popcnt tsc_deadline_time
r aes xsave avx f16c rdrand hypervis
or lahf_lm abm 3dnowprefetch invpcid
_single ssbd ibrs ibpb stibp ibrs_en
hanced fsgsbase tsc_adjust bmi1 hle
avx2 smep bmi2 erms invpcid rtm mpx
avx512f avx512dq rdseed adx smap clf
lushopt clwb avx512cd avx512bw avx51
2vl xsaveopt xsavec xgetbv1 arat avx
512_vnni md_clear flush_l1d arch_cap
abilities
- 问题的具体描述
使用nebula-java并发读写压测nebula graph 3节点的集群,客户端设置超时时间10ms,会频繁触发超时(故意设置短的超时时间为了触发超时)
graph 服务端会经常出现coredump异常重启
调用栈如下
Thread 79 "graph-netio25" received signal SIGSEGV, Segmentation fault.
0x0000000005e86ba9 in apache::thrift::Cpp2Connection::stop() ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
1183 /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h: No such file or directory.
(gdb) bt
#0 0x0000000005e86ba9 in apache::thrift::Cpp2Connection::stop() ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#1 0x0000000005e88889 in apache::thrift::Cpp2Connection::channelClosed(folly::exception_wrapper&&) ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#2 0x0000000005e9c567 in apache::thrift::HeaderServerChannel::messageChannelEOF() ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#3 0x0000000005e401d6 in apache::thrift::Cpp2Channel::processReadEOF() ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#4 0x0000000005e47132 in non-virtual thunk to wangle::ContextImpl<apache::thrift::Cpp2Channel>::readEOF() ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#5 0x0000000005e4568d in non-virtual thunk to wangle::ContextImpl<apache::thrift::FramingHandler>::readEOF() ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#6 0x0000000005e4550a in wangle::ContextImpl<apache::thrift::TAsyncTransportHandler>::fireReadEOF() ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#7 0x0000000005e43570 in non-virtual thunk to apache::thrift::TAsyncTransportHandler::readEOF() ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#8 0x0000000006013bae in folly::AsyncSocket::handleRead() ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#9 0x000000000600898a in folly::AsyncSocket::ioReady(unsigned short) ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#10 0x00000000060e2315 in ?? () at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#11 0x00000000060e297f in event_base_loop ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#12 0x000000000602180e in folly::EventBase::loopBody(int, bool) ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#13 0x00000000060220fe in folly::EventBase::loop() ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#14 0x0000000006024d38 in folly::EventBase::loopForever() ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#15 0x0000000005faf549 in folly::IOThreadPoolExecutor::threadRun(std::shared_ptr<folly::ThreadPoolExecutor::Thread>) ()
at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#16 0x0000000005fbdab7 in void folly::detail::function::FunctionTraits<void ()>::callBig<std::_Bind<void (folly::ThreadPoolExecutor::*(folly::ThreadPoolExecutor*, std::shared_ptr<folly::ThreadPoolExecutor::Thread>))(std::shared_ptr<folly::ThreadPoolExecutor::Thread>)> >(folly::detail::function::Data&) () at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#17 0x0000000003eb1430 in folly::detail::function::FunctionTraits<void ()>::operator()() (this=0x7f9f43212410)
at ../../../third-party/third-party/include/folly/Function.h:400
#18 0x0000000003f3a70a in folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}::operator()() (
--Type <RET> for more, q to quit, c to continue without paging--
__closure=0x7f9f43212410) at ../../../../third-party/third-party/include/folly/executors/thread_factory/NamedThreadFactory.h:40
#19 0x0000000003f7a23b in std::__invoke_impl<void, folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}>(std::__invoke_other, folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}&&) (__f=...)
at /opt/buildtools/gcc-10.3.0/include/c++/10.3.0/bits/invoke.h:60
#20 0x0000000003f79d05 in std::__invoke<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}>(std::__invoke_result&&, (folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}&&)...) (__fn=...)
at /opt/buildtools/gcc-10.3.0/include/c++/10.3.0/bits/invoke.h:95
#21 0x0000000003f79a00 in std::thread::_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=0x7f9f43212410) at /opt/buildtools/gcc-10.3.0/include/c++/10.3.0/thread:264
#22 0x0000000003f7963c in std::thread::_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}> >::operator()() (this=0x7f9f43212410) at /opt/buildtools/gcc-10.3.0/include/c++/10.3.0/thread:271
#23 0x0000000003f78f7e in std::thread::_State_impl<std::thread::_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}> > >::_M_run() (this=0x7f9f43212400) at /opt/buildtools/gcc-10.3.0/include/c++/10.3.0/thread:215
#24 0x00000000065f1e7e in std::execute_native_thread_routine (__p=0x7f9f43212400) at ../../../.././libstdc++-v3/src/c++11/thread.cc:78
#25 0x00007f9f43884f3b in ?? () from /usr/lib64/libpthread.so.0
#26 0x00007f9f437bc840 in clone () from /usr/lib64/libc.so.6