多进程查询报错

nebula 版本:3.4.0

部署方式(分布式 / 单机 / Docker / DBaaS):k8s nebula operator

是否为线上版本:N

问题的具体描述:

目前使用nebula3-python client最多只能创建6-7个session做多进程查询。目前已经把max_sessions_per_ip_per_user设置到300。

如果用更多sessions会报错:

Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Connect 10.43.151.158:9669 failed: socket error connecting to host 10.43.151.158, port 9669 (('10.43.151.158', 9669)): ConnectionRefusedError(111, 'Connection refused')
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Get connection failed: name 'logging' is not defined
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
Retry connect failed
---------------------------------------------------------------------------
_RemoteTraceback                          Traceback (most recent call last)
_RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/nebula3/gclient/net/Connection.py", line 148, in execute_parameter
    resp = self._connection.executeWithParameter(session_id, stmt, params)
  File "/usr/local/lib/python3.8/dist-packages/nebula3/graph/GraphService.py", line 1652, in executeWithParameter
    return self.recv_executeWithParameter()
  File "/usr/local/lib/python3.8/dist-packages/nebula3/graph/GraphService.py", line 1665, in recv_executeWithParameter
    (fname, mtype, rseqid) = self._iprot.readMessageBegin()
  File "/usr/local/lib/python3.8/dist-packages/nebula3/fbthrift/protocol/THeaderProtocol.py", line 157, in readMessageBegin
    self.trans._reset_protocol()
  File "/usr/local/lib/python3.8/dist-packages/nebula3/fbthrift/transport/THeaderTransport.py", line 238, in _reset_protocol
    self.readFrame(0)
  File "/usr/local/lib/python3.8/dist-packages/nebula3/fbthrift/transport/THeaderTransport.py", line 268, in readFrame
    word1 = self.getTransport().readAll(4)
  File "/usr/local/lib/python3.8/dist-packages/nebula3/fbthrift/transport/TTransport.py", line 72, in readAll
    chunk = self.read(need)
  File "/usr/local/lib/python3.8/dist-packages/nebula3/fbthrift/transport/TTransport.py", line 183, in read
    self.__rbuf = StringIO(self.__trans.read(max(sz, self.__rbuf_size)))
  File "/usr/local/lib/python3.8/dist-packages/nebula3/fbthrift/transport/TSocket.py", line 307, in read
    raise TTransportException(type=TTransportException.END_OF_FILE,
nebula3.fbthrift.transport.TTransport.TTransportException: TSocket read 0 bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/nebula3/gclient/net/Session.py", line 42, in execute_parameter
    resp = self._connection.execute_parameter(self._session_id, stmt, params)
  File "/usr/local/lib/python3.8/dist-packages/nebula3/gclient/net/Connection.py", line 156, in execute_parameter
    raise IOErrorException(
nebula3.Exception.IOErrorException: TSocket read 0 bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/tmp/ipykernel_2053800/3861980367.py", line 37, in run
    resp = session.execute(stmt)
  File "/usr/local/lib/python3.8/dist-packages/nebula3/gclient/net/Session.py", line 77, in execute
    return self.execute_parameter(stmt, None)
  File "/usr/local/lib/python3.8/dist-packages/nebula3/gclient/net/Session.py", line 55, in execute_parameter
    raise IOErrorException(
nebula3.Exception.IOErrorException: TSocket read 0 bytes
"""

The above exception was the direct cause of the following exception:

IOErrorException                          Traceback (most recent call last)
Cell In[13], line 61
     58     print("processExecutor: running in {} seconds".format(time.time() - start))
     59     return results
---> 61 results = processExecutor(stmt)
     62 results
     63 connection_pool.close()

Cell In[13], line 51, in processExecutor(stmt)
     49 for future in concurrent.futures.as_completed(runner):
     50     if future.exception() is not None:
---> 51         raise future.exception()
     52     else:
     53         rs = future.result()

IOErrorException: None

此话题已在最后回复的 30 天后被自动关闭。不再允许新回复。