Thrift rpc call failed: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused

问答 问题
1

查看状态：

[root@redhat6 scripts]# ./nebula.service status  all                    
[WARN] The maximum files allowed to open might be too few: 1024
[INFO] nebula-metad: Exited
[INFO] nebula-graphd: Exited
[INFO] nebula-storaged: Running as 15547, Listening on 44500

查看日志文件：
文件：nebula-storaged.ERROR -> nebula-storaged.redhat6.redhat.log.ERROR.20200415-223242.15547：
内容如下：

 Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused

文件：nebula-metad.redhat6.redhat.log.WARNING.20200415-223238.15463：
内容如下：

Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0415 22:32:38.944437 15532 AsyncServerSocket.cpp:762] failed to set SO_REUSEPORT on async server socket Protocol not available
E0415 22:32:38.945001 15510 ThriftServer.cpp:440] Got an exception while setting up the server: 92failed to bind to async server socket: [::]:0: Protocol not available
E0415 22:32:38.945057 15510 RaftexService.cpp:90] Setup the Raftex Service failed, error: 92failed to bind to async server socket: [::]:0: Protocol not available
E0415 22:32:38.949586 15463 NebulaStore.cpp:47] Start the raft service failed
E0415 22:32:38.949597 15463 MetaDaemon.cpp:88] Nebula store init failed
E0415 22:32:38.949796 15463 MetaDaemon.cpp:215] Init kv failed!

初步观察是因为用nebula.service start all的时候没有打开metad组件，graphd组件在打开后也会立马自动关闭，所以无法链接到nebulaGraph，但是日志中出现的问题不知道如何解决？求解

3

可以检查一下系统"open files"的限制，猜测目前应该是1024 ：ulimit -a | grep “open files”
修改系统限制，并重启启动下试试 ： ulimit -n 65536

1 个赞
4

修改open files的限制之后
尝试使用 /usr/local/nebula/bin/nebula -u user -p password 连接
还是报相同的问题

“E0416 02:39:22.163220  3287 GraphClient.cpp:54] Thrift rpc call failed: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused”
5

./nebula.service status all 是什么状态？进程都可以正常启动了吧？
如果通过修改ulimit后，进程能够成功启动，说明第一个问题已经解决了。
那么接下来需要检查一下你配置的端口号。使用以配置的端口号去连接，例如 ：
./bin/nebula -port 6999 -u user -p password

1 个赞
6

./nebula.service status all之后还是第一个nebula-metad无法启动，但是另外两个可以启动，然后nebula-graphd也会自动关闭 最后只有nebula-storaged在启动

7

能否手动尝试启动一下？然后把最新的log贴出来？
./bin/nebula-graphd --flagfile /home/…/etc/nebula-graphd.conf
./bin/nebula-metad --flagfile /home/…/etc/nebula-metad.conf

8

也可以检查一下所有配置文件的端口号，这类问题通常是由于端口号配置错误导致。

参考 ： https://github.com/vesoft-inc/nebula/blob/master/docs/manual-CN/3.build-develop-and-administration/3.deploy-and-administrations/deployment/configuration-description.md

9

配置的端口文件安装的时候没有修改过
日志如下：

nebula-storaged.ERROR
E0416 03:24:55.130048 4884 MetaClient.cpp:59] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E0416 03:25:00.134207 4926 MetaClient.cpp:457] Send request to [127.0.0.1:45500], exceed retry limit

nebula-metad.WARNING
Running on machine: redhat6
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0416 03:18:15.929632 4868 AsyncServerSocket.cpp:762] failed to set SO_REUSEPORT on async server socket Protocol not available
E0416 03:18:15.930140 4845 ThriftServer.cpp:440] Got an exception while setting up the server: 92failed to bind to async server socket: [::]:0: Protocol not available
E0416 03:18:15.930188 4845 RaftexService.cpp:90] Setup the Raftex Service failed, error: 92failed to bind to async server socket: [::]:0: Protocol not available
E0416 03:18:15.935801 4798 NebulaStore.cpp:47] Start the raft service failed
E0416 03:18:15.935808 4798 MetaDaemon.cpp:88] Nebula store init failed
E0416 03:18:15.936002 4798 MetaDaemon.cpp:215] Init kv failed!

nebula-graphd.WARNING
E0416 03:21:34.574152 5039 MetaClient.cpp:457] Send request to [127.0.0.1:45500], exceed retry limit
E0416 03:21:34.575368 5031 MetaClient.cpp:59] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E0416 03:21:39.584635 5039 MetaClient.cpp:457] Send request to [127.0.0.1:45500], exceed retry limit
E0416 03:21:39.584702 5031 MetaClient.cpp:59] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E0416 03:21:44.587388 5039 MetaClient.cpp:457] Send request to [127.0.0.1:45500], exceed retry limit
E0416 03:21:44.587486 5031 MetaClient.cpp:59] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
E0416 03:21:49.598132 5039 MetaClient.cpp:457] Send request to [127.0.0.1:45500], exceed retry limit
E0416 03:21:49.598328 5031 MetaClient.cpp:59] Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
W0416 03:21:49.598767 5031 GraphService.cpp:36] Failed to synchronously wait for meta service ready
E0416 03:21:49.610049 5057 AsyncServerSocket.cpp:762] failed to set SO_REUSEPORT on async server socket Protocol not available
E0416 03:21:49.610462 5031 ThriftServer.cpp:440] Got an exception while setting up the server: 92failed to bind to async server socket: 0.0.0.0:0: Protocol not available
E0416 03:21:49.610579 5031 GraphDaemon.cpp:155] Exception thrown while starting the RPC server: 92failed to bind to async server socket: 0.0.0.0:0: Protocol not available

10

你看看graph/meta 的配置文件，看看对应的端口号，是否被占用了？

1 个赞
11

麻烦您能贴下操作系统的内核版本信息吗？

12

端口号查过了没有被占用
问题最后通过修改系统版本解决了
以前的是centos6.5，我换到了centos7.5就没有什么问题了

13

麻烦您确认下， 出现问题的centos6.5， 内核版本是小于3.9 ？
SO_REUSEPORT 在linux 3.9 及以上才支持的

14

我看了下我的版本是2.6.32的
应该是版本比较低的原因

2 个赞
15

我也遇到这个问题了，目前dockerswarm上三台机器的metad和graphd都起来了，但是storaged都起不来，麻烦帮忙看下可能是什么导致的呢？

# ulimit -a | grep "open files"
open files                      (-n) 327680

image
image1565×216 16.6 KB

16

麻烦把配置文件贴一下，另外storaged 错误日志也贴一下看看

17

感谢回复，之前的环境已经干掉了，重新部署了一个没再出现了……我先使用一段时间看看会不会再出现吧

1 个赞