Spark reader 运行报找不到listTags错误

nicole · 2021 年3 月 29 日 09:40

metad服务是正常的么，你可以在console中执行show tags和show edges看下。

Leo_Chan · 2021 年3 月 29 日 10:30

show tags 、show edges都有返回，请问下还有什么方法可以排查问题呢？

nicole · 2021 年3 月 29 日 10:56

你在console访问的和配置的172.20.216.158是同一个服务不？麻烦你贴一下docker-compose ps的结果信息

Leo_Chan · 2021 年3 月 29 日 11:13

cat docker-compose.yml

version: ‘3.4’
services:
client:
image: vesoft/nebula-http-gateway-v1:latest
environment:
USER: root
ports:
- 8080
networks:
- nebula-web
web:
image: vesoft/nebula-graph-studio:latest
environment:
USER: root
UPLOAD_DIR: ${MAPPING_DOCKER_DIR}
ports:
- 7001
depends_on:
- client
volumes:
- ${UPLOAD_DIR}:${MAPPING_DOCKER_DIR}:rw
networks:
- nebula-web
importer:
image: vesoft/nebula-importer:v1
networks:
- nebula-web
ports:
- 5699
volumes:
- ${UPLOAD_DIR}:${MAPPING_DOCKER_DIR}:rw
command:
- “–port=5699”
- “–callback=http://nginx:7001/api/import/finish”
nginx:
image: nginx:alpine
volumes:
- ./nginx/nginx.conf:/etc/nginx/conf.d/nebula.conf
- ${UPLOAD_DIR}:${MAPPING_DOCKER_DIR}:rw
depends_on:
- client
- web
- importer
networks:
- nebula-web
ports:
- 7001:7001

networks:
nebula-web:
driver: bridge
ipam:
driver: default
config:
- subnet: 192.1.0.1/24
- gateway: 192.1.0.1

nicole · 2021 年3 月 29 日 11:17

你这是studio的docker-compose配置，我说的是看下nebula服务信息，在nebula-docker-compose目录下执行docker-compose ps

Leo_Chan · 2021 年3 月 29 日 11:24

docker-compose ps
Name Command State Ports

nebula-web-docker_client_1 ./nebula-http-gateway Up 0.0.0.0:32778->8080/tcp
nebula-web-docker_importer_1 nebula-importer --port=569 … Up 0.0.0.0:32777->5699/tcp
nebula-web-docker_nginx_1 /docker-entrypoint.sh ngin … Up 0.0.0.0:7001->7001/tcp, 80/tcp
nebula-web-docker_web_1 docker-entrypoint.sh npm r … Up 0.0.0.0:32779->7001/tcp

nicole · 2021 年3 月 29 日 11:36

你这是查看的console的信息，需要你贴出来的是nebula-docker-compose的服务：

Leo_Chan · 2021 年3 月 29 日 13:59

我看了一下，nebula服务没有装的docker上，只有web服务装在了docker上，我刚去nebula/etc/nebula-metad.conf里看了一下，metad的地址是172.20.216.157:45500,172.20.216.158:45500,172.20.216.159:45500。与我在nebula-spark里配置的地址是一致的，并且我在studio上show tags也是有返回数据的。从输出的info日志上看，貌似是连上了metad，但是返回的结果都是null。下面我贴上日志，还请麻烦帮我看下是哪里有问题导致的读不到meta数据。

···
WARN [main] - Your hostname, AppledeMacBook-Pro.local resolves to a loopback address: 127.0.0.1; using 192.168.2.7 instead (on interface en0)
WARN [main] - Set SPARK_LOCAL_IP if you need to bind to another address
INFO [main] - Running Spark version 2.4.4
WARN [main] - Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
INFO [main] - Submitted application: a9b84ab1-d2f4-40d5-b173-9ffd91b22bb9
INFO [main] - Changing view acls to: LeoChan
INFO [main] - Changing modify acls to: LeoChan
INFO [main] - Changing view acls groups to:
INFO [main] - Changing modify acls groups to:
INFO [main] - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(LeoChan); groups with view permissions: Set(); users with modify permissions: Set(LeoChan); groups with modify permissions: Set()
INFO [main] - Successfully started service ‘sparkDriver’ on port 63737.
INFO [main] - Registering MapOutputTracker
INFO [main] - Registering BlockManagerMaster
INFO [main] - Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
INFO [main] - BlockManagerMasterEndpoint up
INFO [main] - Created local directory at /private/var/folders/my/ynmt0wvx0xz8gw22cjy030p80000gp/T/blockmgr-9d90ee83-0d3c-43d6-9a42-cb3a038ba2ad
INFO [main] - MemoryStore started with capacity 2004.6 MB
INFO [main] - Registering OutputCommitCoordinator
INFO [main] - Logging initialized @1340ms
INFO [main] - jetty-9.3.z-SNAPSHOT, build timestamp: unknown, git hash: unknown
INFO [main] - Started @1392ms
INFO [main] - Started ServerConnector@5b58ed3c{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
INFO [main] - Successfully started service ‘SparkUI’ on port 4040.
INFO [main] - Started o.s.j.s.ServletContextHandler@4c168660{/jobs,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@5b970f7{/jobs/json,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@7fd4acee{/jobs/job,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@6175619b{/jobs/job/json,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@2f058b8a{/stages,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@756cf158{/stages/json,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@3f2ef586{/stages/stage,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@64337702{/stages/stage/json,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@2cf92cc7{/stages/pool,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@30ea8c23{/stages/pool/json,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@7b139eab{/storage,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@4e76dac{/storage/json,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@611df6e3{/storage/rdd,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@5f2f577{/storage/rdd/json,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@6273c5a4{/environment,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@5d465e4b{/environment/json,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@53e211ee{/executors,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@41a90fa8{/executors/json,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@3d8bbcdc{/executors/threadDump,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@52500920{/executors/threadDump/json,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@117e0fe5{/static,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@5ac86ba5{/,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@6c67e137{/api,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@53ab0286{/jobs/job/kill,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@63c5efee{/stages/stage/kill,null,AVAILABLE,@Spark}
INFO [main] - Bound SparkUI to 0.0.0.0, and started at http://192.168.2.7:4040
INFO [main] - Starting executor ID driver on host localhost
INFO [main] - Successfully started service ‘org.apache.spark.network.netty.NettyBlockTransferService’ on port 63751.
INFO [main] - Server created on 192.168.2.7:63751
INFO [main] - Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
INFO [main] - Registering BlockManager BlockManagerId(driver, 192.168.2.7, 63751, None)
INFO [dispatcher-event-loop-2] - Registering block manager 192.168.2.7:63751 with 2004.6 MB RAM, BlockManagerId(driver, 192.168.2.7, 63751, None)
INFO [main] - Registered BlockManager BlockManagerId(driver, 192.168.2.7, 63751, None)
INFO [main] - Initialized BlockManager: BlockManagerId(driver, 192.168.2.7, 63751, None)
INFO [main] - Started o.s.j.s.ServletContextHandler@109f5dd8{/metrics/json,null,AVAILABLE,@Spark}
INFO [main] - Setting hive.metastore.warehouse.dir (‘null’) to the value of spark.sql.warehouse.dir (‘file:/Users/LeoChan/za-workspace/za-finance-plotify/spark-warehouse’).
INFO [main] - Warehouse path is ‘file:/Users/LeoChan/za-workspace/za-finance-plotify/spark-warehouse’.
INFO [main] - Started o.s.j.s.ServletContextHandler@7180e701{/SQL,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@4e2c95ee{/SQL/json,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@5942ee04{/SQL/execution,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@5e76a2bb{/SQL/execution/json,null,AVAILABLE,@Spark}
INFO [main] - Started o.s.j.s.ServletContextHandler@6dc1484{/static/sql,null,AVAILABLE,@Spark}
INFO [main] - Registered StateStoreCoordinator endpoint
ERROR [main] - response of listTags is null.
ERROR [main] - response of getEdges is null.
ERROR [main] - response of getTag is null.
INFO [Thread-1] - Invoking stop() from shutdown hook
INFO [Thread-1] - Stopped Spark@5b58ed3c{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
INFO [Thread-1] - Stopped Spark web UI at http://192.168.2.7:4040
INFO [dispatcher-event-loop-7] - MapOutputTrackerMasterEndpoint stopped!
Exception in thread “main” java.lang.NullPointerException
at com.vesoft.nebula.tools.connector.reader.NebulaRelation$$anonfun$getSchema$1.apply(NebulaRelation.scala:61)
at com.vesoft.nebula.tools.connector.reader.NebulaRelation$$anonfun$getSchema$1.apply(NebulaRelation.scala:52)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:174)
at com.vesoft.nebula.tools.connector.reader.NebulaRelation.getSchema(NebulaRelation.scala:52)
at com.vesoft.nebula.tools.connector.reader.NebulaRelation.schema(NebulaRelation.scala:32)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:403)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:208)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
at com.vesoft.nebula.tools.connector.package$NebulaDataFrameReader.loadVerticesToDF(package.scala:95)
at com.test.Test$.main(Test.scala:29)
at com.test.Test.main(Test.scala)
INFO [Thread-1] - MemoryStore cleared
INFO [Thread-1] - BlockManager stopped
INFO [Thread-1] - BlockManagerMaster stopped
INFO [dispatcher-event-loop-4] - OutputCommitCoordinator stopped!
INFO [Thread-1] - Successfully stopped SparkContext
INFO [Thread-1] - Shutdown hook called
INFO [Thread-1] - Deleting directory /private/var/folders/my/ynmt0wvx0xz8gw22cjy030p80000gp/T/spark-dacbecf6-93aa-4bfb-bc2a-d6c33d8ad285

Process finished with exit code 1

nicole · 2021 年3 月 30 日 01:23

那你的nebula服务就不是docker部署的

你在nebula/script目录下执行 ./nebula.service status metad看下metad的服务状态，每台机器都看下，是否都是绿色状态。
studio连接的metad服务不一定是你代码中设置的158这个服务，所以可以先把157和159机器上的metad服务先停掉再用studio操作show tags看下结果。

Leo_Chan · 2021 年3 月 30 日 02:45

刚按照您的步骤试了一下，关闭了157 和 159的metad服务，然后在console里还能正常的show tags，然后我把158 metad服务关闭之后console就显示获取不到数据了。之后我三个metad服务全部重启了，并且status查看状态都是正常的。但是spark连上去之后还是报这个错。
ERROR [main] - response of listTags is null.
ERROR [main] - response of getEdges is null.
ERROR [main] - response of getTag is null.

nicole · 2021 年3 月 30 日 04:03

奇怪，metaClient拿不到metad的返回结果。在clone下来的代码的examples模块跑一下com.vesoft.nebula.examples.MetaClientExample这个类，Debug下面代码

MetaClientImpl metaClient = new MetaClientImpl(args[0], Integer.valueOf(args[1]));
metaClient.connect();
metaClient.getSpaces();

应该在获取space信息的时候就是空的。

Leo_Chan · 2021 年3 月 30 日 06:02

examples模块依赖有些问题还跑不了，我刚把代码放在spark-nebula里跑了一下，发现getSpaces()方法是能获取到我们的my_space_test项目空间的，但是进一步获取tags，返回的都是null。

nicole · 2021 年3 月 30 日 06:10

我知道了，感谢反馈，这是我们存在的一个问题，我提个Pr修复下
你可以使用maven仓库中的jar包，操作步骤：

把本地maven仓库中的1.1.0版本的java client删除；
在nebula-spark目录下只编译当前模块，不要编译client模块。

nicole · 2021 年3 月 30 日 06:55

已修复，你可以采用上面的方案，也可以重新拉下v1.0分支的代码。

Leo_Chan · 2021 年3 月 31 日 03:07

重新拉了v1.0分支，现在能从metad里读到信息了～
但是有报了另一个错UnsatisfiedLinkError的错，麻烦帮我看一下～。
mac本地环境，spark local[*]的模式

Caused by: java.lang.UnsatisfiedLinkError: /private/var/folders/my/ynmt0wvx0xz8gw22cjy030p80000gp/T/nativeutils108537239121005/libnebula_codec.so: dlopen(/private/var/folders/my/ynmt0wvx0xz8gw22cjy030p80000gp/T/nativeutils108537239121005/libnebula_codec.so, 1): no suitable image found.  Did find:
	/private/var/folders/my/ynmt0wvx0xz8gw22cjy030p80000gp/T/nativeutils108537239121005/libnebula_codec.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x03
	/private/var/folders/my/ynmt0wvx0xz8gw22cjy030p80000gp/T/nativeutils108537239121005/libnebula_codec.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x03
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817)
	at java.lang.Runtime.load0(Runtime.java:810)
	at java.lang.System.load(System.java:1086)
	at com.vesoft.nebula.utils.NativeUtils.loadLibraryFromJar(NativeUtils.java:56)
	at com.vesoft.nebula.data.RowReader.<clinit>(RowReader.java:35)
	at com.vesoft.nebula.client.storage.processor.ScanVertexProcessor.process(ScanVertexProcessor.java:47)
	at com.vesoft.nebula.tools.connector.reader.NebulaVertexIterator.hasNext(NebulaVertexIterator.scala:64)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
	at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
	at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$13$$anon$1.hasNext(WholeStageCodegenExec.scala:636)
	at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:255)
	at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:247)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:836)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:836)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

nicole · 2021 年3 月 31 日 05:22

1.0不支持在mac下运行java client、Exchange、connector等，因为用到了JNI。你可以在论坛搜一下，有关于UnsatisfiedLinkError的说明。

Leo_Chan · 2021 年3 月 31 日 06:01

那windows可以吗？还是只能在linux上运行？另外2.0现在支持在本地的mac或者windows运行吗？

nicole · 2021 年3 月 31 日 07:45

1.0 windows也不可以。
2.0支持mac和windows。
1.0 不能在mac下执行，是因为1.0存在JNI的调用，而1.0的动态链接库.so文件是不支持在mac下调用的。你可以看下这个帖子 mac环境下nebula-java client读取数据时无法调用jni接口 - #8 由 nicole

Leo_Chan · 2021 年3 月 31 日 08:36

我按照你给的链接现在走到了在mac下编译nebula_codec这步，我执行cmake … 报错，请问是什么原因呢？

CMake Warning (dev) in CMakeLists.txt:
  No project() command is present.  The top-level CMakeLists.txt file must
  contain a literal, direct call to the project() command.  Add a line of
  code such as

    project(ProjectName)

  near the top of the file, but after cmake_minimum_required().

  CMake is pretending there is a "project(Project)" command on the first
  line.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Error at CMakeLists.txt:1 (nebula_add_subdirectory):
  Unknown CMake command "nebula_add_subdirectory".


CMake Warning (dev) in CMakeLists.txt:
  No cmake_minimum_required command is present.  A line of code such as

    cmake_minimum_required(VERSION 3.19)

  should be added at the top of the file.  The version specified may be lower
  if you wish to support older CMake versions for this project.  For more
  information run "cmake --help-policy CMP0000".
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring incomplete, errors occurred!
See also "/Users/LeoChan/my-workspace/software/nebula/src/jni/CMakeFiles/CMakeOutput.log".
See also "/Users/LeoChan/my-workspace/software/nebula/src/jni/CMakeFiles/CMakeError.log".

nicole · 2021 年3 月 31 日 10:44

不知道你是基于什么代码cmake的， 1.0的客户端是不建议在mac下使用的。
你可以使用2.0的spark-connector，这个是可以再mac环境使用的。

Spark reader 运行报找不到listTags错误

docker-compose ps Name Command State Ports

docker-compose ps
Name Command State Ports