提问参考模版:
- nebula 版本:3.3.0, spark版本3.2.2,python版本 3.7.6
- 部署方式:分布式
- 是否为线上版本: N
问题描述如下,求各位大佬帮忙提供一下解决思路:
执行命令 :
spark-submit \
--master yarn \
--deploy-mode client \
--jars file:///data1/hive/account/Sky/server/nebula-spark-connector-3.3.0.jar \
/data1/hive/account/Sky/graph_oppside_spark.py
代码内容:
from pyspark.sql import SparkSession
def nebula_search():
""" nebula官网的demo"""
spark = SparkSession.builder.config("nebula-spark-connector-3.3.0.jar","/data1/hive/account/Sky/server/nebula-spark-connector-3.3.0.jar").appName("nebula-connector").getOrCreate()
# read vertex
df = spark.read.format("com.vesoft.nebula.connector.NebulaDataSource").\
option("type", "vertex").option("spaceName", "mpay_risk_graph_sit").option("label", "partner").\
option("returnCols", "name").option("metaAddress", "xxxx:9559,yyyy:9559,zzzz:9559").option("partitionNumber", 1).load()
df.show(n=2)
执行结果:
File "/data1/hive/account/Sky/server/spark-3.2.0-bin-hadoop2.7/python/lib/py4j-0.10.9.2-src.zip/py4j/protocol.py", line 328, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o49.load.
: java.lang.NoClassDefFoundError: org/apache/spark/sql/sources/v2/ReadSupport
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)