nebula-flink-connector-3.5.0 的 Flink 作业,提交到 Yarn 之后无法运行,报类名冲突。

环境

  • nebula 版本:3.5.0

  • 部署方式:单机

  • 安装方式:Docker / RPM

  • 是否上生产环境:Y

  • 硬件信息

    • 磁盘( 推荐使用 SSD)
    • CPU、内存信息
  • 问题的具体描述

    • 现象:nebula-flink-connector-3.5.0 的 Flink 作业,提交到 Yarn 之后无法运行,报错信息如下:
    Caused by: java.lang.LinkageError: loader constraint violation: loader (instance of sun/misc/Launcher$AppClassLoader) previously initiated loading for a different type with name "com/vesoft/nebula/client/graph/net/Session"
    	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_211]
    	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_211]
    	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_211]
    	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_211]
    	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_211]
    	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_211]
    	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_211]
    	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_211]
    	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_211]
    	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_211]
    	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_211]
    	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_211]
    	at java.lang.Class.getDeclaredFields0(Native Method) ~[?:1.8.0_211]
    	at java.lang.Class.privateGetDeclaredFields(Class.java:2583) ~[?:1.8.0_211]
    	at java.lang.Class.getDeclaredField(Class.java:2068) ~[?:1.8.0_211]
    	at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1857) ~[?:1.8.0_211]
    	at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:79) ~[?:1.8.0_211]
    	at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:506) ~[?:1.8.0_211]
    	at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:494) ~[?:1.8.0_211]
    	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_211]
    	at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:494) ~[?:1.8.0_211]
    	at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:391) ~[?:1.8.0_211]
    	at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:490) ~[?:1.8.0_211]
    	at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:391) ~[?:1.8.0_211]
    	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1134) ~[?:1.8.0_211]
    	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) ~[?:1.8.0_211]
    	at org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:632) ~[flink-core-1.16.2.jar:1.16.2]
    	at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:143) ~[flink-core-1.16.2.jar:1.16.2]
    	at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:132) ~[flink-core-1.16.2.jar:1.16.2]
    	at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:69) ~[flink-core-1.16.2.jar:1.16.2]
    	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean(StreamExecutionEnvironment.java:2308) ~[flink-streaming-java-1.16.2.jar:1.16.2]
    	at org.apache.flink.streaming.api.datastream.DataStream.clean(DataStream.java:202) ~[flink-streaming-java-1.16.2.jar:1.16.2]
    	at org.apache.flink.streaming.api.datastream.DataStream.addSink(DataStream.java:1242) ~[flink-streaming-java-1.16.2.jar:1.16.2]
    	at com.XXX.jobs.portraits.XXXJober.initDckSink(XXXJober.java:82) ~[xxx-23.1114.1746.jar:?]
    	at com.XXX.jobs.portraits.XXXJober.arrange(XXXJober.java:42) ~[xxx-23.1114.1746.jar:?]
    	at com.XXX.common.launcher.IBdJober$Builder.lambda$execute$8(IBdJober.java:143) ~[xxx-23.1114.1746.jar:?]
    	at java.util.HashMap$EntrySet.forEach(HashMap.java:1044) ~[?:1.8.0_211]
    	at com.XXX.common.launcher.IBdJober$Builder.execute(IBdJober.java:137) ~[xxx-23.1114.1746.jar:?]
    	at com.XXX.BigDataMain.main(BigDataMain.java:31) ~[xxx-23.1114.1746.jar:?]
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_211]
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_211]
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_211]
    	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_211]
    	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-clients-1.16.2.jar:1.16.2]
    	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-clients-1.16.2.jar:1.16.2]
    	at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:98) ~[flink-clients-1.16.2.jar:1.16.2]
    	at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:301) ~[flink-clients-1.16.2.jar:1.16.2]
    	... 12 more
    
    • 初步发现的原因
      官方的 com.vesoft.nebula.client.graph.net.Session 这个 Java 类,在 com.vesoft:nebula-flink-connector:3.5.0.jar 及其依赖的 com.vesoft:client:3.5.0.jar 中均有定义。

:thinking: 同 GitHub issue 上的问题是同一个问题,为了避免回复的同学两遍作答浪费时间,本帖的回复以上面的 issue 回复为准。(我们各个渠道都是有人在看 issue 的,所以一个问题提一个地方就好啦)。

先是在 GitHub 上提的 issues,然后官方人员建议我来论坛提。
后续有其它问题,我都只在官方论坛这里提吧

嗯嗯,都可以的。

此话题已在最后回复的 30 天后被自动关闭。不再允许新回复。