nebula-exchange_spark_2.4本地编译报错:org/bouncycastle/bcprov-jdk15on和Unsupported class file major version 59

[jiaenqiu]

想在本地编译一个exchange-3.0.0的包，spark2.4，依赖环境都满足:jdk1.8,scala2.11，但是很遗憾报错尝试未解决。

—满足依赖的环境。

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project nebula-exchange_spark_2.4: Error creating shaded jar: Problem shading JAR /Users/jiaenqiu/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.69/bcprov-jdk15on-1.69.jar entry META-INF/versions/15/org/bouncycastle/jcajce/provider/asymmetric/edec/SignatureSpi$Ed25519.class: java.lang.IllegalArgumentException: Unsupported class file major version 59 → [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :nebula-exchange_spark_2.4

[xjc]

maven啥版本？

[jiaenqiu]

apache-maven-3.8.5,jdk1.8.0_321

[jiaenqiu]

问题描述补充：

提交到sparkOnYarn上后报错，异常信息如下

Job aborted due to stage failure: 
Task 0 in stage 3.0 failed 4 times, most recent failure: 
Lost task 0.3 in stage 3.0 (TID 16, 10.203.85.73, executor 6): 
java.lang.NoSuchMethodError: com.google.common.base.Stopwatch.createStarted()
Lcom/google/common/base/Stopwatch;

解决思路：

• 1.NoSuchMethodError一般就是依赖jar冲突导致的，上述异常是guava依赖包
• 2.查看guava源码，发现14.0.1版本中确实没有com.google.common.base.Stopwatch.createStarted()，而16.0.1中有，所以把所有14.x的版本都排除掉
• 3.提交任务还是报错，那就是yarn依赖的guava依赖包版本比较低，jar冲突。由于我这边环境无法私自改动，所以只能修改本地guava包路径，利用maven-shade-plugin插件，这样编译时就可以加载自定义包名就可以忽略yarn上低版本的guava包了
• 4.另外所有依赖中的org.bouncycastle.bcprov-jdk15on也得在xml中排除掉，否则会影响shade打包，引起如博客主文描述的异常

解决方案：

<exclusions>
                <exclusion>
                    <groupId>org.bouncycastle</groupId>
                    <artifactId>bcprov-jdk15on</artifactId>
                </exclusion>
            </exclusions>

<plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.2.1</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <relocations>
                                <relocation>
                                    <!-- 源包名 -->
                                    <pattern>com.google.common</pattern>
                                    <!-- 目的包名 -->
                                    <shadedPattern>my_guava.common</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>org.bouncycastle</pattern>
                                    <shadedPattern>bcprov-jdk15on</shadedPattern>
                                </relocation>
                            </relocations>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

• 如有异议，请回复

[xjc]

看下JAVA_HOME，那个报错看上去还是和jdk有关啊。
mvn编译的命令是啥？要不按它的建议 -e 看下错误堆栈？
我是怎么编都不报错

[jiaenqiu]

➜echo $JAVA_HOME
➜/Library/Java/JavaVirtualMachines/jdk1.8.0_321.jdk/Contents/Home
➜mvn clean package -Dmaven.test.skip=true -Dgpg.skip -Dmaven.javadoc.skip=true -pl nebula-exchange_spark_2.4 -am -Pscala-2.11 -Pspark-2.4

• 不知道为啥这个版本编译就报错，可能和我使用shade插件的原因，必须排除bcprov-jdk15on才行