189 8069 5689
这篇文章主要讲解了“spark yarn 提交任务异常怎么办”，文中的讲解内容简单清晰，易于学习与理解，下面请大家跟着小编的思路慢慢深入，一起来研究和学习“spark yarn 提交任务异常怎么办”吧！
创新互联2013年开创至今，是专业互联网技术服务公司，拥有项目成都网站制作、成都做网站网站策划，项目实施与项目整合能力。我们以让每一个梦想脱颖而出为使命，1280元仪征做网站,已为上家服务,为仪征各地企业和个人服务,联系电话:18980820575
Exception in thread "main" java.lang.IllegalStateException: Library directory 'C:\eclipse-workspace\SparkDemo\assembly\target\scala-2.11\jars' does not exist; make sure Spark is built. at org.apache.spark.launcher.CommandBuilderUtils.checkState(CommandBuilderUtils.java:248) at org.apache.spark.launcher.CommandBuilderUtils.findJarsDir(CommandBuilderUtils.java:342) at org.apache.spark.launcher.YarnCommandBuilderUtils$.findJarsDir(YarnCommandBuilderUtils.scala:38) at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:545) at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:865) at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:179) at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:57) at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:188) at org.apache.spark.SparkContext.(SparkContext.scala:501) at org.apache.spark.api.java.JavaSparkContext. (JavaSparkContext.scala:58) at my.test.WordCountYarn.main(WordCountYarn.java:34) [Thread-1] [INFO] [2021-04-12 18:12:44] Shutdown hook called [Thread-1] [INFO] [2021-04-12 18:12:44] Deleting directory C:\Users\zengming.zhang\AppData\Local\Temp\spark-ed6b5d87-e618-4375-9ee4-264c0d832447
SparkConf conf = new SparkConf().setAppName("WordCountYarn").setMaster("yarn-client")
.set("spark.yarn.dist.files", "C:\\eclipse-workspace\\SparkDemo\\src\\main\\resources\\yarn-site.xml")
//添加此参数解决
.set("spark.yarn.jars", "*.jars")
https://blog.csdn.net/qq_40651753/article/details/86239630
感谢各位的阅读，以上就是“spark yarn 提交任务异常怎么办”的内容了，经过本文的学习后，相信大家对spark yarn 提交任务异常怎么办这一问题有了更深刻的体会，具体使用情况还需要大家实践验证。这里是创新互联，小编将为大家推送更多相关知识点的文章，欢迎关注！