页面树结构

2017-11-09 ApacheCN 开源组织,第二期邀请成员活动,一起走的更远 : http://www.apachecn.org/member/209.html


MachineLearning 优酷地址 : http://i.youku.com/apachecn

转至元数据结尾
转至元数据起始

spark-submit 报错 java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;

启动

bin/spark-submit \
--jars /www/spark-1.6.1-bin-hadoop2.6/lib/spark-streaming-kafka-assembly_2.11-1.6.1.jar \
examples/src/main/python/streaming/kafka_wordcount.py localhost:2181 topic_my

错误
Traceback (most recent call last):
  File "~/spark/examples/src/main/python/streaming/kafka_wordcount.py", line 48, in <module>
    kvs = KafkaUtils.createStream(ssc, zkQuorum, "spark-streaming-consumer", {topic: 1})
  File "/www/spark-1.6.1-bin-hadoop2.6/python/lib/pyspark.zip/pyspark/streaming/kafka.py", line 80, in createStream
py4j.protocol.Py4JJavaError: An error occurred while calling o25.createStream.
: java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;
	at org.apache.spark.streaming.kafka.KafkaUtils$.createStream(KafkaUtils.scala:170)
	at org.apache.spark.streaming.kafka.KafkaUtilsPythonHelper.createStream(KafkaUtils.scala:632)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
	at py4j.Gateway.invoke(Gateway.java:259)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.GatewayConnection.run(GatewayConnection.java:209)
	at java.lang.Thread.run(Thread.java:745)

解决方案

## 这应该是Scala的版本问题,默认spark1.6.1用的是Scala2.10.5,所以下载对于的jar,也就OK叻

bin/spark-submit \
--jars /www/spark-1.6.1-bin-hadoop2.6/lib/spark-streaming-kafka-assembly_2.10-1.6.1.jar \
examples/src/main/python/streaming/kafka_wordcount.py localhost:2181 topic_my

 

该地址案例可以参考: http://blog.csdn.net/zhong_han_jun/article/details/50721736

  • 无标签