twitter
Odd error calling NoSuchMethodError on com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy
This is being run from Spark.
Actually scrolling through the code I have no idea why we are getting this, seems to have arisen since our development project included titan which has set the kyro version on the class path to 2.222
java.lang.NoSuchMethodError: com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(Lorg/objenesis/strategy/InstantiatorStrategy;)V
at com.twitter.chill.KryoBase.setInstantiatorStrategy(KryoBase.scala:86) ~[chill_2.10-0.5.1.jar:0.5.1]
at com.twitter.chill.EmptyScalaKryoInstantiator.newKryo(ScalaKryoInstantiator.scala:59) ~[chill_2.10-0.5.1.jar:0.5.1]
at org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:62) ~[spark-core_2.10-1.1.1.jar:1.1.1]
at org.apache.spark.serializer.KryoSerializerInstance.<init>(KryoSerializer.scala:148) ~[spark-core_2.10-1.1.1.jar:1.1.1]
at org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:110) ~[spark-core_2.10-1.1.1.jar:1.1.1]
Annoyingly given our current setup the titan dependencies have to be on the same class path so I'm going to be trying titan using 2.21 and also raising an issue in kryo to find out where it was moved to or why it was just removed, but I thought i would point this out for you guys to know about it.
I think what happened is that it went from returning void to returning boolean.
The diamond pains here are no joke, but since Kryo is used by many low-layer systems without any classloaders, we get into pain. Storm, Scalding, Spark all use it.
It is a signature change, but not the return value. The type of the argument to setInstantiatorStrategy changed between 2.21 to 2.22; as the package of InstantiatorStrategy moved from org.objenesis.strategy to com.esotericsoftware.shaded.org.objenesis.strategy -the objenesis library was shaded.
This is the stack trace you see if you are running code built against 2.21 (here. Chill 0.50) with Kryo 2.22 on the classpath and something tries to set the instantiator strategy.
Anybody shortcut to get this error out of Jupyter-Spark1.5.2 on HD Insights java.lang.NoSuchMethodError: com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(Lorg/objenesis/strategy/InstantiatorStrategy
That is due to a version mismatch between what chill was compiled against and what kryo you have on the classpath. Without knowing what versions of each it is hard to offer more help than that. On Sat, Jan 7, 2017 at 11:01 Miel Hostens [email protected] wrote:
Anybody shortcut to get this error out of Jupyter-Spark1.5.2 on HD Insights java.lang.NoSuchMethodError: com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(Lorg/objenesis/strategy/InstantiatorStrategy
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/twitter/chill/issues/209#issuecomment-271109914, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEJdqNxg5OKt0M5py78ml_yndD7HvY6ks5rP_0qgaJpZM4DGDKK .
The version of chill and kryo on your classpath. On Sat, Jan 7, 2017 at 11:07 Miel Hostens [email protected] wrote:
What do you need as input from me?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/twitter/chill/issues/209#issuecomment-271110286, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEJdt7Fma4L7GjzY6dQKWmAcMHYTihdks5rP_6JgaJpZM4DGDKK .
Hmm, i guess hd insights is hdp -hortonworks cluster, so i would have to dive in, any suggestions were to look for ?
talking about chill here https://issues.apache.org/jira/browse/PIG-4693
Solved sudo rm /usr/bin/livy/jars/kryo-2.22.jar replace with kryo-2.21.jar