2

When I try to run a command from my scala prompt i get the following error:

(TID 0, 192.168.20.81): java.lang.IllegalArgumentException:  

Class is not registered: scala.Tuple3[]    
Class is not registered: org.apache.spark.sql.Row[]

Note: To register this class use: kryo.register(scala.Tuple3[].class);
Note: To register this class use: kryo.register(org.apache.spark.sql.Row[].class);

When I run the "recommended" command to register the class i get:

scala> kryo.register(scala.Tuple3[].class)
<console>:1: error: identifier expected but ']' found.
   kryo.register(scala.Tuple3[].class)
mithrix
  • 542
  • 1
  • 4
  • 21
  • try kyro.register(classOf[Tuple3]) – Łukasz Jan 14 '16 at 18:30
  • scala> kyro.register(classOf[Tuple3]) :51: error: not found: value kyro kyro.register(classOf[Tuple3]) – mithrix Jan 14 '16 at 18:53
  • How would I just place this in the spark-defaults.conf in the spark.kryo.classesToRegister option – mithrix Jan 14 '16 at 18:54
  • google gave me this example: http://stackoverflow.com/questions/31394140/require-kryo-serialization-in-spark-scala – Łukasz Jan 14 '16 at 18:59
  • That is for command line. I need to put it in the spark-default.conf file. – mithrix Jan 14 '16 at 19:13
  • just follow the links in there, do some effort: http://spark.apache.org/docs/latest/configuration.html#compression-and-serialization this page claims that for key "spark.kryo.classesToRegister" you should "If you use Kryo serialization, give a comma-separated list of custom class names to register with Kryo. See the tuning guide for more details." Remember that they need to be full qualified names – Łukasz Jan 14 '16 at 19:26
  • I had already looked at the links you list prior to posting here. Ive made the modifications to the spark-default.conf file already. – mithrix Jan 14 '16 at 19:44
  • so what is the problem actually? In your question you have problem with registering class in console, in link I gave you there is clear example how to do this. Then you say you need to configure .conf file, which is explained as well in another link. So what do you have and what is the problem? – Łukasz Jan 14 '16 at 20:19
  • One. I found out you can't register a class directly from the console and two that register via conf doesn't work. – mithrix Jan 15 '16 at 16:53
  • So `conf.registerKryoClasses(Array(classOf[scala.Tuple3]))` in console doesn't work? You get any error? What do you put in conf and how do you conclude that this is an error? – Łukasz Jan 15 '16 at 18:10
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/100797/discussion-between-mithrix-and-lukasz). – mithrix Jan 15 '16 at 18:13

0 Answers0