Question for tutorial : serialization-exceptions-and-memory-leaks-no-ws.md

[VadymBoikov]

In your tutorial for somewhere between quiz 5 and quiz 6:

I'm playing with your examples in shell, and cant figure out why does the following work?

scala> def addOne(i: Int): Int = i + 1
addOne: (i: Int)Int

scala> sc.makeRDD(1 to 100).map(addOne).reduce(_ + _)
res24: Int = 5150

deffOne is still a method - and it should not be serialized. (btw, it was defined in the shell )

[samthebest]

@VadymBoikov Are you doing this inside spark-shell or sbt console?

These shells do weird things implicitly, they could be wrapping your method in a hidden anonymous serialisable class.

[VadymBoikov]

I'm doing it inside spark-shell

Is there a way to detect if my method was wrapped imlicitly?

[samthebest]

@VadymBoikov If you make it throw an exception it shows some weirdness which implies it is. I did the following in a regular shell:

scala> def foo(i: Int): Int = ???
foo: (i: Int)Int


scala> foo(10)
scala.NotImplementedError: an implementation is missing
  at scala.Predef$.$qmark$qmark$qmark(Predef.scala:225)
  at .foo(<console>:10)
  ... 33 elided