spark icon indicating copy to clipboard operation
spark copied to clipboard

[FEATURE REQUEST]: Support for unit testing using `Show()` in Dataframe

Open selvavm opened this issue 3 years ago • 0 comments

Is your feature request related to a problem? Please describe. When we write unit test and set breakpoints, one cannot debug the DataFrame in Quick Watch window because the Console.WriteLine will not work. Hence, having trouble doing Test Driven Development and quick prototyping.

Describe the solution you'd like Could you add/replace/support Console.WriteLine in Microsoft.Spark.Sql.DataFrame.Show() to System.Diagnostics.Debug.WriteLine so it works in the immediate window. Then I can write df.Show() in the code or in the Quick Watch and it will work.

Describe alternatives you've considered I have to use Collect() which is difficult to interpret and hangs in the Quick Watch

Additional context Add any other context or screenshots about the feature request here.

selvavm avatar Aug 29 '22 07:08 selvavm