Shahram Sabzevari

icon indicating a website link https://www.linkedin.com/in/shahramsabzevari/ icon indicating an email [email protected]

icon location I am in the IT industry for more than 20 years. I have 10+ years of Data Science and Machine Learning experience across multiple industries.

Results 5 comments of Shahram Sabzevari

Coverage of pyspark user defined function

I have the same issue when testing pySpark UDFs using pytest. But as a workaround, the UDFs are python functions and I can create tests that specifically test that function...

[Feature Request] Multi header from multi index dataframe

As a workaround I am converting the df using the function below. This is not the best possible solution or a well written code but does the job. Help me...

Table export to xlsx, applying styles

I second that. I need this on my dashboard. I have a lot of heatmaps that need the color to be exported either in excel or as an image.

Large prediction results unless using repartition(1) in databricks with lgbm model

@imatiach-msft Any update on this issue? We are facing the same issue and using the reparation(1) workaround however it is not feasible for large datasets.

Document how to convert a pandas multi-index dataframe into a DataTable

As a workaround I am converting the df using the function below. I don't recommend using empty strings because you will lose sort functionality. If you sort you end up...