guptaat

Thank you for the response and pointer. Environment Variables and Initialization for spark `%env PYSPARK_PYTHON=python` `%env PYSPARK_DRIVER_PYTHON_OPTS=notebook` `%env PYSPARK_DRIVER_PYTHON=jupyter` `%env JAVA_HOME=C:\Program Files\Eclipse Adoptium\jdk-11.0.14.9-hotspot` `from pyspark.sql import SparkSession` `import pyspark.sql.functions as...

@fdosani apologies. got sidetracked in between due to other projects. Hopefully will pick it back again this week and report back asap. Thank you

@fdosani I installed the spark compare refactor branch using `pip install --upgrade git+https://github.com/capitalone/datacompy@spark-compare-refactor` It did not seem to solve the problem as got the results as below  Looked at...

@fdosani Yes spark.core.py does have eqNullSafe ( i was previously looking at sparkcompare.py)  I explicitly added the import statement as u suggested  Got the following results  but...