AnimatorJoe issues

Repositories
Issues
Comments

Results 2 issues of


                                            AnimatorJoe

Changing upload process for lazy loaded DataFrames

In the original DataFrame upload implementation for Snowflake and PySpark DataFrames, the DataFrames are loaded entire into memory first before being uploaded ([PySpark](https://github.com/cleanlab/cleanlab-studio/blob/e2736b0599e3c0ee3960d841236d5537e156497f/cleanlab_studio/internal/dataset_source/pyspark_dataset_source.py#L19), [Snowflake](https://github.com/cleanlab/cleanlab-studio/blob/e2736b0599e3c0ee3960d841236d5537e156497f/cleanlab_studio/internal/dataset_source/snowpark_dataset_source.py#L19C9-L19C45)). However, this can cause problems if...

Supporting Importing of Image Datasets from Snowflake Stages

Adding support for importing image datasets stored from Snowflake stages and DBFS following [simple layout](https://help.cleanlab.ai/guide/concepts/datasets/#simple-zip) and [metadata layout](https://help.cleanlab.ai/guide/concepts/datasets/#metadata-zip).