`azure` is required as a dependency when using `S3StorageClient`

[foragerr]

Describe the bug Azure dependency is required to use S3 as storage_client

To Reproduce from chainlit.data.storage_clients import S3StorageClient

Expected behavior I'm trying to use S3 as a storage client, I should be able to do without adding azure as a dependency.

Instead I see:

  File "...app.py", line 9, in <module>
    from chainlit.data.storage_clients import S3StorageClient
  File ".../site-packages/chainlit/data/storage_clients.py", line 4, in <module>
    from azure.storage.filedatalake import DataLakeServiceClient, FileSystemClient, DataLakeFileClient, ContentSettings
ModuleNotFoundError: No module named 'azure'

[jc-magnan]

+1 on that request.

Both storage clients are in the same file under backend/chainlit/data. Thus, using one force to load packages related to the other for no reason beyond satisfying python imports, which is not really desired (especially if you run chainlit in a container).

Fixing that issue shouldn't be much trouble: Addition of a subfolder storage to backend/chainlit/data and two separate files aws.py for s3 storage and azure .py for azure storage. Python import of storage clients would now be chainlit.data.storage.aws(or azure) and the separation would prevent the need to import azure packages for s3 storage solution and vice-versa.

[foragerr]

The real question is, how do you do it in a way that does not break backwards compatibility. 🤔

[jc-magnan]

Fair point ^^

What about a try except:

try:
    import azure
    class AzureStorage {}
except ImportError:
    log error(or warning) azure not there


try:
    import boto3
    class S3Storage {}
except ImportError:
    log error(or warning) boto3 not there

Import of AzureStorage or S3Storage should then only fail if user wants to use it and forgot to install related lib.

[antoniordz96]

For anyone who hits this and has never touched azure before you have to add the following deps

'azure-storage-blob==12.20.0',
'azure-storage-file-datalake==12.15.0', 

+1 to separating these out

[pmercier]

+1 Creating one module per storage then importing them in storage_clients to provide backwards compatibility ?

[github-actions[bot]]

This issue is stale because it has been open for 14 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 7 days since being marked as stale.