Allow to pass boto3 client to S3Client

[abourramouss]

Currently the S3Client needs the credentials to be used, but what happens if i already have a boto3 client somewhere initialized and i want to use the S3Client class?

Is it possible to allow for the S3Client to take a boto3 client that has already been created? It would be simply passing the client at the constructor

[pjbull]

I think it should be possible. It may also require that you pass the session as well (which we currently support).

Out of curiosity, what is the scenario where you already have a client? What parameters are you using in constructing the client that you can't use with the current instantiation approach?

[abourramouss]

Out of curiosity, what is the scenario where you already have a client? What parameters are you using in constructing the client that you can't use with the current instantiation approach?

I use the Lithops framework to execute serverless functions. Lithops has a storage class, and this storage class is just a wrapper around boto3, Lithops exposes a get_client from whom i could get the boto3 client.

The idea of course is that Lithops has it's own config file and from where initializes everything, and to not pass the credentials through the code, it would be interesting to initialize the S3Client with the boto3 client.

https://github.com/lithops-cloud/lithops/

[HWiese1980]

This would be good because it would allow for unit tests to inject a mocked S3 client object. Currently we use moto for mocking AWS S3, but it would be one dependency less and way easier to mock S3 if cloudpathlib allowed for injecting existing S3 clients. And it would in general fulfill the dependency injection paradigm.

Of course it is also very convenient as it is right now. So it would be even better if this way of injecting the S3 client would remain optional.

[nielsneerhoff]

@pjbull the scenario I am currently in is where I need to set the verify argument of the client that is created in S3Client.__init__. Now, before passing an instance of boto3.Session to the init, I monkey-patch the boto3.Session.client , like so:

_boto3_session = boto3.Session(
    aws_access_key_id=os.environ.get("S3_ACCESS_KEY"),
    aws_secret_access_key=os.environ.get("S3_SECRET_KEY"),
    region_name=os.environ.get("S3_REGION", "default"),
)
_boto3_session.client = functools.partial(
    _boto3_session.client, verify=False  # or some path to cert
)

S3_CLIENT = S3Client(
    # ...
    boto3_session=_boto3_session
)

But it would be much more convenient if either these client-specific parameters or an already instantiated boto3 client could be passed to the __init__ of S3Client.

Hope this gives a bit more context. Feedback or suggestions are welcome - maybe I am overlooking other ways to achieve the same.