Adding S3 Support for Artifacts
I have enhanced the logartifact function to accommodate MLFlow instances configured with S3 storage for artifact management. This enhancement includes an added verification step to determine if the specified URI corresponds to an S3 bucket. In cases where the URI is not an S3 bucket, the function retains its original behavior of saving artifacts to the disk. Conversely, when an S3 bucket is identified, the function leverages AWS and Minio services for artifact storage in the bucket. This implementation supports both AWS and Minio, although its compatibility with AWS S3 buckets has not been verified due to the lack of an AWS S3 bucket for testing. However, extensive testing has been conducted with an MLFlow instance utilizing Minio S3 buckets, and the functionality aligns with expected outcomes. It's important to note that when using Minio or similar S3 services, the MLFLOW_S3_ENDPOINT_URL environment variable must be set to ensure proper operation.
Codecov Report
Attention: 15 lines in your changes are missing coverage. Please review.
Comparison is base (
5a4237f) 77.64% compared to head (290ac2a) 74.71%. Report is 3 commits behind head on main.
| Files | Patch % | Lines |
|---|---|---|
| src/loggers.jl | 42.30% | 15 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #34 +/- ##
==========================================
- Coverage 77.64% 74.71% -2.94%
==========================================
Files 9 9
Lines 331 348 +17
==========================================
+ Hits 257 260 +3
- Misses 74 88 +14
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
@pebeto all the docs I looked at for AWS said to use both the packages, but I can try to leave one off.
@deyandyankov yea I can make those changes here soon. I'm pretty busy right now, catching up from the holidays but should be able to get to it this weekend 🤞
You can find Mocking.jl useful to write tests for this specific implementation.