Backblaze
Android compatibility
Hi, I am looking in to integrating your SDK in to my app as another cloud service it supports. However, I am concerned about the following information.
Readme says that "If the content is less than about 2 GB, we read the content into an in-memory array". This might work on a desktop app but would create issues on a mobile phone.
Is there anyway controlling this behaviour? I would prefer providing InputStream or File to the SDK for upload. Thanks
The SDK is designed to handle InputStreams -- see the ContentSource abstraction. The Apache-HttpClient-based implementation is does streaming. We have a new OkHttp implementation that's intended for use on Android, but OkHttp doesn't support using InputStreams because of the way it follows redirects and handles retries. We're working around it for now and considering our options.
What API do you use on Android for your HTTP requests today?
thanks, ab
On Mon, Jun 3, 2019 at 7:51 PM NLL APPS [email protected] wrote:
Hi, I am looking in to integrating your SDK in to my app as another cloud service it supports. However, I am concerned about the following information.
Readme says that "If the content is less than about 2 GB, we read the content into an in-memory array". This might work on a desktop app but would create issues on a mobile phone.
Is there anyway controlling this behaviour? I would prefer providing InputStream or File to the SDK for upload. Thanks
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Backblaze/b2-sdk-java/issues/75?email_source=notifications&email_token=ABHJFCWSVJ7RIYERTOHWYQDPYVY4JA5CNFSM4HSQH6T2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GXLYYWQ, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHJFCTNB7IQKOAMZQFEAIDPYVY4JANCNFSM4HSQH6TQ .
I am using OkHttp. I am aware of Inputstream limitation of OkHttp and working around it by copying file to a temporary location in my WebDAV and WebHook modules.
I don't mind doing the same for your SDK. Would it be possible override reading to memory behaviour?
On June 3, 2019 7:56:39 PM UTC, certainmagic [email protected] wrote:
The SDK is designed to handle InputStreams -- see the ContentSource> abstraction. The Apache-HttpClient-based implementation is does> streaming. We have a new OkHttp implementation that's intended for use on> Android, but OkHttp doesn't support using InputStreams because of the way> it follows redirects and handles retries. We're working around it for now> and considering our options.>
What API do you use on Android for your HTTP requests today?>
thanks,> ab>
On Mon, Jun 3, 2019 at 7:51 PM NLL APPS [email protected] wrote:>
Hi, I am looking in to integrating your SDK in to my app as another cloud> service it supports.> However, I am concerned about the following information.>
Readme says that "If the content is less than about 2 GB, we read the> content into an in-memory array". This might work on a desktop app but> would create issues on a mobile phone.>
Is there anyway controlling this behaviour? I would prefer providing> InputStream or File to the SDK for upload.> Thanks>
—> You are receiving this because you are subscribed to this thread.> Reply to this email directly, view it on GitHub>
or mute the thread>
https://github.com/notifications/unsubscribe-auth/ABHJFCTNB7IQKOAMZQFEAIDPYVY4JANCNFSM4HSQH6TQ>
.>
-- > You are receiving this because you authored the thread.> Reply to this email directly or view it on GitHub:> https://github.com/Backblaze/b2-sdk-java/issues/75#issuecomment-498402870
Today, the SDK is set up such that clients always provide data to the SDK using InputStreams. I'm still thinking about if/how that might be able to change in the future.
Here's a separate question -- what if there was a way for the SDK to use Java's built-in HTTP support (java.net.URLConnection) instead of OkHttp? How would you feel about that?
thanks, ab
On Mon, Jun 3, 2019 at 9:56 PM NLL APPS [email protected] wrote:
I am using OkHttp. I am aware of Inputstream limitation of OkHttp and working around it by copying file to a temporary location in my WebDAV and WebHook modules.
I don't mind doing the same for your SDK. Would it be possible override reading to memory behaviour?
On June 3, 2019 7:56:39 PM UTC, certainmagic [email protected] wrote:
The SDK is designed to handle InputStreams -- see the ContentSource> abstraction. The Apache-HttpClient-based implementation is does> streaming. We have a new OkHttp implementation that's intended for use on> Android, but OkHttp doesn't support using InputStreams because of the way> it follows redirects and handles retries. We're working around it for now> and considering our options.>
What API do you use on Android for your HTTP requests today?>
thanks,> ab>
On Mon, Jun 3, 2019 at 7:51 PM NLL APPS [email protected] wrote:>
Hi, I am looking in to integrating your SDK in to my app as another cloud> service it supports.> However, I am concerned about the following information.>
Readme says that "If the content is less than about 2 GB, we read the> content into an in-memory array". This might work on a desktop app but> would create issues on a mobile phone.>
Is there anyway controlling this behaviour? I would prefer providing> InputStream or File to the SDK for upload.> Thanks>
—> You are receiving this because you are subscribed to this thread.> Reply to this email directly, view it on GitHub>
< https://github.com/Backblaze/b2-sdk-java/issues/75?email_source=notifications&email_token=ABHJFCWSVJ7RIYERTOHWYQDPYVY4JA5CNFSM4HSQH6T2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GXLYYWQ>,>
or mute the thread>
< https://github.com/notifications/unsubscribe-auth/ABHJFCTNB7IQKOAMZQFEAIDPYVY4JANCNFSM4HSQH6TQ>>
.>
-- > You are receiving this because you authored the thread.> Reply to this email directly or view it on GitHub:> https://github.com/Backblaze/b2-sdk-java/issues/75#issuecomment-498402870
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Backblaze/b2-sdk-java/issues/75?email_source=notifications&email_token=ABHJFCVC4EQUI6OLQCA5S3LPYWHQZA5CNFSM4HSQH6T2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW2Z2IA#issuecomment-498441504, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHJFCQFKOOBPRLAHNPY7IDPYWHQZANCNFSM4HSQH6TQ .
Less dependencies would be better from a developer point of view.
Some might what to use different client though. Perhaps buid it in a way that developers can swap http client at will.
I think Microsoft Graph SDK does something like that.
You might also want to have a look at openFileDescriptor. Something like https://gist.github.com/davidliu/dbebbe0a58a5035ecd541b573518c6cf
However, I am not sure if it would work
Thanks. The SDK can be used with different implementations of an interface called B2WebApiClient https://github.com/Backblaze/b2-sdk-java/blob/master/core/src/main/java/com/backblaze/b2/client/webApiClients/B2WebApiClient.java; see the "Structure https://github.com/Backblaze/b2-sdk-java#structure" section of the README for some more info. Here at Backblaze, we use the Apache HttpClient-based implementation that's in in the SDK; it's packaged in a separate jar to keep its dependency on HttpClient separate from the core of the SDK, just as the OkHttp-based implementation is in a separate jar. (By the way, you're always welcome to use the HttpClient-based implementation on Android.) The issue is that the B2WebApiClient interface is specified in terms of InputStreams and OkHttp isn't lining up well with that.
Are you open to trying the Apache HttpClient-based implementation or providing your own B2WebApiClient implementation? I've actually prototyped a UrlConnection-based implementation, but I'm not currently working on it; I'd be willing to share that with you as well if you're interested in making a PR.
ttfn, ab
On Tue, Jun 4, 2019 at 8:52 AM NLL APPS [email protected] wrote:
You might also want to have a look at openFileDescriptor. Something like https://gist.github.com/davidliu/dbebbe0a58a5035ecd541b573518c6cf
However, I am not sure if it would work
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Backblaze/b2-sdk-java/issues/75?email_source=notifications&email_token=ABHJFCSZ5ZTWIJO2J6LWBOLPYYUOHA5CNFSM4HSQH6T2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW34OGY#issuecomment-498583323, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHJFCQNC4P5GEPZI74VWY3PYYUOHANCNFSM4HSQH6TQ .
Would a configuration option to always use a temporary local File instead of memory buffering work? Or a configurable threshold that determines when to switch from memory to file?
I can only spare a limited time for this and would prefer using default implementation
Even if you don't care about okhttp issues, or use some android repackage of httpclient ala: https://github.com/smarek/httpclient-android, you're still going to have issues as a lot of the date time util and other jdk 8 usages of classes in this b2 official client require API Level 26. Unless you're only supporting Oreo and above (unlikely), this client is unusable on Android.
Mark is running an experiment internally using a web client implemented on the normal java UrlConnection classes. (see the urlCxnExperiment branch.)
I'm not enough of an expert on Android to address the date/time class versioning. Maybe @OldSkoolMark to reply to that when he's around next week.
Using https://github.com/JakeWharton/ThreeTenABP you can work around the java.time issues on older versions of android, leaving the Supplier.get issues but androidx appcompat now has a Supplier implementation. Once you've refactored to use ThreeTenABP, httpclient-android and androidx supplier you're left with a handful of java.util.Map, List, Comparator, Stream, and Base64 issues which should be relatively easy to work around.
Thanks, 2fours. Your info is spot on. We're looking into maybe supporting old Android API levels.
thanks, ab