opendal icon indicating copy to clipboard operation
opendal copied to clipboard

Handling Large Offsets for read on 32-bit Systems May Exceed usize Capacity

Open shbhmrzd opened this issue 1 year ago • 4 comments

A potential issue in the system where i64 is used for file offsets. On 32-bit architectures, usize is limited to 2^31-1, which means that offsets near the upper limit of i64 could exceed usize capacities. This can lead to runtime errors or panics if not properly handled.

shbhmrzd avatar Apr 22 '24 09:04 shbhmrzd

@Xuanwo Just changing u64 in api.rs for all offset to usize followed by changes in all implementations should be fine, right? Can I take it up? Thank you!

shbhmrzd avatar Apr 22 '24 09:04 shbhmrzd

Just changing u64 in api.rs for all offset to usize followed by changes in all implementations should be fine, right?

I don't have plan to change it yet. Please note that u64 is fine on 32 bit systems is ok.

Xuanwo avatar Apr 22 '24 11:04 Xuanwo

Just changing u64 in api.rs for all offset to usize followed by changes in all implementations should be fine, right?

I don't have plan to change it yet. Please note that u64 is fine on 32 bit systems is ok.

I think if the underlying client uses usize then it could be an issue if the offset provided is pretty large. If the client itself has u64 then it should not be an issue. Please let me know if this understanding is incorrect.

shbhmrzd avatar Apr 22 '24 11:04 shbhmrzd

I think if the underlying client uses usize then it could be an issue if the offset provided is pretty large.

Your understanding is correct. However, the proper solution is to have the underlying client accept u64 instead. Changing u64 to usize is incorrect.

That said, there's no immediate need to address this. We can revisit the issue if we encounter it with users on 32-bit systems.

Xuanwo avatar Apr 22 '24 11:04 Xuanwo