py/objarray: Raise error on out-of-bound memoryview slice start.
32 bit platforms only support a slice offset start of 24 bit max due to the limited size of https://github.com/micropython/micropython/blob/9dfabcd6d3d080aced888e8474e921f11dc979bb/py/objarray.h#L47
As it stands, if a memoryview object is sliced with a start index of 16MB or more then it's aliased back to the start of the array providing incorrect results.
I initially tried to increase the size of size_t free in the link above when used on a 32bit platform, however this breaks a lot more. Not sure why, presuably the total size of that structure is hardcoded / assumed elsewhere perhaps?
For now, this PR just raises an exception if the start index is too high to prevent accidental use of the wrong data.
Codecov Report
Merging #9028 (5c4153e) into master (d6bc34a) will increase coverage by
0.00%. The diff coverage is100.00%.
@@ Coverage Diff @@
## master #9028 +/- ##
=======================================
Coverage 98.50% 98.50%
=======================================
Files 155 155
Lines 20537 20539 +2
=======================================
+ Hits 20229 20231 +2
Misses 308 308
| Impacted Files | Coverage Δ | |
|---|---|---|
| py/objarray.h | 100.00% <ø> (ø) |
|
| py/objarray.c | 100.00% <100.00%> (ø) |
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
It's rather rare that 32bit platforms will need to essentially memory map a >16MB file. For my own use case I've got a workaround using uctypes that's also quite performant. As such I think a section about this on the https://docs.micropython.org/en/latest/genrst/modules.html#array page is probably enough, possibly with my workaround snippet as example.
Instead of:
memoryview(buff)[start:end]
you can use
uctypes.bytearray_at(uctypes.address_of(buff)+start, end-start)
Code size report:
bare-arm: +0 +0.000%
minimal x86: +0 +0.000%
I've simplified the error message and updated the test so that it works on 32-bit and 64-bit archs.
Thanks for the fix, this was definitely a bad gotcha.