appengine-python-standard icon indicating copy to clipboard operation
appengine-python-standard copied to clipboard

Extend / Fix the "NDB_PY2_UNPICKLE_COMPAT" flag

Open kaansoral opened this issue 1 year ago • 3 comments

'latin1' encoding is the fix to most common fix of the DecodeError's, throughout the docs it's the adopted fix as well, which can be seen here: https://cloud.google.com/appengine/docs/standard/python3/services/access

After this fix, I believe the "NDB_PY2_UNPICKLE_COMPAT" flag should be added to the official docs as well

Fixes #111

kaansoral avatar Jan 30 '24 09:01 kaansoral

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

google-cla[bot] avatar Jan 30 '24 09:01 google-cla[bot]

I filed issue #103 asking basically "what is this thing" and got no responses so good luck ever getting this merged :/

If you don't want to use a modified appengine lib, you could try something like this (YMMV, no guarantees)

try:
    _ = self.data
except UnicodeDecodeError:  # we know there's data, probably stored in Python 2
    try:
        prop = self._properties.get('data')
        self.data = pickle.loads(prop._get_base_value(self).b_val, encoding='latin1')
        # we 'fixed' it, we can use self.data now
    except Exception:
        # couldn't fix it, could raise here or just log something scary and give up:
        self.data = None

pnico avatar Feb 01 '24 18:02 pnico

Thank you, this was the response to my second issue tracker issue: https://issuetracker.google.com/issues/322838721

I do share the same sentiment with you, I doubt this will be fixed for others too, I suspect we all push the wagon ourselves at this point those who are left

Have no idea what 'bytes' encoding is for, but I kept it in place so that whatever edge case it fixed, is still fixed

kaansoral avatar Feb 04 '24 13:02 kaansoral