django-celery icon indicating copy to clipboard operation
django-celery copied to clipboard

setup_loader() should not add unicode strings to os.environ

Open hheimbuerger opened this issue 11 years ago • 3 comments

In djcelery/init.py:20, the function setup_loader() sets the CELERY_LOADER environment variable. However, the file has from __future__ import unicode_literals, so both key and values are actually set to unicode strings, while all other entries of os.environ normally are byte strings.

I can't quite interpret the official documentation as to whether unicode strings are allowed in the os.environ mapping object, but various other projects struggle with this and raise a TypeError: environment can only contain strings exception.

For example IPython, when starting its kernel, and https://github.com/mitsuhiko/werkzeug/issues/467, often used via django_extensions, both crash hard when djcelery is active.

This could be fixed by either removing the future import from this file, or by explicitly prefixing these literals as byte strings with b''. (Although I assume that syntax was only introduced in Python 2.7 and would break compatibility with <=2.6?)

hheimbuerger avatar Sep 24 '14 13:09 hheimbuerger

Alas, in Python 3:

>>> os.environ[b'FOO']
Traceback (most recent call last):
  File "", line 1, in 
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/os.py", line 628, in __getitem__
    value = self._data[self.encodekey(key)]
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/os.py", line 704, in encode
    raise TypeError("str expected, not %s" % type(value).__name__)
TypeError: str expected, not bytes

ask avatar Sep 24 '14 16:09 ask

Hmpf, that kinda had to happen. How about just dropping the unicode_literals future import on this module? Then it should default to a byte string on Python 2.x and a unicode string on Python 3.x (and that's under the assumption that in Python 3.x, os.environ is actually expected to contain unicode strings), no?

hheimbuerger avatar Sep 24 '14 18:09 hheimbuerger

This change needs to be reversed. With the change reversed, the tests work under Python2 and Python3.

brianmay avatar Oct 26 '15 00:10 brianmay