django-rest-framework Incorrect Serializer behaviour for BooleanField(required=False) when passed a QueryDict

Checklist

[ ] Raised initially as discussion #...
[x] This cannot be dealt with as a third party library. (We prefer new functionality to be in the form of third party libraries where possible.)
[x] I have reduced the issue to the simplest possible case.

The Documentation states the following about the required keyword argument: "Normally an error will be raised if a field is not supplied during deserialization. Set to false if this field is not required to be present during deserialization.

Setting this to False also allows the object attribute or dictionary key to be omitted from output when serializing the instance. If the key is not present it will simply not be included in the output representation. "

However this doesn't work correctly with django.http.QueryDict. When passing a QueryDict to the Serializer, the not required fields are not omitted in the output but instead are set to False.

Affected Version:

django_rest_framework <= 3.12 found when upgrading from 3.11 to latest version

A Code Example

from django.http import QueryDict
from rest_framework import serializers

data_dict = {'required': True}
data_querydict = QueryDict('required=True')


class TestSerializer(serializers.Serializer):
    required = serializers.BooleanField(required=True)
    param = serializers.BooleanField(required=False)


serializer1 = TestSerializer(data=data_dict)
serializer1.is_valid()

serializer2 = TestSerializer(data=data_querydict)
serializer2.is_valid()

print(f"Serialized Dict: {serializer1.validated_data}")
print(f"Serialized QueryDict: {serializer2.validated_data}")

if serializer1.validated_data != serializer2.validated_data:
    print("There is a Bug")

When executing this Example you will see that: serializer1.validated_data == OrderedDict([('required', True)]) while serializer2.validated_data == OrderedDict([('required', True), ('param', False)]).
This diverging behaviour is not present in other types of Field (Tested myself: CharField, IntegerField) This is potentially an API breaking change

Possible Workaround

As Described here

from rest_framework import fields, serializers


class DefaultEmptyBooleanField(fields.BooleanField):
    default_empty_html = fields.empty


class MySerializer(serializers.Serializer):
    boolean_field = DefaultEmptyBooleanField(required=False)

Expected Behavior

not required fields are omitted in the output for both Dict and QueryDict

Dec 16 '21 14:12 Florian-Projects

Oh we have a same problem with NullBooleanFields:

class TestSerializer(serializers.Serializer):
    test = serializers.NullBooleanField(required=False)

bla = TestSerializer(data=QueryDict())
bla.is_valid()
bla.validated_data

OrderedDict([('test', False)])

Quick fix for us atm TestSerializer(data=self.request.query_params.dict())

Feb 01 '22 16:02 sspross

You can also use partial option for temporary purposes.

serializer_partial = TestSerializer(data=data_querydict, partial=True)
serializer_partial.is_valid()
serializer_partial.validated_data

And the BooleanField's test cases show the reason: for HTML checkboxes.

https://github.com/encode/django-rest-framework/blob/f46c33e4e2c545e3fd90de692242c53a6fe6e7e9/tests/test_fields.py#L362-L385

Doc: https://www.django-rest-framework.org/api-guide/serializers/#partial-updates

Feb 10 '22 13:02 sudosubin

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Apr 16 '22 19:04 stale[bot]

I have this exact same issue, but with a URLField. In my scenario, the field is not present on the data, but given it has allow_null=True, allow_blank=True and also required=False, the field, even if not present on the initial JSON data, is set to None on the serialized data, after is_valid() is called.

Apr 21 '22 17:04 grazzolini

I think a few more checks should be done here: https://github.com/encode/django-rest-framework/blob/master/rest_framework/fields.py#L440-L447. I'll try to put a PR up to fix this.

Apr 21 '22 18:04 grazzolini

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Aug 13 '22 07:08 stale[bot]

This is not stale, this bug is still present on DRF.

Aug 17 '22 14:08 grazzolini

from rest_framework.serializers import Serializer
from rest_framework.serializers import BooleanField
from django.http import QueryDict
class TestSerializer(Serializer):
    bool_field_1 = BooleanField(required=False)
    bool_field_2 = BooleanField(required=False, default=False)
s = TestSerializer(data=QueryDict(), partial=True)
s.is_valid(raise_exception=True)
s.validated_data
# output
# OrderedDict()

s = TestSerializer(data={}, partial=True)
s.is_valid(raise_exception=True)
s.validated_data
# output
# OrderedDict()

bool_field_2 = BooleanField(required=False, default=False) bool_field_2 had set default value so

Expected output should be:

OrderedDict([('bool_field_2', False)])

send partial=True, It will not solve the problem completely, but will create new problems

Oct 27 '22 11:10 TonyMistark

You can also use partial option for temporary purposes.
serializer_partial = TestSerializer(data=data_querydict, partial=True)
serializer_partial.is_valid()
serializer_partial.validated_data
And the BooleanField's test cases show the reason: for HTML checkboxes.

https://github.com/encode/django-rest-framework/blob/f46c33e4e2c545e3fd90de692242c53a6fe6e7e9/tests/test_fields.py#L362-L385

Doc: https://www.django-rest-framework.org/api-guide/serializers/#partial-updates

from rest_framework.serializers import Serializer
from rest_framework.serializers import BooleanField
from django.http import QueryDict
class TestSerializer(Serializer):
    bool_field_1 = BooleanField(required=False)
    bool_field_2 = BooleanField(required=False, default=False)
s = TestSerializer(data=QueryDict(), partial=True)
s.is_valid(raise_exception=True)
s.validated_data
# output
# OrderedDict()

s = TestSerializer(data={}, partial=True)
s.is_valid(raise_exception=True)
s.validated_data
# output
# OrderedDict()

bool_field_2 = BooleanField(required=False, default=False) bool_field_2 had set default value so

Expected output should be:

OrderedDict([('bool_field_2', False)])

send partial=True, It will not solve the problem completely, but will create new problems

Oct 27 '22 11:10 TonyMistark

Oh we have a same problem with NullBooleanFields:
class TestSerializer(serializers.Serializer):
    test = serializers.NullBooleanField(required=False)

bla = TestSerializer(data=QueryDict())
bla.is_valid()
bla.validated_data
OrderedDict([('test', False)])

Quick fix for us atm TestSerializer(data=self.request.query_params.dict())

NullBooleanField resolved my problom

Both types of empty dictionary input give the same result

from rest_framework.serializers import Serializer
from rest_framework.serializers import NullBooleanField
from django.http import QueryDict

class TestSerializer(Serializer):
    null_bool = NullBooleanField(required=False)
s = TestSerializer(data={})
s.is_valid(raise_exception=True)
print("input {}: ", s.validated_data)
# output
# OrderedDict()

s = TestSerializer(data=QueryDict())
s.is_valid(raise_exception=True)
print("input QueryDict: ", s.validated_data)
# output
# OrderedDict()

input {}:  OrderedDict()
input QueryDict:  OrderedDict()

SET the default value None situation

from rest_framework.serializers import Serializer
from rest_framework.serializers import NullBooleanField
from django.http import QueryDict

class TestSerializer(Serializer):
    # SET the default value None
    null_bool = NullBooleanField(required=False, default=None)
s = TestSerializer(data={})
s.is_valid(raise_exception=True)
print("input {}: ", s.validated_data)
# output
# OrderedDict()

s = TestSerializer(data=QueryDict())
s.is_valid(raise_exception=True)
print("input QueryDict: ", s.validated_data)
# output
# OrderedDict()

input {}:  OrderedDict([('null_bool', None)])
input QueryDict:  OrderedDict([('null_bool', None)])

Oct 28 '22 08:10 TonyMistark

I would love your inputs and trying this PR https://github.com/encode/django-rest-framework/pull/8614. also please contribute some suggested test cases to the test suite of this project.

Dec 04 '22 16:12 auvipy

I want to contribute to this issue please give me permission

Jan 17 '23 22:01 tanushree05

I don’t think you need permission, you can report if the changes in #8614 fix the problems and if not submit a pull request!

Jan 17 '23 22:01 merwok

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Apr 02 '23 23:04 stale[bot]

This issue is not stale, and still needs resolved

from rest_framework.serializers import Serializer from rest_framework.views import APIView

class TestAPI(APIView):

class InputSerializer(serializers.Serializer):
    id = serializers.UUIDField()
    required_field1 = serializers.BooleanField(required=True)
    required_field2 = serializers.BooleanField(required=True)

def put(self, request):
    serializer = self.InputSerializer(data= request.data)
    print(serializer.initial_data) 
    serializer.is_valid(raise_exception= True)
    print(serializer.validated_data)

request.data is <QueryDict: {'id: ['uuid_value'], 'required_field1': ['true']}>

    serializer.validated_data prints: OrderedDict([('division_setting_id', UUID('uuid_value')), ('require_field1', True), **('required_fields2', False)])**

Dec 08 '23 02:12 trevnocap

In the web api, I want to add the first_id and second_id parameters, but these parameters are not required in the web form, although in the model from which the serializer was created, these ForeigenKey parameters are required.

I think it's wrong to close the problem just because of the activity.

Jan 28 '24 08:01 ArtemIsmagilov

you can use partial option to fix the problem serializer_partial = testserializer(data=data_querydict, partial=True) serializer_partial.is_valid() serializer_partial.validated_data

May 10 '24 04:05 JIVAN-SUBEDI