Support for other primary keys than 'pk' via a configuration

[dalberto]

Custom primary key support has been worked on previously.

However, I believe I've encountered a use-case that has not been yet addressed: a primary key field that is not named pk and cannot be derived from the model definition via the Django _meta API.

Concretely, I've encountered this use-case on an existing project that uses multi-table inheritance extensively. I'm trying to integrate drf-writable-nested to simplify existing serializers. It has proven to be a perfect fit except for models that utilize multi-table inheritance.

Using the Django documentation's models as an example:

class Place(models.Model):
    name = models.CharField(max_length=50)
    address = models.CharField(max_length=80)

class Restaurant(Place):
    serves_hot_dogs = models.BooleanField(default=False)
    serves_pizza = models.BooleanField(default=False)

With the following additional example model to illustrate the problem:

class UserFavorite(models.Model):
    user = models.ForeignKey( settings.AUTH_USER_MODEL)
    restaurant = models.ForeignKey(Restaurant)

In my API, I'd like to create a UserFavorite resource as follows:

{
   "user": 1068,
   "restaurants": [
        {
            "id": 34,
            "name": "Joe's Pizza",
            "address": "150 E 14th St",
            "serves_hot_dogs": false,
            "serves_pizza": true
        }
    ]
}

Assuming the correct serializers are configured, this is currently not possible with drf-writable-nested because the primary key field is not called pk and the attribute name for the primary key derived from the _meta API for the Restaurant model is place_ptr (which is automatically created by Django and as far as I know cannot be overridden). The automatically derived key (place_ptr) is both ugly and is difficult to build a convention around because each serializer that corresponds to multi-table inheritance models would need different key names.

I propose one (or both) of the following solutions:

• Adding a setting that allows the user to globally define their convention for primary keys (e.g. pk, id, etc). django-rest-framework-jwt provides a good example of how settings could be added.
• Adding an instance variable to BaseNestedModelSerializer that allows users to override the primary key field name on a per-serializer basis.

I look forward to adding a PR to address this use-case but would appreciate feedback before proceeding. Beyond projects that use multi-table inheritance, I see such a feature providing value to existing projects that want to use drf-writable-nested and have existing conventions around API payloads (i.e. where they have a single consistent name for primary keys that is not pk).

[ruscoder]

@dalberto Thank you for your detail explanation of the issue.

Currently drf-writable-nested doesn't support MTI. I'll think about the possible solution or workaround in few days

[dalberto]

@ruscoder Thanks!

Another solution I thought about is recursively using the _meta API to find the parent model's PK. This approach would be consistent with the current approach, albeit significantly more complex.

Anyway, let me know if there's anything I can do to help.

[beautifulDrifter]

@dalberto @ruscoder, I want to make sure I am in line with this issue, kick over a quick mixin I through together and then hopefully spark some conversation around this that may give some inspiration. I think the primary benefit of what you are asking for would be to deserialize external service responses (where you cannot control the json that is passed) and then either create or update the record based on if said records exist or not. I built a quick mixin that was working on flat json and then exploded in my face once I got into the nesting. I would also like to note I am pretty new to DRF so I may be missing something here, but from what I have gathered from the documentation, the following is where my code falls apart.

Scenerio: I am passed json with and identifier that I store in external_id. I need to:

1. Determine if the record exists If true then:
2. Populate the pk in the data to reflect the existing record
3. Add an instance for the record

Considerations I am working on figuring out.

1. You cannot only tell the serializer which field you want to use as an identifier, you also need to tell it what the data source is, since you could in theory pull from multiple data sources, and their external_id's could overlap. Ran in to this one.
2. I am actually using the id below since I get the error "UnboundLocalError: local variable 'pk' referenced before assignment" constantly when trying to update the pk. This is a big fail once you are pulling your external_id from a field called id in your json since you overwrite it.
3. I believe this would work if the inner serializer does not have many = True, but once it becomes a list serializer the whole thing falls apart. I believe I would also need to override the _many_init, but still doing further research there.

Further scenerios that I am planning to test:

1. Parent object exists, but the children do not exist,
2. Parent object exists, but only a subset of the children exist
3. Using a custom field type be used to evaluate the field, populate pk and add instance
4. Using a queryset to update the children based on the parent if exists.

Below, I stripped out what was the working code from the mess that I am building right now which will populate the top level.

class items(models.Model):
    #id = generated
    name = models.CharField(max_length=255, help_text="")
    external_id =  models.IntegerField(help_text="",blank=True,null=True)

class AddPkInstanceMixin(serializers.ModelSerializer):
    def __init__(self, **kwargs):
        object_identifier = kwargs.pop('object_identifier', None)
        object_identifier_source = kwargs.pop('object_identifier_source', None)
        super(AddPkInstanceMixin, self ).__init__(**kwargs)

    def _add_pk(self,data):
        model = self.Meta.model
        id_value = data[self.object_identifier]
        if model is not None and id_value is not None:
            pk_id = model.objects.values('id').get(**{self.object_identifier_source : id_value}) 
            data['id']=pk_id['id']
        return data

    def to_internal_value(self,data):
        model = self.Meta.model
        if not isinstance(data, list):
            data = self._add_pk(data)
            self.instance = model.objects.get(pk = data['pk'])
        return super().to_internal_value(data)

class itemsDeserializer(AddPkInstanceMixin, WritableNestedModelSerializer):
    object_identifier = 'id'
    object_identifier_source = 'external_id'
    
    name = serializer.CharField()
    items_id = s.IntegerField(source='external_id')
    class Meta:
        model = category
        fields = (
                'pk',
                'name ',
                'items_id',
        )

I want to be able to make this work, and I am going to put some more time into it for now, but at some point I will just need to come up with another solution which will probably entail evaluating the json and then routing to drf-writable-nested for create, and to a custom serializer for update. Look forward to your responses.

[dalberto]

@beautifulDrifter thank you for following up. I think your mixin does the trick, although my use-case is quite a bit simpler so I'm not sure if its the optimal solution, since it requires some configuration.

I'm actually not dealing with 3rd party data; my API simply uses id as the standard primary key field, whereas drf-writable-nested currently assumes pk represents the primary key in the request body. For now, I've simply decided to use a forked version of drf-writable-nested where all attribute access for pk is replaced with id.

[xanderim]

@dalberto Thank you for your detail explanation of the issue.

Currently drf-writable-nested doesn't support MTI. I'll think about the possible solution or workaround in few days

I think this should be stated in the README