encode
Highlight `select_related` and `prefetch_related` usage.
In both the generic view and serializer sections we need to make sure we're pointing out appropriate usage of select_related/prefetch_related. (To review: is there anything in there at the moment?)
Also see #1976.
I note this issue is still open. Does that imply that the documentation improvements have yet to be made? If some draft or partial documentation is available please include a link. Thanks
I'm going to throw a note down here for whoever ends up doing this ticket: prefetch_related can only be used on read-only APIs. The reasoning has to do with https://github.com/tomchristie/django-rest-framework/issues/2442.
@kevin-brown > Or at least the prefetched objects associated with the instance should not be modified. Or your code updating the relations should also update the prefetch cache. Or you can always manually reload the instance after updating the relations.
Referencing #4553 here, as it fixes the issue (though is an unsatisfactory way imho).
I had issues described in #2442 with OneToOne relation using select_related - it returned old, not-updated objects after put/patch requests.
It works fine with prefetch_related (just one more query to fetch related objects instead of joining them).
It would be worth mentioning in documentation. I didn't expect this behaviour and it was not that simple to debug.
@carltongibson Any update this a couple years later? As someone new to DRF and hitting some massive performance issues (that are almost certainly related to not doing select_related and prefetch_related correctly), some docs and examples would be great! I'd be happy to contribute but I haven't been able to solve this issue myself yet.
Some links I've accumulated on the topic that may be useful:
- https://ses4j.github.io/2015/11/23/optimizing-slow-django-rest-framework-performance/
- https://thebookofjoel.com/blog/rest-framework-dynamic-prefetch-select-related
- https://www.dabapps.com/blog/api-performance-profiling-django-rest-framework/
- https://stackoverflow.com/q/26593312/1398841
- https://stackoverflow.com/questions/31237042/whats-the-difference-between-select-related-and-prefetch-related-in-django-orm
- https://stackoverflow.com/questions/9176430/django-does-prefetch-related-follow-reverse-relationship-lookup -https://stackoverflow.com/questions/97197/what-is-the-n1-selects-problem-in-object-relational-mapping
- https://stackoverflow.com/a/39680090/1398841
Hi @johnthagen. Good set of links.
I haven't been able to solve this issue myself yet.
I'm not sure there is a solution, in the sense of anything that will do this automatically for you. Bottom line is you need to look at the queries performed by each end point and carefully optimise them.
I'm not entirely convinced this is an actionable issue. (Other than adding some links back to Django's docs maybe.) It's a good topic for blog posts.
@carltongibson
I'm not sure there is a solution, in the sense of anything that will do this automatically for you
Yeah I tend to agree that there isn't something automatic, but I think that examples / some discussion can go a long way. The issue with only using blog posts is they become stale / out-of-date. Having official docs allows people to contribute / keep the information up to date.
The caching docs seem to be set up in a way I was imagining:
- https://www.django-rest-framework.org/api-guide/caching/
It has a concrete example and some discussion.
@carltongibson
I'm not sure there is a solution, in the sense of anything that will do this automatically for you
Yeah I tend to agree that there isn't something automatic, but I think that examples / some discussion can go a long way. The issue with only using blog posts is they become stale / out-of-date. Having official docs allows people to contribute / keep the information up to date.
The caching docs seem to be set up in a way I was imagining:
* https://www.django-rest-framework.org/api-guide/caching/It has a concrete example and some discussion.
it would be great if you come up with a PR
got this https://pypi.org/project/django-auto-prefetching/ and https://pypi.org/project/django-auto-prefetch/ as well.
IMHO it's more of a Django ORM good practices with DRF. what if modifying some examples using the aforementioned possible current good practices?
Hello everyone,
We faced the same issues in some projects and I have created a simple package to use in our projects. Then I have decided to publish it in github and pypi for anyone who is interested. Btw it is still being tested and there might be bugs. You can check it out on https://github.com/thetarby/django-auto-related
Simply what it does:
What django-rest gives us is the ability to know what will be accessed from the object before it is accessed by inspecting the source attribute of the fields defined in a serializer.
Then it traces those sources back on django fields instances and translates them to select_related, prefetch_related and only methods of django. Note that it does not use django-rest fields to decide what to prefetch but django fields because what django-rest says a related_field may not be a related field for the django or vice versa.
For instance, serializers.CharField(source=related_model.title) is not a related field for djangorest but in reality related_model should be prefetched to avoid n+1 problems.
If you have any feedback I would be glad to hear from you on github.
Hello everyone,
We faced the same issues in some projects and I have created a simple package to use in our projects. Then I have decided to publish it in github and pypi for anyone who is interested. Btw it is still being tested and there might be bugs. You can check it out on https://github.com/thetarby/django-auto-related
Simply what it does:
What django-rest gives us is the ability to know what will be accessed from the object before it is accessed by inspecting the source attribute of the fields defined in a serializer.
Then it traces those sources back on django fields instances and translates them to
select_related,prefetch_relatedandonlymethods of django. Note that it does not use django-rest fields to decide what to prefetch but django fields because what django-rest says a related_field may not be a related field for the django or vice versa.For instance,
serializers.CharField(source=related_model.title)is not a related field for djangorest but in reality related_model should be prefetched to avoid n+1 problems.If you have any feedback I would be glad to hear from you on github.
great package! i checked the code and like it. will use in my projects in near future.
great package! i checked the code and like it. will use in my projects in near future.
Thank you. I am glad that you think that way.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
Bumping this issue to un-stale it, as I think this is an important topic for beginners that could be improved in the docs.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
can we close this based on https://github.com/encode/django-rest-framework/pull/7610?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.