Production guideline is missing information about using memory over storage when running Dapr in HA mode with actors
Describe the issue
The documentation which provides Dapr users with production guidelines for installing Dapr fails to mention any information about the fact that there can be issues when running Dapr in HA mode with actors. Specifically, there needs to be information about setting the following configuration for the placement service when running in HA mode:
forceInMemory: true
URL of the docs
https://docs.dapr.io/operations/hosting/kubernetes/kubernetes-production#high-availability-mode
Expected content
As mentioned above, there should be a paragraph under the HA mode header that states that there are known issues when sharing a PVC for storage of the actor addresses as after a long time with a large number of pod/node restarts, the state can become an issue for the placement service. Thus we need two sections of documentation:
- For new installations, users should set
forceInMemory: truebefore installing. - For those who have already installed Dapr with HA mode, they will need to first delete the stateful set and then run a Dapr upgrade with the
forceInMemory: trueset to true.
Screenshots
N/A
Additional context
Discord thread: https://discord.com/channels/778680217417809931/1277561831988265021
This was indeed a recommendation a while ago and it wasn't documented, but this issue was addressed in 1.13 and 1.14. I did a few benchmarks and adjustments of the default values in this pr: https://github.com/dapr/dapr/pull/7837
Great! If the community is happy this is no longer a recommendation, then please feel free to close this issue based on your experience and judgement.
This has been addressed in https://v1-15.docs.dapr.io/operations/hosting/kubernetes/kubernetes-production/#recommended-placement-service-configuration.