Universal re-try limits on celery tasks

[jchate6]

If a task crashes a pod, It will keep doing so for quite some until we stop generating pods. We should impose some reasonable retry limit with a descriptive log message for celery tasks.

https://lcoglobal.redmineup.com/issues/2707