improvement(kb): optimize processes, add more robust fallbacks for large file ops

[waleedlatif1]

Summary

• optimize processes, add more robust fallbacks for large file ops
• drop refresh/manual polling in favor of react query refetchInterval when requests are in flight
• make batch insertions idempotent
• added more fallbacks for doc/docx formats to other libs
• increased trigger.dev machine size for knowledge task to handle large docs
• batch calls to mistral to avoid 1k page limit
• stronger typing
• upgraded turborepo

Type of Change

• [x] Bug fix

Testing

Tested manually

Checklist

• [x] Code follows project style guidelines
• [x] Self-reviewed my changes
• [ ] Tests added/updated and passing
• [x] No new warnings introduced
• [x] I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)