dify
dify copied to clipboard
Fix: Add a INFO-level log when fallback to gpt2tokenizer
Summary
To improve user noticefication when fallback to gpt2tokenizer, I added INFO logs to the following script. dify/api/core/model_runtime/model_providers/__base/tokenizers/gpt2_tokenzier.py Fixes #12486
[!Tip] Close issue syntax:
Fixes #<issue number>orResolves #<issue number>, see documentation for more details.
Screenshots
| Before | After |
|---|---|
| ... | ... |
Checklist
[!IMPORTANT]
Please review the checklist below before submitting your pull request.
- [ ] This change requires a documentation update, included: Dify Document
- [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
- [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
- [x] I've updated the documentation accordingly.
- [x] I ran
dev/reformat(backend) andcd web && npx lint-staged(frontend) to appease the lint gods