Huarong

Save trained model

24

Hi, can you add the method of saving trained model for future prediction ?

Recent Qwen1.5-14B-Chat-GPTQ-Int4 quantize details

5

You have update the weights of `Qwen1.5-14B-Chat-GPTQ-Int4` and intermediate_size from 14436 to 14336 about 12 days ago. It seems that is the int4 version is not quantized directly from `Qwen1.5-14B-Chat`...

Fix Error: TypeError: Object of type object is not JSON serializable

2

Make json dump more robust. Support object type. Fix the error like this, which occur in using langraph. ``` File "/Users/huohuarong/.pyenv/versions/3.11.2/lib/python3.11/json/encoder.py", line 258, in iterencode return _iterencode(o, 0) │ └...

### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info - `llamafactory` version: 0.9.3.dev0 - Platform: Linux-5.4.0-125-generic-x86_64-with-glibc2.31 - Python version: 3.11.2 -...

### Motivation In /v1/completions and /v1/chat/completions endpoint, can we support the parameter `n`? So that we can sampling multiple outputs for the same input. Currently, we can only call the...