Serge Gotsuliak
Serge Gotsuliak
As of latest Comet v1.1.2 you can get the client's IP address from REMOTE_ADDR attribute of ServerParams: $params = $request->getServerParams(); $remote_ip = $params['REMOTE_ADDR'];
Most of the popular programming language use some form of "dot import" without any hassles. Hows ftm.Printf is more clear then simple print or echo?
Will it work if I just load LoftQ adapters with CLI parameter? --adapter_name_or_path /home/loftq/adapters
@danielhanchen Do you think it's possible with LLaMA-Factory? Seems like there some hope there: https://github.com/hiyouga/LLaMA-Factory/wiki/Performance-Comparison#nvidia-a100--2
That helped me a lot ! I had JIT 2.1 by default and toolkit as is do not aware of new header for it :(
> Oh, I forgot to add that support for 2.1 bytecode should be added. Currently only 2.0 is supported. Are there plans for 2.1 support in this toolkit ?
Oh, it's not so easy, there non-standard PEFT needed https://github.com/huggingface/peft/compare/main...fxmeng:peft:main
So no plans for supporting QLoRA ?
That's interesting, I'm getting really different results with `-t 8` and `-t 10` on M1 Pro laptop / 7B model. _system_info: **n_threads = 10 / 10** | AVX = 0...
Tried with different thread count and it seems this affect not only performance but the core inference quality. Looks like choose 1, 4, 8 threads are safe on my machine.