swiftLLM icon indicating copy to clipboard operation
swiftLLM copied to clipboard
verified publisher icon interestingLSY

Support for the Llama3.2-1B

Open ambitiousCC opened this issue 8 months ago • 0 comments

Basic Test:

  1. offline mode: python3 examples/offline.py --model-path ./models/Llama-3.2-1B image
  2. online mode: python3 examples/online.py --model-path ./models/Llama-3.2-1B image
  3. api_server: python3 swiftllm/server/api_server.py --model-path ./models/Llama-3.2-1B/ --host 0.0.0.0 --port 8082 image

The 3B model is not supported due to its hidden layer dimension of 3072

ambitiousCC avatar May 26 '25 10:05 ambitiousCC