Phu Tran

Feature flag metal: Fails to load model when n_gpu_layers > 0

8

Can't utilize GPU on Mac with ``` llama_cpp_rs = { git = "https://github.com/mdrokz/rust-llama.cpp", version = "0.3.0", features = [ "metal", ] } ``` Code ``` use llama_cpp_rs::{ options::{ModelOptions, PredictOptions}, LLama,...

Update Upload/Download actions to v4. v3 to be deprecated on 11/30/2024 https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/ test run: https://github.com/phudtran/stable-diffusion.cpp/actions/runs/9118990338

ggml-metal.m - GGML_ASSERT: ne00 % 4 == 0 when generating images of dimensions 640x640

4

Seems to be an issue with group_norm on metal, haven't tried with other backends. ``` ./bin/sd -m models/gsdf/Counterfeit-V2.5/Counterfeit-V2.5_pruned.safetensors -p "a cat" --steps 2 -H 640 -W 640 ggml_metal_init: allocating ggml_metal_init:...

Artifacts in lower portion of image when generating images of size 1024x1024

4

Tried with different models (SDXL, SD1.5, etc) Only setting that reproduced the issue was resolution. Known issue when decoding latents for image sizes larger than 512x512 Example

Different behavior when running as container vs on host

1

I'm using cloud-provider-kind for integration testing with Gateway-API (I'm aware it's not officially supported yet based on #227. So perhaps this can just be a tracking issue for the future)...