Fails to run inside Docker from Ubuntu 22.04

[tadasgedgaudas]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [ :white_check_mark: ] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• [:white_check_mark: ] I carefully followed the README.md.
• [ :white_check_mark: ] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• [ :white_check_mark: ] I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

Expecting it not to fail when running via Docker

Current Behavior

Fails when running via Docker

Please provide a detailed written description of what llama.cpp did, instead.

Command:

docker run -v ./weights:/app/weights llama:latest llama -m ./weights/ggml-alpaca-7B-q4_0.bin -p "hello, how are you?"

Output (no error :thinking:)

main: seed = 1679783577
llama_model_load: loading model from './weights/ggml-alpaca-7B-q4_0.bin' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx   = 512
llama_model_load: n_embd  = 4096
llama_model_load: n_mult  = 256
llama_model_load: n_head  = 32
llama_model_load: n_layer = 32
llama_model_load: n_rot   = 128
llama_model_load: f16     = 2
llama_model_load: n_ff    = 11008
llama_model_load: n_parts = 1
llama_model_load: type    = 1

Environment and Context

Tried 3 different machines:

1. Local Intel CPU and 64gb RAM running Ubuntu 22.04. Runs fine without Docker - Inside Docker the above error
2. DigitalOcean Droplet - AMD CPU 4 Core and 8GB Ram running Ubuntu 22.04. Runs fine without Docker - Inside Docker the above error
3. Contabo - Intel CPU 8GB Ram running Ubuntu 22.04. Runs fine without Docker - Inside Docker the above error

• Physical (or virtual) hardware you are using, e.g. for Linux: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 39 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Vendor ID: GenuineIntel Model name: Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz CPU family: 6 Model: 94 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 1 Stepping: 3 CPU max MHz: 3600,0000 CPU min MHz: 800,0000 BogoMIPS: 6399.96 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mc a cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_ tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cp l vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid ss e4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_f ault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_sh adow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adj ust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx sma p clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dt herm ida arat pln pts hwp hwp_notify hwp_act_window hwp _epp md_clear flush_l1d arch_capabilities Virtualization features: Virtualization: VT-x Caches (sum of all): L1d: 128 KiB (4 instances) L1i: 128 KiB (4 instances) L2: 1 MiB (4 instances) L3: 6 MiB (1 instance) NUMA: NUMA node(s): 1 NUMA node0 CPU(s): 0-3 Vulnerabilities: Itlb multihit: KVM: Mitigation: VMX disabled L1tf: Mitigation; PTE Inversion; VMX conditional cache flushe s, SMT disabled Mds: Mitigation; Clear CPU buffers; SMT disabled Meltdown: Mitigation; PTI Mmio stale data: Mitigation; Clear CPU buffers; SMT disabled Retbleed: Mitigation; IBRS Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Spectre v2: Mitigation; IBRS, IBPB conditional, RSB filling, PBRSB- eIBRS Not affected Srbds: Mitigation; Microcode Tsx async abort: Mitigation; TSX disabled

• Operating System, e.g. for Linux:

Linux tg 5.19.0-35-generic #36~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 17 15:17:25 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

• SDK version, e.g. for Linux:

Everything's from this repository's Docker image

Steps to Reproduce

Just simple docker run from Ubuntu to Docker Ubuntu

docker run -v ./weights:/app/weights llama:latest llama -m ./weights/ggml-alpaca-7B-q4_0.bin -p "hello, how are you?"

[DHG-Dav]

also, the --seed argument and --do_not_save_grid are ignored, they are sent with the entire chain directly to SD (outside negative_prompt that seems to work as intended)