feat: Support MacOS with AVX on Intel processor

[CameronNg]

Problem OS: MacOS (on Mac Pro 2013) Processor: Intel® Xeon® Processor E5-2697 v2

Nitro is failing with the following log:

2024-04-11T02:01:19.057Z [NITRO]::CPU informations - 122024-04-11T02:01:19.058Z [NITRO]::Debug: Request to kill Nitro
2024-04-11T02:01:19.062Z [NITRO]::Debug: Nitro process is terminated
2024-04-11T02:01:19.063Z [NITRO]::Debug: Spawning Nitro subprocess...
2024-04-11T02:01:19.063Z [NITRO]::Debug: Spawn nitro at path: /Users/luciano/jan/extensions/@janhq/inference-nitro-extension/dist/bin/mac-x64/nitro, and args: 1,127.0.0.1,3928
2024-04-11T02:01:19.370Z [NITRO]::Debug: Nitro is ready
2024-04-11T02:01:19.370Z [NITRO]::Debug: Loading model with params {"ctx_len":4096,"prompt_template":"[INST] <<SYS>>\n{system_message}<</SYS>>\n{prompt}[/INST]","llama_model_path":"/Users/luciano/jan/models/llama2-chat-7b/llama-2-7b-chat.Q4_K_M.gguf","system_prompt":"[INST] <<SYS>>\n","user_prompt":"<</SYS>>\n","ai_prompt":"[/INST]","cpu_threads":12,"ngl":100}
2024-04-11T02:01:19.375Z [NITRO]::Debug: Nitro exited with code: null
2024-04-11T02:01:20.890Z [NITRO]::Error: Load model failed with error TypeError: fetch failed
2024-04-11T02:01:20.890Z [NITRO]::Error: TypeError: fetch failed

Success Criteria Nitro works properly.

Additional context

• This is the first case with Intel AVX on MacOS.
• The processor does not support AVX2.

[lucianosantilli]

I was the one with this request. I have a Mac Pro 2013 (12 cores), where I got this error and also, lying around, a much "worst" mac book pro 2012 if you want me to test a possible solution in 2 different hardwares.

[freelerobot]

This should be address after Jan is migrated to Cortex backend instead of Nitro. @vansangpfiev to double check and close this issue :)

[freelerobot]

stale