vansangpfiev

I have created new repository for llamacpp backend https://github.com/janhq/cortex.llamacpp. Still need some CI and examples to run cortex.llamacpp standalone. Basically, cortex_cpp will pull shared library from cortex.llamacpp repository, then load...

Split into sub-issues: - https://github.com/janhq/cortex.llamacpp/issues/1 - https://github.com/janhq/cortex.llamacpp/issues/2 - https://github.com/janhq/cortex.llamacpp/issues/3 - https://github.com/janhq/cortex.llamacpp/issues/4

Fixing branch is: https://github.com/janhq/nitro/pull/508

``` [TensorRT-LLM][INFO] Engine version 0.9.0 found in the config file, assuming engine(s) built by new builder API. ``` Hi @gabrielle-ong, I think you are running old version, on Windows we...

Hi @0xSage, may I know which cortex version are you running?

Thanks for your information. I will try to reproduce and fix the issue soon.

> @vansangpfiev and @hahuyhoang411 - can I get your thoughts to add to this list from my naive understanding? > > To support Vision models on Cortex, we need the...

We are using gcc-11 on ubuntu (20.04 for cortex.cpp, cortex.llamacpp; 22.04 for cortex.tensorrt-llm) to compile our sources. - gcc-11 is mandatory for cortex.tensorrt-llm - we can use gcc-9 for cortex.cpp...

Steps to build cortex for android: - NDK: https://developer.android.com/ndk/downloads - Configure cortex and build ``` # In /engine mkdir builld && cd build cmake .. -DCMAKE_TOOLCHAIN_FILE=../vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-android -DCMAKE_SYSTEM_NAME=Android -DCMAKE_SYSTEM_VERSION=28 -DCMAKE_ANDROID_ARCH_ABI=x86_64...

Our CLI and Server communicate by TCP IP socket for now and the Server does not know about `cortex` variant, so it is expected. This will be solved after we...