cortex.cpp icon indicating copy to clipboard operation
cortex.cpp copied to clipboard
verified publisher icon janhq

epic: Refactor Inference logic from Server Logic

Open dan-menlo opened this issue 1 year ago • 4 comments

  • Dynammic linking of "Inference Engines

dan-menlo avatar Apr 17 '24 04:04 dan-menlo

@vansangpfiev to research and estimate feasibility

tikikun avatar Apr 17 '24 06:04 tikikun

I have created new repository for llamacpp backend https://github.com/janhq/cortex.llamacpp. Still need some CI and examples to run cortex.llamacpp standalone.
Basically, cortex_cpp will pull shared library from cortex.llamacpp repository, then load it to use.

vansangpfiev avatar Apr 25 '24 00:04 vansangpfiev

Posted here for more visibility a

Image

The details of the structure of refactoring

tikikun avatar Apr 25 '24 07:04 tikikun

Issues for migration to the new design:

  • Cameron:
  • https://github.com/janhq/cortex/issues/534
  • Sang:
  • https://github.com/janhq/cortex/issues/535
  • Alan
  • https://github.com/janhq/cortex/issues/536

tikikun avatar Apr 25 '24 08:04 tikikun

It's done alrd @vansangpfiev

louis-jan avatar Jul 05 '24 03:07 louis-jan