gemma.cpp
gemma.cpp copied to clipboard
google
lightweight, standalone C++ inference engine for Google's Gemma models.
Whether compile support for Windows will be rolled out？
Just a proposal, but I thought it was easier to discuss with a proposal as opposed to an issue. The `add_compile_options($)` line adds the `-O2` on Release builds, but the...
In some compilers that support only some C++17 features like gcc6.3.0, it may encounter a situation where std filesystem lib can't be found, like: ``` /usr/bin/ld: CMakeFiles/gemma.dir/gemma.cc.o: in function `std::filesystem::__cxx11::path::path(std::__cxx11::basic_string...
It's not possible to build `gemma.cpp` with the standard MSVC front-end as it doesn't support arrays more than `0x7ffffffff` bytes (see [Compiler Error C2148](https://learn.microsoft.com/en-us/cpp/error-messages/compiler-errors-1/compiler-error-c2148?view=msvc-170)), however this isn't a problem with...
On a $10,000 AMD Ryzen 7995WX (znver4 avx512) Gemma 7b instruct sfp is able to solve mathematical riddles.  But on a $600 Intel i9-14900K (raptorlake avx2) the same Gemma...
is this support Android aarch64? or Android Studio? or AOSP ? It seems easy! aha
Failed to read cache gating_ein_0 (error 294) │ Abort at /home/.../models/gemma.cpp/./gemma.cc:117: Failed│ to open model file - does it exist? │
Would be nice if Bazel build would also be supported by this project by providing WORKSPACE/MODLUE/BUILD files
I'd love an example of using this from Rust. Perhaps using C ABI.
