Elixir bindings

[joelpaulkoch]

This PR moves the sqlite_vec package into this repository.

Here the remaining open points:

License

I'm guessing you would want to copy the Apache and MIT license from the root repository? In any case, after updating the license, we must make sure it's included in the published package on hex.

Transfer ownership of the package on hex

I can transfer the ownership of the package on hex.pm. If you give me the go, I'll do that with this user.

CI

I've included my github workflows file, not sure what you want to do with that.

[MiraiTunga]

Can this be merged?

[joelpaulkoch]

You can use the package as git dependency in mix: https://github.com/joelpaulkoch/sqlite_vec.

I want to refactor the PR so that it doesn't depend on octo_fetch. Instead we can use elixir_make, this way we don't have a runtime dependency and it will integrate better with the ecosystem (cross compilling with burrito, probably also nerves). I expect this to work pretty good given that we can just reuse the existing Makefile, sqlite-vec.c and sqlite-vec.h.

[asg017]

@joelpaulkoch lmk if I can help with that in anyway! A no-dep Makefile option would be awesome

[joelpaulkoch]

So, here's the updated PR. A few notable points:

1. it's not no-dep, but elixir_make is a compile time only dependency, and pretty established (as you can see by the fact that it's in the elixir-lang org on github). It's for instance used by burrito to detect a native library and recompile it when doing cross compilation (that is to say, there are benefits to have elixir_make as dependency compared to manually calling make).
2. To compile the library from the Makefile, we must have Makefile, VERSION, sqlite-vec.c and sqlite-vec.h.tmpl as part of the published package. I solved that by adding symlinks and a publish script which replaces the symlinks with the actual files before publishing the package.
3. I had to update the Makefile so that the install prefix can be set from env.
4. This approach requires that users have make and a c compiler available. There is the concept of precompilers in elixir_make that check for existing binaries to download before compilation but I think it's not quite fitting the scenario here where the native library is not only made for Elixir. So maybe we can later add a step to download from github releases and make the compilation with make and c compiler optional.

[asg017]

Thank you @joelpaulkoch for your hard work here!

One ask — instead of relying on the Makefile + sqlite-vec.h.tmpl from this project, can we instead use the "amalgamation" build, that has pre-processed sqlite-vec.c and sqlite-vec.h files? Here's a sample: https://github.com/asg017/sqlite-vec/releases/download/v0.1.7-alpha.2/sqlite-vec-0.1.7-alpha.2-amalgamation.zip

Hopefully that will mean we don't need to include Makefile/VERSION in here, and maybe avoid the need for the symlinking

Also — do you have a link to docs that talk about the "precompilers" for elixir make? I ask because the sqlite-dist project is for pre-compiling/bundling SQLite extensions for various platforms, so it might be possible to include elixir_make precompiled support in there (which sqlite-vec uses in it's publishing process)

[joelpaulkoch]

Regarding amalgamation, that would mean we would include those preprocessed files in the package (tarball), right? Then have a simple Makefile to build from sqlite-vec.c and sqlite-vec.h, and no symlinks. Then, the publishing process would require to first build the amalgamation, then proceed with the Elixir package. I think that would work and avoid symlinking.

Regarding precompilers in elixir_make, it's mentioned here in the docs. If I get it right, it's just an integrated way to compile the native library of an Elixir package and have a known URL for the artifact so that you can first try to download instead of compiling.