Unable to insert vector data libsql

[dariye]

Bug report

I'm unable to persist a vector column with embeddings. I followed the turso guide here here for defining the types And the Drizzle documentation. I've also taken inspiration from this blog post for the setup with expo.

I've open an issue here https://github.com/drizzle-team/drizzle-orm/issues/3899

Expected behaviour

I can insert a vector into my sqlite database

Libs and versions

"@libsql/client": "^0.14.0" "expo": "^52.0.23" "expo-sqlite": "~15.0.5" "drizzle-orm": "^0.38.3" @op-engineering/op-sqlite": "^11.2.12

Steps

0. Database configuration + Drizzle config

// db/index.ts

import { drizzle, OPSQLiteDatabase } from "drizzle-orm/op-sqlite";
import {
  open,
  DB
} from "@op-engineering/op-sqlite";
import { logError } from "~/lib/utils";

let db: OPSQLiteDatabase
let sqlite: DB

try { 
  sqlite = open({
    name: "meadowmind.local.db",
  });
  
  db = drizzle(sqlite);

} catch (error) {
  const updatedError = logError('Error initializing database client', error);
  throw updatedError;
}


export { db }

// drizzle.config.ts

import type { Config } from "drizzle-kit";
 
export default {
  schema: "./db/schema.ts",
  out: "./db/generated",
  dialect: "sqlite",
  driver: 'expo',
  casing: "snake_case"
} satisfies Config;

1. Define custom vector type

const embedding = customType<{
  data: number[];
  config: { dimensions: number };
  configRequired: true;
  driverData: Buffer;
}>({
  dataType(config) {
    return `F32_BLOB(${config.dimensions})`;
  },
  fromDriver(value: Buffer) {
    return Array.from(new Float32Array(value.buffer));
  },
  toDriver(value: number[]) {
    return sql`vector32(${JSON.stringify(value)})`;
  },
});

2. Define table with vector column

export const content = sqliteTable('content', {
  id: integer('id').primaryKey({ autoIncrement: true }),
title: text('title').notNull(),
  data: text('data', { mode: 'json'}).notNull().$type<{value: string, type: string, extension: string}>(),
  embedding: embedding('embedding', { dimensions: 512 })
});

3. Add an index in my generated/migrations .sql file

-- Custom SQL migration file, put your code below! --
CREATE INDEX IF NOT EXISTS embedding_index
ON content(embedding)
USING vector_cosine(512)

4. Run migration to add index

npm run db:migrate

5. Loading migrations in my _layout.tsx

import { useMigrations } from 'drizzle-orm/op-sqlite/migrator';

const { success, error } = useMigrations(db, migrations);

6. Insert vector data

import {content } from "../schema";

 await tx.insert(content).values([{
                title,
                data,
                embedding: sql`vector32(${JSON.stringify(embedding)})`
            }])

Observation

In the insert operation in 6) title and data are stored just alright; however, the embedding field is an empty array.