Full multimodal search
without the data leaving the building.

What we tested—and what business problems this solves

If you ask an engineering team today to make a folder of real company files searchable—a mix of Markdown docs, multi-column PDF contracts, banker PowerPoint decks, scanned vendor invoices, screenshots, source code, and short voice notes—the standard architecture gets expensive and brittle fast. You wire up a cloud OCR parser for the PDFs and invoices, a vision captioner for the slide charts, Whisper for the audio, a metered cloud embedding API, and a separate vector database server alongside your application database. Every confidential file leaves the machine, and every format needs its own glue service before a single query runs.

We wanted to know whether two local components can replace that entire stack for small-to-midsize workloads:

1. Google DeepMind’s embeddinggemma-2:latest running locally in Ollama (740M total parameters across text/code, vision, and audio encoders; 1.33 GB on disk and 1.27 GB resident RAM in Apple Silicon MLX nvfp4), which projects strings, PNG/JPEG images, 16 kHz WAV audio, and bimodal { image, text } pairs into the same 768-float vector space.
2. A single embedded SQLite file via @libsql/client, which gives you native F32_BLOB columns, an in-process DiskANN cosine index (libsql_vector_idx), and SQLite FTS5 keyword ranking (bm25) inside one ACID file—no Docker containers, no sidecar daemon, no API keys.

We built a working reference system (The-Focus-AI/gemmaembeddings) and indexed 122 heterogeneous units—enterprise architecture records, runbooks, source code, six non-English tickets, zero-caption scanned invoices, photographs, 16 kHz audio clips, 76 real arXiv PDF pages, and 14 PowerPoint slides (with embedded charts and presenter speaker notes)—on a 64 GB Apple M4 Max. Here is what this tool is actually good for, how you get data in, how you get answers out, how fast it runs, and where its sharp edges are.

Four business use cases where this stack wins

Getting data IN: how every format enters SQLite

Ollama exposes EmbeddingGemma 2 through a single HTTP endpoint (POST http://127.0.0.1:11434/api/embed), and it only accepts four input shapes:

1. Text / Code / Markdown string — "title: ADR-014 | text: All settlement retries..."
2. Image-only object — { image: "<raw-base64-png-or-jpeg>" }
3. Audio-only object — { audio: "<raw-base64-16khz-wav>" }
4. Fused bimodal object — { image: "<raw-base64-png>", text: "title: ... | text: ..." }, which fuses visual layout and text tokens into one 768d vector in a single forward pass.

Notice what is not on that list: .pdf files, .pptx decks, .docx contracts, or data:image/png;base64,... browser URLs. Getting real business files into SQLite cleanly means converting each source into one of those four primitives first.

1. Creating the SQLite vector + keyword schema (@libsql/client)

In @libsql/client, vectors live in standard SQLite tables as F32_BLOB(N) columns alongside your metadata, indexed by libsql_vector_idx (DiskANN cosine) and paired with an FTS5 virtual table for exact tokens:

-- 1. Main table: metadata + 256d Matryoshka vector (1,024 bytes/row)
CREATE TABLE IF NOT EXISTS docs (
  id             TEXT PRIMARY KEY,
  parent_doc_id  TEXT,
  page_number    INTEGER,
  modality       TEXT NOT NULL,          -- 'document' | 'code' | 'pdf_page' | 'pptx_slide' | 'visdoc' | 'audio'
  title          TEXT NOT NULL,
  extracted_text TEXT NOT NULL DEFAULT '',
  speaker_notes  TEXT NOT NULL DEFAULT '',
  asset_path     TEXT,                   -- keep PNGs/PDFs on disk, NOT as huge base64 blobs in SQLite!
  emb_256        F32_BLOB(256) NOT NULL
);

-- 2. Native in-SQLite DiskANN cosine index
CREATE INDEX IF NOT EXISTS idx_docs_emb_256
  ON docs (libsql_vector_idx(emb_256, 'metric=cosine'));

-- 3. SQLite FTS5 keyword index (weights: title 2.0, body 1.0, speaker notes 1.4)
CREATE VIRTUAL TABLE IF NOT EXISTS docs_fts USING fts5(
  id UNINDEXED,
  title,
  extracted_text,
  speaker_notes,
  tokenize = 'unicode61'
);

Why F32_BLOB(256) instead of all 768 floats? Because EmbeddingGemma 2 is trained with Matryoshka Representation Learning (MRL): the coarsest semantic signal is packed into the leading coordinates. When we sliced every 768d vector down to its first 256 floats, re-normalized to unit length, and re-indexed in SQLite, storage dropped 3× (3,072 B → 1,024 B/row, or 97.7 MB per 100k items instead of 293 MB) while Hit@1 actually rose from 93.3% to 100.0% (0.019 mean cosine drift). Only at 128d (512 B/row) did retrieval hit a real cliff (86.7% Hit@1, 0.128 drift).

2. Embedding & inserting items from TypeScript

Here is the complete ingestion path—calling Ollama, truncating to 256d (or keeping 768d), and writing both the DiskANN vector and FTS5 row in one atomic SQLite batch:

import { createClient } from "@libsql/client";

const db = createClient({ url: "file:data/knowledge.db" });

function sliceAndNormalize(vec: number[], dim = 256): Float32Array {
  const out = new Float32Array(dim);
  let normSq = 0;
  for (let i = 0; i < dim; i++) {
    out[i] = vec[i]!;
    normSq += out[i]! * out[i]!;
  }
  const inv = normSq > 0 ? 1 / Math.sqrt(normSq) : 1;
  for (let i = 0; i < dim; i++) out[i]! *= inv;
  return out;
}

// Accepts string | { image } | { audio } | { image, text }
async function embedItem(input: string | { text?: string; image?: string; audio?: string }) {
  const res = await fetch("http://127.0.0.1:11434/api/embed", {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ model: "embeddinggemma-2:latest", input }),
  });
  const { embeddings } = (await res.json()) as { embeddings: number[][] };
  return sliceAndNormalize(embeddings[0]!, 256);
}

// Atomic upsert into both DiskANN table and FTS5 keyword index
async function indexDocumentUnit(unit: {
  id: string;
  modality: string;
  title: string;
  text: string;
  notes?: string;
  pngBase64?: string; // raw base64 WITHOUT "data:image/png;base64," prefix!
}) {
  const formattedText = `title: ${unit.title} | text: ${[unit.text, unit.notes].filter(Boolean).join("\n")}`;
  const payload = unit.pngBase64
    ? { image: unit.pngBase64, text: formattedText } // Bimodal fusion!
    : formattedText;

  const vec256 = await embedItem(payload);
  const vecJson = `[${Array.from(vec256).join(",")}]`;

  await db.batch([
    {
      sql: `INSERT OR REPLACE INTO docs (id, modality, title, extracted_text, speaker_notes, emb_256)
            VALUES (?, ?, ?, ?, ?, vector32(?))`,
      args: [unit.id, unit.modality, unit.title, unit.text, unit.notes ?? "", vecJson],
    },
    { sql: `DELETE FROM docs_fts WHERE id = ?`, args: [unit.id] },
    {
      sql: `INSERT INTO docs_fts (id, title, extracted_text, speaker_notes) VALUES (?, ?, ?, ?)`,
      args: [unit.id, unit.title, unit.text, unit.notes ?? ""],
    },
  ], "write");
}

3. How to ingest PDFs, PowerPoints, scans, and audio without losing information

Getting data OUT: querying SQLite in 16 milliseconds

Getting high-precision answers back out of SQLite takes three small steps—and skipping any one of them measurably degrades quality or speed.

Step 1: Prepend an asymmetric task prefix to the user’s query (+25.5% margin for free)

EmbeddingGemma 2 expects documents to be indexed with title: ... | text: ... and queries to declare what kind of retrieval they are performing:

• General document / slide / image search: task: search result | query: ${userQuery}
• Direct factual question: task: question answering | query: ${userQuery}
• Source code lookup: task: code retrieval | query: ${userQuery}

Across our 15-query benchmark, adding that static prefix string cost 0 ms of extra latency and widened the cosine margin between the right answer and the closest distractor by +25.5% (0.0984 → 0.1235) on 14 out of 15 queries.

Step 2: Run DiskANN vector_top_k and FTS5 bm25() in parallel

Neither dense vectors nor keyword search wins every business query on its own:

• SQLite FTS5 alone (0.21 ms): Scored 73.3% Hit@1 on text/code—and 0% on zero-caption scans, charts, audio, zero-word-overlap paraphrases, and cross-lingual queries (where a Japanese, Spanish, Arabic, Chinese, or Hindi query searches English manuals).
• Dense vectors alone (0.69 ms at 256d): Scored 93.3%–100% Hit@1 across paraphrases, 6 languages, scans, and charts—but when an engineer pastes an exact 3 a.m. pager code like ERR_UPSTREAM_RST_499 or an exact invoice ID, keyword matching is unambiguous.

With @libsql/client, both live in the same SQLite file and run in under 1 ms combined:

async function searchHybrid(userQuery: string, limit = 10) {
  // 1. Embed query locally via Ollama (~14.9 ms on M4 Max)
  const qVec = await embedItem(`task: search result | query: ${userQuery}`);
  const qJson = `[${Array.from(qVec).join(",")}]`;

  // 2. Sanitize FTS5 tokens ("42M letter of intent" -> "42M AND letter AND intent")
  const ftsExpr = userQuery
    .replace(/[^\p{L}\p{N}_-]+/gu, " ")
    .trim()
    .split(/\s+/)
    .filter((w) => w.length > 2)
    .join(" AND ");

  // 3. Query DiskANN (~0.69 ms) + FTS5 (~0.21 ms) in parallel
  const [vecRows, ftsRows] = await Promise.all([
    db.execute({
      sql: `SELECT d.id, d.parent_doc_id, d.page_number, d.modality, d.title,
                   vector_distance_cos(d.emb_256, vector32(?)) AS cos_dist
            FROM vector_top_k('idx_docs_emb_256', vector32(?), 25) AS top_k
            JOIN docs d ON d.rowid = top_k.id
            ORDER BY cos_dist ASC`,
      args: [qJson, qJson],
    }),
    ftsExpr
      ? db.execute({
          sql: `SELECT id, bm25(docs_fts, 2.0, 1.0, 1.4) AS score
                FROM docs_fts WHERE docs_fts MATCH ? ORDER BY score ASC LIMIT 25`,
          args: [ftsExpr],
        })
      : Promise.resolve({ rows: [] }),
  ]);

  // 4. Fuse ranks with Reciprocal Rank Fusion (k = 12) — avoids cross-modal cosine scale mismatch!
  const rrf = new Map<string, { score: number; row: any }>();
  vecRows.rows.forEach((r, i) => {
    rrf.set(String(r.id), { score: 1 / (12 + i + 1), row: r });
  });
  ftsRows.rows.forEach((r, i) => {
    const id = String(r.id);
    const boost = (i === 0 ? 1.35 : 1.0) / (12 + i + 1);
    const prev = rrf.get(id);
    if (prev) prev.score += boost;
  });

  return [...rrf.values()].sort((a, b) => b.score - a.score).slice(0, limit);
}

Two retrieval traps to avoid in production

1. Never filter mixed modalities with a single hard cosine threshold (e.g., cosine > 0.75). Even though text, scans, photos, and audio share one 768d sphere, their similarity ceilings differ: text-to-scanned-document peaks around 0.76–0.85, text-to-photo around 0.72–0.77, and text-to-WAV-audio around 0.65–0.69 (where all 3 audio clips still ranked #1 cleanly). Rank fusion (RRF) handles this naturally; a naive 0.75 cutoff silently deletes every audio result.
2. Never pull vectors into JavaScript on the hot path. Calling vector_top_k('idx_docs_emb_256', ...) inside libSQL’s native C engine took 0.69 ms. When our diagnostic UI instead called SELECT vector_extract(emb_vis), vector_extract(emb_txt), vector_extract(emb_fused) across 90 pages to serialize 270 vectors as JSON strings and score them in JS, database time jumped from 0.69 ms to 88.5 ms (128× slower). Keep distance math inside SQLite.

How fast is it? Measured speed, throughput, and footprint

All timings below were measured warm on an Apple M4 Max (64 GB unified memory) running Ollama (embeddinggemma-2:latest, safetensors nvfp4 on the mlx runner) and local @libsql/client (file:data/embeddings.db):

Put in business terms: indexing a 500-page deal room of PDFs and PowerPoint decks takes roughly 4.5 minutes on one laptop with zero cloud spend, occupies under 1 MB of 256d SQLite vectors, and answers every search in 16 milliseconds. Indexing 10,000 text/code chunks takes under 4 minutes.

How good of a tool is it? Engineering scorecard

After wiring the full pipeline end-to-end and stress-testing it on messy documents, here is our honest evaluation of EmbeddingGemma 2 + Ollama + @libsql/client as a production tool:

Bottom line

If you are building private document search, local agent memory, a dataroom/research workbench, or an offline knowledge pack up to a few hundred thousand chunks or tens of thousands of pages, this is one of the highest-leverage local AI tools available right now. You trade four external services and an OCR bill for 1.27 GB of local RAM, ~50 lines of MuPDF/JSZip glue for PDFs and PowerPoints, and a single SQLite .db file that answers hybrid multimodal queries in 16 ms.

Run the reference studio & CLI yourself

The complete Bun + TypeScript implementation—including the MuPDF PDF rasterizer, PowerPoint XML + speaker-note renderer, 256d Matryoshka slicer, freeform cross-document search matcher (mise run search), and interactive web studio (http://127.0.0.1:3456)—is in The-Focus-AI/gemmaembeddings:

git clone https://github.com/The-Focus-AI/gemmaembeddings.git
cd gemmaembeddings
mise trust && mise install && mise run install

# 1. Pull embeddinggemma-2:latest (1.33 GB) and index all 122 multimodal units into data/embeddings.db
mise run model:pull
mise run seed
mise run seed:docs

# 2. Query everything from the CLI (~16 ms SQLite + Ollama) or open the web studio on :3456
mise run search -- "42M letter of intent Project Horizon"
mise run search -- --scope=pdf --lens=vision "ViDoRe table benchmark results"
mise run dev