Walleye docs

JavaScript

The client is @lancedb/lancedb. Walleye speaks the LanceDB remote protocol, so there is nothing Walleye-specific to install.

npm install @lancedb/lancedb
import * as lancedb from "@lancedb/lancedb";

const db = await lancedb.connect({
  uri: "db://walleye",
  apiKey: process.env.WALLEYE_TOKEN,
  hostOverride: process.env.WALLEYE_URL,
  region: "us-east-1",
});

See connect for where the token comes from and why region is there.

Tables

await db.createTable("clicks", rows);
await db.openTable("clicks");
await db.tableNames();
await db.dropTable("clicks");

Writing

await table.add(rows);

add resolves once the batch is durable. Rows are keyed, so re-adding a row you already wrote replaces it rather than duplicating it.

Reading

await table.countRows();
await table.countRows("city = 'seattle'");

await table.query().where("id > 1").select(["id", "city"]).limit(10).toArray();
await table.search([0.0, 0.9]).limit(5).toArray();
await table.search([0.0, 0.9]).where("city = 'portland'").limit(5).toArray();

A vector search returns _distance on each row.

Indexes

await table.createIndex("vector", { config: lancedb.Index.hnswSq({ distanceType: "cosine" }) });
await table.listIndices();

Vector columns are indexed from the first row without this. The call is how you choose the metric, and it rewrites what is already flushed before it resolves.

What the node takes from the call is the column and the metric — l2, cosine or dot. A query then uses that metric, and asking it for a different one is an error.

Scalar indexes are refused. Index.btree(), Index.bitmap() and Index.labelList() all reject: the node accepts only index types beginning IVF or HNSW, and there is one layout behind all of them, so ivfPq, ivfFlat, hnswSq and the rest are accepted and then built the same way. listIndices reports IVF_HNSW_SQ whichever you asked for.

index type BTREE is not supported; vector columns use IVF_HNSW_SQ

Filtering a scalar column still works — where is evaluated as a scan, so it is correct, and the cost is the scan.

SQL

No client call; post to the node. It is read-only DataFusion SQL with a sixty second deadline and 8 MiB of JSON back; on three nodes it spans the cluster.

const rows = await fetch(`${process.env.WALLEYE_URL}/v1/query`, {
  method: "POST",
  headers: {
    authorization: `Bearer ${process.env.WALLEYE_TOKEN}`,
    "content-type": "application/json",
  },
  body: JSON.stringify({ sql: "SELECT city, count(*) AS n FROM clicks GROUP BY city" }),
}).then((response) => response.json());

The HTTP surface has the rest of the routes.

What rejects

update, delete and mergeInsert are not implemented, and they are not a rejection with a reason either: there is no route, so the node answers 404 with an empty body. Write a row with the same key instead — the newest one wins. The same goes for addColumns, alterColumns, dropColumns, listVersions, restore, the tag calls and per-index stats.

Everything else here is a 400 with the reason as the body:

  • Index.btree(), Index.bitmap() and Index.labelList() — only IVF* and HNSW* index types are accepted
  • full-text search and Index.fts()
  • ordering on a search; use SQL
  • column expressions in select; use SQL
  • add with mode: "overwrite"; drop and recreate instead
  • a column named with a leading _, or a table name outside [A-Za-z0-9_-]
  • searching a table with more than one vector column without naming vectorColumn
  • creating a table that already exists with a different schema, whatever mode you asked for

Ordering is refused on a plain query as well as on a search, and the metric check is a string compare against the index’s own spelling, so asking for euclidean against an l2 index is an error rather than a synonym.

Namespaces are the exception that is neither: they are accepted and ignored. The HTTP surface has the message each one returns, and what the extractor does with a bad JSON body.