Walleye docs
Python
The client is lancedb. Walleye speaks the LanceDB remote protocol, so there is nothing Walleye-specific to import.
pip install lancedb pyarrow
import os
import lancedb
db = lancedb.connect(
"db://walleye",
api_key=os.environ["WALLEYE_TOKEN"],
host_override=os.environ["WALLEYE_URL"],
region="us-east-1",
)
See connect for where the token comes from and why
region is there.
Tables
db.create_table("clicks", data=rows) # schema inferred from the rows
db.create_table("docs", schema=schema) # or from an Arrow schema
db.open_table("clicks")
db.table_names()
db.drop_table("clicks")
create_table on a name that already exists raises; the error says so.
Writing
table.add(rows) # list of dicts
table.add(pa.Table.from_pylist(rows, schema=schema))
add returns once the batch is durable. Rows are keyed — by the field you
marked lance-schema:unenforced-primary-key, otherwise by a hash of the row —
so re-adding a row you already wrote replaces it instead of duplicating it. See
your first table.
Reading
table.count_rows()
table.count_rows("city = 'seattle'")
table.search().where("id > 1").select(["id", "city"]).limit(10).offset(20).to_list()
table.search([0.0, 0.9]).limit(5).to_list()
table.search([0.0, 0.9]).where("city = 'portland'").limit(5).to_list()
where is a SQL expression, select takes column names — not expressions —
and a vector search returns _distance on each row.
Indexes
table.create_index(metric="cosine")
table.list_indices()
Vector columns are indexed from the first row without this. create_index
chooses the metric — l2, cosine or dot — and rewrites the flushed
generations before it returns. Queries use the index’s metric; asking a query
for a different one is an error.
Scalar indexes are refused. Index.btree(), Index.bitmap() and
Index.labelList() all fail: the node accepts only index types beginning
IVF or HNSW, because it maintains one layout.
index type BTREE is not supported; vector columns use IVF_HNSW_SQ
Filtering a scalar column still works — where is evaluated as a scan, so it
is correct, and the cost is the scan.
SQL
The client has no SQL call, so post to the node:
import requests
rows = requests.post(
f"{os.environ['WALLEYE_URL']}/v1/query",
headers={"authorization": f"Bearer {os.environ['WALLEYE_TOKEN']}"},
json={"sql": "SELECT city, count(*) AS n FROM clicks GROUP BY city"},
).json()
Read-only DataFusion SQL, sixty second deadline, 8 MiB of JSON back. On one
node it reads every table on the node; in a cluster it also gathers the tables
this node does not own from their owners, so a SELECT spans the cluster.
The HTTP surface has the rest of the routes.
What raises
update, delete and merge_insert are not implemented, and they are not a
refusal with a reason either: there is no route, so the node answers 404 with
an empty body. Whatever the client raises for that is what you will see. Write
a row with the same key instead — the newest one wins.
The same is true of add_columns, alter_columns, drop_columns,
list_versions, restore, the tag calls and per-index stats: no route, 404,
no reason.
Everything else here is a 400 with the reason as the body:
Index.btree(),Index.bitmap()andIndex.labelList()— onlyIVF*andHNSW*index types are accepted- full-text search and
Index.fts() order_byon a search; use SQL- column expressions in
select; use SQL add(..., mode="overwrite"); drop and recreate instead- a column named with a leading
_, or a table name outside[A-Za-z0-9_-] - searching a table with more than one vector column without naming
vector_column - creating a table that already exists with a different schema, whatever
modeyou asked for
order_by is refused on a plain scan as well as on a search, and the metric
check is a string compare against the index’s own spelling, so asking a query
for euclidean against an l2 index is an error rather than a synonym.
Namespaces are the exception that is neither: they are accepted and ignored, and every namespace id lists the one root. The HTTP surface has the message each one returns, and what the extractor does with a bad JSON body.