LanceDB connector

Write and upsert rows to LanceDB tables with schemas derived from Rust structs, vector columns via VectorSchemaProvider, and both system-managed and user-managed table lifecycles.

Version: v 1.0.14
Last reviewed: Nov 4, 2026

ExperimentalThe Rust SDK is experimental. It is outside CocoIndex's compatibility promise: any release, including a minor one, may change or remove its API. It is not yet published on crates.io.

The lancedb connector provides utilities for writing rows to LanceDB tables, with automatic schema inference from Rust structs and support for declaring vector and full-text search (FTS) indexes. CocoIndex manages the table lifecycle (creating, dropping, and evolving the schema) and keeps rows in sync via incremental upserts and deletions.

use cocoindex::connectors::lancedb::{self, LanceDatabase, TableSchema};

Dependencies

The module is compiled with the lancedb Cargo feature of the cocoindex crate. It is built on the native lancedb crate and Arrow.

Connection setup

A LanceDatabase wraps a lancedb::Connection. It is cheap to clone: the connection is shared.

impl LanceDatabase {
    pub async fn connect(uri: &str) -> Result<Self>
    pub async fn connect_with_options(uri: &str, options: ConnectOptions) -> Result<Self>
    pub fn state_id(&self) -> &str
    pub fn connection(&self) -> &lancedb::Connection
}

#[derive(Clone, Debug, Default)]
pub struct ConnectOptions {
    /// How long a table handle may serve reads from a stale version before it
    /// checks for a newer one. `None` reads a table version once per handle;
    /// `Duration::ZERO` checks on every read.
    pub read_consistency_interval: Option<Duration>,
    /// Object-store options (`aws_access_key_id`, `timeout`, ...) for a
    /// cloud-storage URI.
    pub storage_options: BTreeMap<String, String>,
}

Parameters:

uri — LanceDB URI (local path like "./lancedb_data" or object-store URI like "s3://bucket/path"). The URI is also the database’s state_id(), its stable identity; the options are not part of it, since they may carry credentials.
options — connect_with_options only: the read consistency interval and the object-store options. ConnectOptions::default() is the same as connect(uri). LanceDB Cloud (db://) URIs are not supported.

Returns: A LanceDatabase. connection() returns the underlying lancedb::Connection, for queries.

Example:

let db = LanceDatabase::connect(&lancedb_uri()).await?;

// With options
let db = LanceDatabase::connect_with_options(
    uri,
    lancedb::ConnectOptions {
        read_consistency_interval: Some(std::time::Duration::ZERO),
        storage_options: [("timeout".to_string(), "30s".to_string())].into(),
    },
)
.await?;

As target

The lancedb connector provides target state APIs for writing rows to tables. CocoIndex tracks what rows should exist and automatically handles upserts and deletions.

Declaring target states

Setting up a connection

Declare a context key for your LanceDatabase, then provide the connection to the environment:

Note

The key name is load-bearing across runs — it’s the stable identity CocoIndex uses to track managed tables. See ContextKey as stable identity before renaming.

use cocoindex::connectors::lancedb::LanceDatabase;

cocoindex::context_key!(static DB: LanceDatabase);

let db = LanceDatabase::connect(&lancedb_uri()).await?;
let app = Environment::builder()
    .db_path(PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(".cocoindex_db"))
    .provide_key(&DB, db)
    .build()
    .await?
    .app("CodeEmbeddingLanceDBRust")
    .await?;

context_key!(static DB: LanceDatabase) declares the key; its name is the stable identity. provide_key(&DB, db) provides the connection, in a #[cocoindex::lifespan] function or on Environment::builder() as here.

Tables (parent state)

Declares a table as a target state. Returns a LanceTableTarget handle for declaring rows.

pub async fn mount_table_target(
    ctx: &Ctx,
    db: &ContextKey<LanceDatabase>,
    table_name: impl Into<String>,
    table_schema: TableSchema,
) -> Result<LanceTableTarget>

pub fn declare_table_target(
    ctx: &Ctx,
    db: &ContextKey<LanceDatabase>,
    table_name: impl Into<String>,
    table_schema: TableSchema,
) -> Result<LanceTableTarget>

pub fn table_target(
    ctx: &Ctx,
    db: &ContextKey<LanceDatabase>,
    table_name: impl Into<String>,
    table_schema: TableSchema,
) -> Result<TargetState<TableSpec>>

Each has a _with_options variant (mount_table_target_with_options, declare_table_target_with_options, table_target_with_options) that takes a trailing options: ManagedTargetOptions.

Parameters:

ctx — The component’s Ctx.
db — The ContextKey<LanceDatabase> identifying the connection to use.
table_name — Name of the table. It must start with an ASCII letter or _ and contain only ASCII letters, digits and _.
table_schema — Schema definition including columns and primary key (see Table Schema).
options — _with_options variants only. ManagedTargetOptions::default() lets CocoIndex manage the table lifecycle; ManagedTargetOptions::user_managed() assumes the table exists and never creates or drops it.

Returns: mount_table_target and declare_table_target return a LanceTableTarget, ready for declaring rows. mount_table_target is the usual choice; declare_table_target is a sync form for declaring the table and its rows in the same component. table_target returns the raw TargetState for mount_target / declare_target_state_with_child.

A LanceTableTarget is Clone and Serialize. It can be an argument of mount!, use_mount!, mount_each! or a memoized function: it fingerprints by its stable key, the context key name and the table name. The schema and managed_by are not part of that identity.

#[cocoindex::function]
async fn declare_row_in_component(
    ctx: &Ctx,
    id: i64,
    table: lancedb::LanceTableTarget,
) -> Result<()> {
    table.declare_row(
        ctx,
        &Row {
            id,
            text: format!("doc{id}"),
            embedding: vec![1.0, 0.0, 0.0],
        },
    )
}

let table = lancedb::mount_table_target(&ctx, &DB, table_name, schema()).await?;
cocoindex::use_mount_each!(vec![("1".to_string(), 1_i64)], |id| {
    declare_row_in_component(ctx, id, table.clone())
})
.await?;

CocoIndex automatically runs LanceDB table.optimize() once every 50 applied row batches of a table target in a run. All row batches of a run write through one table handle, opened by the first batch and moved to the table’s latest version before each later one.

Different from Python

Optimization is counted in row batches, not driven by table and index statistics as in Python. A run with fewer than 50 row batches does not optimize the table.

Table optimization compacts small fragments, cleans up deletion files, and reindexes new data so the freshest rows are available through declared indexes; see LanceDB’s compaction and cleanup guidance.

Note

Automatic optimize runs inline with CocoIndex’s row-mutation path. To avoid LanceDB optimistic commit conflicts, use CocoIndex as the owner of writes that sync source data into the table. External LanceDB connections outside CocoIndex should generally be read-only while CocoIndex is updating the table.

Rows (child states)

Once you have a LanceTableTarget, declare rows to be upserted:

impl LanceTableTarget {
    pub fn declare_row<R: Serialize>(&self, ctx: &Ctx, row: &R) -> Result<()>
}

table.declare_row(&ctx, row)?;

Parameters:

row — Any Serialize value that serializes to an object with the schema’s columns. It must include all primary key columns. Extra fields are dropped and missing ones become null.

Vector indexes (attachment)

Declare a vector index on a vector column to accelerate similarity search. Vector indexes are an attachment to a LanceTableTarget:

impl LanceTableTarget {
    pub fn declare_vector_index(
        &self,
        ctx: &Ctx,
        column: &str,
        options: VectorIndexOptions,
    ) -> Result<()>
}

pub struct VectorIndexOptions {
    /// Index name; defaults to `<column>_idx`.
    pub name: Option<String>,
    /// Distance metric: "cosine" (default), "l2", or "dot".
    pub metric: &'static str,
    /// Index type: "ivf_pq" (default) or "ivf_hnsw_pq".
    pub index_type: &'static str,
    pub num_partitions: Option<u32>,
    pub num_sub_vectors: Option<u32>,
    pub num_bits: Option<u32>,
    pub m: Option<u32>,
    pub ef_construction: Option<u32>,
}

Different from Python

Two names differ from Python: the HNSW index type is "ivf_hnsw_pq", not "hnsw_pq", and the default index name is <column>_idx, not the column name.

Parameters:

name — Logical index name (defaults to <column>_idx).
column — Vector column to index.
metric — Distance metric: "cosine" (default), "l2", or "dot".
index_type — Index algorithm: "ivf_pq" (IVF-PQ, default) or "ivf_hnsw_pq" (HNSW-PQ).
num_partitions — (IVF-PQ only) Number of IVF partitions.
num_sub_vectors — (IVF-PQ / HNSW-PQ) Number of PQ sub-vectors.
num_bits — (IVF-PQ / HNSW-PQ) Number of bits per PQ code.
m — (HNSW-PQ only) Maximum number of HNSW edges per node.
ef_construction — (HNSW-PQ only) Size of the HNSW candidate list during build.

Parameters left as None fall back to LanceDB’s defaults.

Example:

table.declare_vector_index(&ctx, "embedding", lancedb::VectorIndexOptions::default())?;

FTS indexes (attachment)

Declare a full-text search (FTS) index on a text column to enable keyword and phrase search. Like vector indexes, FTS indexes are an attachment to a LanceTableTarget:

impl LanceTableTarget {
    pub fn declare_fts_index(
        &self,
        ctx: &Ctx,
        column: &str,
        options: FtsIndexOptions,
    ) -> Result<()>
}

pub struct FtsIndexOptions {
    /// Index name; defaults to `<column>_fts_idx`.
    pub name: Option<String>,
    /// Record token positions (enables phrase queries). Defaults to `true`.
    pub with_position: bool,
    /// Stemming/stop-word language. `None` uses LanceDB's default (English).
    pub language: Option<String>,
}

Different from Python

The default FTS index name is <column>_fts_idx, not the column name.

Parameters:

name — Logical index name (defaults to <column>_fts_idx).
column — Text column to index.
language — Tokenizer language (e.g. "English", "Chinese"). None uses LanceDB’s default, English.
with_position — Whether to store token positions (enables phrase queries). Defaults to true.

Example:

table.declare_fts_index(&ctx, "text", lancedb::FtsIndexOptions::default())?;

// With a language
table.declare_fts_index(
    &ctx,
    "text",
    lancedb::FtsIndexOptions {
        language: Some("English".to_string()),
        ..Default::default()
    },
)?;

Note

Indexes are reconciled as part of the table’s target state: changing a declaration replaces the index in place, removing a declaration drops the index, and dropping the table removes all its indexes.

Table schema: from a Rust struct

Define the table structure with a struct that derives SchemaFields (and Serialize):

impl TableSchema {
    pub fn from_row<T: SchemaFields>(
        primary_key: impl IntoIterator<Item = impl Into<String>>,
    ) -> Result<Self>

    pub fn with_vector_dim(self, field_name: &str, dim: usize) -> Result<Self>
    pub async fn with_vector_schema(
        self,
        field_name: &str,
        provider: &(impl VectorSchemaProvider + ?Sized),
    ) -> Result<Self>
    pub fn with_column_type(self, field_name: &str, col_type: ColumnType) -> Result<Self>
}

Per-column overrides are builder calls on the derived schema: with_vector_schema or with_vector_dim for a vector column, with_column_type for a type override. with_column_type keeps the column’s nullability. Each returns an error for a column that is not in the schema.

Parameters:

T — The row struct whose fields define table columns.
primary_key — Column names forming the primary key. A primary key value must serialize to a string or an integer.

Every column name must start with an ASCII letter or _ and contain only ASCII letters, digits and _.

Example:

#[derive(Clone, Serialize, Deserialize, SchemaFields)]
struct DocEmbedding {
    id: i64,
    filename: String,
    chunk_start: i64,
    chunk_end: i64,
    text: String,
    #[coco(vector)]
    embedding: Vec<f32>,
}

fn doc_embedding_schema() -> Result<TableSchema> {
    TableSchema::from_row::<DocEmbedding>(["id"])?.with_vector_dim("embedding", EMBED_DIM)
}

#[coco(vector)] marks a vector column whose dimension is set with with_vector_dim or with_vector_schema; #[coco(vector = 384)] fixes it in the struct. Option<T> fields become nullable columns.

Rust field types are automatically mapped to Arrow types:

Rust Type	`ColumnType`	Arrow Type
`bool`	`Bool`	`bool`
`i16`, `i32`, `i64`	`Int16`, `Int32`, `Int64`	`int16`, `int32`, `int64`
`f32`, `f64`	`Float32`, `Float64`	`float32`, `float64`
`String`, `Uuid`, dates and times	`Text`	`string`
`Vec<u8>`	`Binary`	`binary`
other `Vec<T>`, maps, nested structs, `Decimal`, `Duration`, `#[coco(json)]`	`Json`	`string` (JSON encoded)
`Vec<f32>` with `#[coco(vector)]`	`Vector(dim)`	`fixed_size_list<float32>`
`Vec<f32>` with `#[coco(vector, half)]`	`HalfVector(dim)`	`fixed_size_list<float16>`

ColumnType also has Int8, UInt8, UInt16, UInt32, UInt64 and Float16, for with_column_type and explicit columns. A #[coco(type = "…")] attribute is an error for LanceDB: use with_column_type with a ColumnType.

Column type overrides

Use with_column_type to store a column with another ColumnType:

use cocoindex::connectors::lancedb::{ColumnType, TableSchema};

let schema = TableSchema::from_row::<MyRow>(["id"])?
    .with_column_type("id", ColumnType::Int32)?
    .with_column_type("value", ColumnType::Float32)?;

Or declare the field as i32 / f32 in the struct.

Different from Python

There is no per-column encoder. A Rust row is encoded by its Serialize impl: to convert a value before it is written, put #[serde(serialize_with = "…")] on the field.

VectorSchemaProvider

For a #[coco(vector)] field, with_vector_schema takes the vector dimension and element type from a VectorSchemaProvider: an embedder, or a VectorSchema. For an embedder held in the context, pass ctx.get_key(&EMBEDDER)?. An f16 provider selects a float16 list; f32 and f64 select float32. See Vector Schema for details.

let schema = TableSchema::from_row::<DocEmbedding>(["id"])?
    .with_vector_schema("embedding", ctx.get_key(&EMBEDDER)?)
    .await?;

Table schema: explicit column definitions

Define columns directly using ColumnDef:

impl TableSchema {
    pub fn new(
        columns: impl IntoIterator<Item = (impl Into<String>, ColumnDef)>,
        primary_key: impl IntoIterator<Item = impl Into<String>>,
    ) -> Result<Self>
}

impl ColumnDef {
    pub fn new(col_type: ColumnType) -> Self
    pub fn nullable(self) -> Self
}

ColumnDef::new(col_type) is not nullable; .nullable() opts in. TableSchema::new returns an error for an empty primary key, a primary key column that is not in columns, or a duplicate column.

Example:

use cocoindex::connectors::lancedb::{ColumnDef, ColumnType, TableSchema};

TableSchema::new(
    [
        ("id", ColumnDef::new(ColumnType::Int64)),
        ("text", ColumnDef::new(ColumnType::Text)),
        ("summary", ColumnDef::new(ColumnType::Text).nullable()),
        ("embedding", ColumnDef::new(ColumnType::Vector(3))),
    ],
    ["id"],
)?

Example

use std::path::PathBuf;

use cocoindex::connectors::lancedb::{self, LanceDatabase, TableSchema};
use cocoindex::prelude::*;

const EMBED_DIM: usize = 384;
const TABLE: &str = "doc_embeddings";

cocoindex::context_key!(static DB: LanceDatabase);

#[derive(Clone, Serialize, Deserialize, SchemaFields)]
struct DocEmbedding {
    id: i64,
    filename: String,
    chunk_start: i64,
    chunk_end: i64,
    text: String,
    #[coco(vector)]
    embedding: Vec<f32>,
}

#[cocoindex::function]
async fn process_file(ctx: &Ctx, file: FileEntry) -> Result<Vec<DocEmbedding>> {
    // chunk and embed the file ...
}

fn doc_embedding_schema() -> Result<TableSchema> {
    TableSchema::from_row::<DocEmbedding>(["id"])?.with_vector_dim("embedding", EMBED_DIM)
}

async fn app_main(ctx: Ctx, sourcedir: PathBuf) -> Result<()> {
    let table = lancedb::mount_table_target(&ctx, &DB, TABLE, doc_embedding_schema()?).await?;

    let files = walk_items(&sourcedir, &["**/*.md"])?;
    let rows_by_file = use_mount_each!(files, |file| process_file(ctx, file)).await?;

    for rows in &rows_by_file {
        for row in rows {
            table.declare_row(&ctx, row)?;
        }
    }
    Ok(())
}

See examples/rust/text_embedding_lancedb for the full program, including the lancedb::vector_search query.