# LanceDB connector > **CocoIndex v1, Rust SDK (experimental).** This page documents the CocoIndex **v1** Rust SDK (the `cocoindex` crate). The Rust SDK is experimental: any release may change or remove its API, and it is not yet published on crates.io. The Python SDK pages are at https://cocoindex.io/docs/. > > Source: https://cocoindex.io/docs/rust/connectors/lancedb/ · Rust docs index: https://cocoindex.io/docs/rust/llms.txt · Rust SDK tour: https://github.com/cocoindex-io/cocoindex/blob/main/rust/sdk/SHOWCASE.md · Rust examples: https://github.com/cocoindex-io/cocoindex/tree/main/examples/rust The `lancedb` connector provides utilities for writing rows to LanceDB tables, with automatic schema inference from Rust structs and support for declaring vector and full-text search (FTS) indexes. CocoIndex manages the table lifecycle (creating, dropping, and evolving the schema) and keeps rows in sync via incremental upserts and deletions. ```rust use cocoindex::connectors::lancedb::{self, LanceDatabase, TableSchema}; ``` **Note — Dependencies** The module is compiled with the `lancedb` Cargo feature of the `cocoindex` crate. It is built on the native [`lancedb`](https://docs.rs/lancedb) crate and Arrow. ## Connection setup A `LanceDatabase` wraps a `lancedb::Connection`. It is cheap to clone: the connection is shared. ```rust impl LanceDatabase { pub async fn connect(uri: &str) -> Result pub async fn connect_with_options(uri: &str, options: ConnectOptions) -> Result pub fn state_id(&self) -> &str pub fn connection(&self) -> &lancedb::Connection } #[derive(Clone, Debug, Default)] pub struct ConnectOptions { /// How long a table handle may serve reads from a stale version before it /// checks for a newer one. `None` reads a table version once per handle; /// `Duration::ZERO` checks on every read. pub read_consistency_interval: Option, /// Object-store options (`aws_access_key_id`, `timeout`, ...) for a /// cloud-storage URI. pub storage_options: BTreeMap, } ``` **Parameters:** - `uri` — LanceDB URI (local path like `"./lancedb_data"` or object-store URI like `"s3://bucket/path"`). The URI is also the database's `state_id()`, its stable identity; the options are not part of it, since they may carry credentials. - `options` — `connect_with_options` only: the read consistency interval and the object-store options. `ConnectOptions::default()` is the same as `connect(uri)`. LanceDB Cloud (`db://`) URIs are not supported. **Returns:** A `LanceDatabase`. `connection()` returns the underlying `lancedb::Connection`, for queries. **Example:** ```rust let db = LanceDatabase::connect(&lancedb_uri()).await?; // With options let db = LanceDatabase::connect_with_options( uri, lancedb::ConnectOptions { read_consistency_interval: Some(std::time::Duration::ZERO), storage_options: [("timeout".to_string(), "30s".to_string())].into(), }, ) .await?; ``` ## As target The `lancedb` connector provides target state APIs for writing rows to tables. CocoIndex tracks what rows should exist and automatically handles upserts and deletions. ### Declaring target states #### Setting up a connection Declare a context key for your `LanceDatabase`, then provide the connection to the environment: **Note** The key name is load-bearing across runs — it's the stable identity CocoIndex uses to track managed tables. See [ContextKey as stable identity](/docs/rust/programming_guide/context/#contextkey-as-stable-identity) before renaming. ```rust use cocoindex::connectors::lancedb::LanceDatabase; cocoindex::context_key!(static DB: LanceDatabase); let db = LanceDatabase::connect(&lancedb_uri()).await?; let app = Environment::builder() .db_path(PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(".cocoindex_db")) .provide_key(&DB, db) .build() .await? .app("CodeEmbeddingLanceDBRust") .await?; ``` `context_key!(static DB: LanceDatabase)` declares the key; its name is the stable identity. `provide_key(&DB, db)` provides the connection, in a `#[cocoindex::lifespan]` function or on `Environment::builder()` as here. #### Tables (parent state) Declares a table as a target state. Returns a `LanceTableTarget` handle for declaring rows. ```rust pub async fn mount_table_target( ctx: &Ctx, db: &ContextKey, table_name: impl Into, table_schema: TableSchema, ) -> Result pub fn declare_table_target( ctx: &Ctx, db: &ContextKey, table_name: impl Into, table_schema: TableSchema, ) -> Result pub fn table_target( ctx: &Ctx, db: &ContextKey, table_name: impl Into, table_schema: TableSchema, ) -> Result> ``` Each has a `_with_options` variant (`mount_table_target_with_options`, `declare_table_target_with_options`, `table_target_with_options`) that takes a trailing `options: ManagedTargetOptions`. **Parameters:** - `ctx` — The component's `Ctx`. - `db` — The `ContextKey` identifying the connection to use. - `table_name` — Name of the table. It must start with an ASCII letter or `_` and contain only ASCII letters, digits and `_`. - `table_schema` — Schema definition including columns and primary key (see [Table Schema](#table-schema-from-a-rust-struct)). - `options` — `_with_options` variants only. `ManagedTargetOptions::default()` lets CocoIndex manage the table lifecycle; `ManagedTargetOptions::user_managed()` assumes the table exists and never creates or drops it. **Returns:** `mount_table_target` and `declare_table_target` return a `LanceTableTarget`, ready for declaring rows. `mount_table_target` is the usual choice; `declare_table_target` is a sync form for declaring the table and its rows in the same component. `table_target` returns the raw `TargetState` for `mount_target` / `declare_target_state_with_child`. A `LanceTableTarget` is `Clone` and `Serialize`. It can be an argument of `mount!`, `use_mount!`, `mount_each!` or a memoized function: it fingerprints by its stable key, the context key name and the table name. The schema and `managed_by` are not part of that identity. ```rust #[cocoindex::function] async fn declare_row_in_component( ctx: &Ctx, id: i64, table: lancedb::LanceTableTarget, ) -> Result<()> { table.declare_row( ctx, &Row { id, text: format!("doc{id}"), embedding: vec![1.0, 0.0, 0.0], }, ) } let table = lancedb::mount_table_target(&ctx, &DB, table_name, schema()).await?; cocoindex::use_mount_each!(vec![("1".to_string(), 1_i64)], |id| { declare_row_in_component(ctx, id, table.clone()) }) .await?; ``` CocoIndex automatically runs LanceDB `table.optimize()` once every 50 applied row batches of a table target in a run. All row batches of a run write through one table handle, opened by the first batch and moved to the table's latest version before each later one. Optimization is counted in row batches, not driven by table and index statistics as in Python. A run with fewer than 50 row batches does not optimize the table. Table optimization compacts small fragments, cleans up deletion files, and reindexes new data so the freshest rows are available through declared indexes; see LanceDB's [compaction and cleanup](https://docs.lancedb.com/performance#compaction-and-cleanup) guidance. **Note** Automatic optimize runs inline with CocoIndex's row-mutation path. To avoid LanceDB optimistic commit conflicts, use CocoIndex as the owner of writes that sync source data into the table. External LanceDB connections outside CocoIndex should generally be read-only while CocoIndex is updating the table. #### Rows (child states) Once you have a `LanceTableTarget`, declare rows to be upserted: ```rust impl LanceTableTarget { pub fn declare_row(&self, ctx: &Ctx, row: &R) -> Result<()> } table.declare_row(&ctx, row)?; ``` **Parameters:** - `row` — Any `Serialize` value that serializes to an object with the schema's columns. It must include all primary key columns. Extra fields are dropped and missing ones become null. #### Vector indexes (attachment) Declare a vector index on a vector column to accelerate similarity search. Vector indexes are an [attachment](/docs/rust/advanced_topics/custom_target_connector/#implementing-attachment-providers) to a `LanceTableTarget`: ```rust impl LanceTableTarget { pub fn declare_vector_index( &self, ctx: &Ctx, column: &str, options: VectorIndexOptions, ) -> Result<()> } pub struct VectorIndexOptions { /// Index name; defaults to `_idx`. pub name: Option, /// Distance metric: "cosine" (default), "l2", or "dot". pub metric: &'static str, /// Index type: "ivf_pq" (default) or "ivf_hnsw_pq". pub index_type: &'static str, pub num_partitions: Option, pub num_sub_vectors: Option, pub num_bits: Option, pub m: Option, pub ef_construction: Option, } ``` Two names differ from Python: the HNSW index type is `"ivf_hnsw_pq"`, not `"hnsw_pq"`, and the default index name is `_idx`, not the column name. **Parameters:** - `name` — Logical index name (defaults to `_idx`). - `column` — Vector column to index. - `metric` — Distance metric: `"cosine"` (default), `"l2"`, or `"dot"`. - `index_type` — Index algorithm: `"ivf_pq"` (IVF-PQ, default) or `"ivf_hnsw_pq"` (HNSW-PQ). - `num_partitions` — *(IVF-PQ only)* Number of IVF partitions. - `num_sub_vectors` — *(IVF-PQ / HNSW-PQ)* Number of PQ sub-vectors. - `num_bits` — *(IVF-PQ / HNSW-PQ)* Number of bits per PQ code. - `m` — *(HNSW-PQ only)* Maximum number of HNSW edges per node. - `ef_construction` — *(HNSW-PQ only)* Size of the HNSW candidate list during build. Parameters left as `None` fall back to LanceDB's defaults. **Example:** ```rust table.declare_vector_index(&ctx, "embedding", lancedb::VectorIndexOptions::default())?; ``` #### FTS indexes (attachment) Declare a full-text search (FTS) index on a text column to enable keyword and phrase search. Like vector indexes, FTS indexes are an [attachment](/docs/rust/advanced_topics/custom_target_connector/#implementing-attachment-providers) to a `LanceTableTarget`: ```rust impl LanceTableTarget { pub fn declare_fts_index( &self, ctx: &Ctx, column: &str, options: FtsIndexOptions, ) -> Result<()> } pub struct FtsIndexOptions { /// Index name; defaults to `_fts_idx`. pub name: Option, /// Record token positions (enables phrase queries). Defaults to `true`. pub with_position: bool, /// Stemming/stop-word language. `None` uses LanceDB's default (English). pub language: Option, } ``` The default FTS index name is `_fts_idx`, not the column name. **Parameters:** - `name` — Logical index name (defaults to `_fts_idx`). - `column` — Text column to index. - `language` — Tokenizer language (e.g. `"English"`, `"Chinese"`). `None` uses LanceDB's default, English. - `with_position` — Whether to store token positions (enables phrase queries). Defaults to `true`. **Example:** ```rust table.declare_fts_index(&ctx, "text", lancedb::FtsIndexOptions::default())?; // With a language table.declare_fts_index( &ctx, "text", lancedb::FtsIndexOptions { language: Some("English".to_string()), ..Default::default() }, )?; ``` **Note** Indexes are reconciled as part of the table's target state: changing a declaration replaces the index in place, removing a declaration drops the index, and dropping the table removes all its indexes. ### Table schema: from a Rust struct Define the table structure with a struct that derives `SchemaFields` (and `Serialize`): ```rust impl TableSchema { pub fn from_row( primary_key: impl IntoIterator>, ) -> Result pub fn with_vector_dim(self, field_name: &str, dim: usize) -> Result pub async fn with_vector_schema( self, field_name: &str, provider: &(impl VectorSchemaProvider + ?Sized), ) -> Result pub fn with_column_type(self, field_name: &str, col_type: ColumnType) -> Result } ``` Per-column overrides are builder calls on the derived schema: `with_vector_schema` or `with_vector_dim` for a vector column, `with_column_type` for a type override. `with_column_type` keeps the column's nullability. Each returns an error for a column that is not in the schema. **Parameters:** - `T` — The row struct whose fields define table columns. - `primary_key` — Column names forming the primary key. A primary key value must serialize to a string or an integer. Every column name must start with an ASCII letter or `_` and contain only ASCII letters, digits and `_`. **Example:** ```rust #[derive(Clone, Serialize, Deserialize, SchemaFields)] struct DocEmbedding { id: i64, filename: String, chunk_start: i64, chunk_end: i64, text: String, #[coco(vector)] embedding: Vec, } fn doc_embedding_schema() -> Result { TableSchema::from_row::(["id"])?.with_vector_dim("embedding", EMBED_DIM) } ``` `#[coco(vector)]` marks a vector column whose dimension is set with `with_vector_dim` or `with_vector_schema`; `#[coco(vector = 384)]` fixes it in the struct. `Option` fields become nullable columns. Rust field types are automatically mapped to Arrow types: | Rust Type | `ColumnType` | Arrow Type | |-----------|--------------|------------| | `bool` | `Bool` | `bool` | | `i16`, `i32`, `i64` | `Int16`, `Int32`, `Int64` | `int16`, `int32`, `int64` | | `f32`, `f64` | `Float32`, `Float64` | `float32`, `float64` | | `String`, `Uuid`, dates and times | `Text` | `string` | | `Vec` | `Binary` | `binary` | | other `Vec`, maps, nested structs, `Decimal`, `Duration`, `#[coco(json)]` | `Json` | `string` (JSON encoded) | | `Vec` with `#[coco(vector)]` | `Vector(dim)` | `fixed_size_list` | | `Vec` with `#[coco(vector, half)]` | `HalfVector(dim)` | `fixed_size_list` | `ColumnType` also has `Int8`, `UInt8`, `UInt16`, `UInt32`, `UInt64` and `Float16`, for `with_column_type` and explicit columns. A `#[coco(type = "…")]` attribute is an error for LanceDB: use `with_column_type` with a `ColumnType`. #### Column type overrides Use `with_column_type` to store a column with another `ColumnType`: ```rust use cocoindex::connectors::lancedb::{ColumnType, TableSchema}; let schema = TableSchema::from_row::(["id"])? .with_column_type("id", ColumnType::Int32)? .with_column_type("value", ColumnType::Float32)?; ``` Or declare the field as `i32` / `f32` in the struct. There is no per-column encoder. A Rust row is encoded by its `Serialize` impl: to convert a value before it is written, put `#[serde(serialize_with = "…")]` on the field. #### VectorSchemaProvider For a `#[coco(vector)]` field, `with_vector_schema` takes the vector dimension and element type from a `VectorSchemaProvider`: an embedder, or a `VectorSchema`. For an embedder held in the context, pass `ctx.get_key(&EMBEDDER)?`. An `f16` provider selects a `float16` list; `f32` and `f64` select `float32`. See [Vector Schema](/docs/rust/common_resources/vector_schema/#vectorschemaprovider) for details. ```rust let schema = TableSchema::from_row::(["id"])? .with_vector_schema("embedding", ctx.get_key(&EMBEDDER)?) .await?; ``` ### Table schema: explicit column definitions Define columns directly using `ColumnDef`: ```rust impl TableSchema { pub fn new( columns: impl IntoIterator, ColumnDef)>, primary_key: impl IntoIterator>, ) -> Result } impl ColumnDef { pub fn new(col_type: ColumnType) -> Self pub fn nullable(self) -> Self } ``` `ColumnDef::new(col_type)` is not nullable; `.nullable()` opts in. `TableSchema::new` returns an error for an empty primary key, a primary key column that is not in `columns`, or a duplicate column. **Example:** ```rust use cocoindex::connectors::lancedb::{ColumnDef, ColumnType, TableSchema}; TableSchema::new( [ ("id", ColumnDef::new(ColumnType::Int64)), ("text", ColumnDef::new(ColumnType::Text)), ("summary", ColumnDef::new(ColumnType::Text).nullable()), ("embedding", ColumnDef::new(ColumnType::Vector(3))), ], ["id"], )? ``` ### Example ```rust use std::path::PathBuf; use cocoindex::connectors::lancedb::{self, LanceDatabase, TableSchema}; use cocoindex::prelude::*; const EMBED_DIM: usize = 384; const TABLE: &str = "doc_embeddings"; cocoindex::context_key!(static DB: LanceDatabase); #[derive(Clone, Serialize, Deserialize, SchemaFields)] struct DocEmbedding { id: i64, filename: String, chunk_start: i64, chunk_end: i64, text: String, #[coco(vector)] embedding: Vec, } #[cocoindex::function] async fn process_file(ctx: &Ctx, file: FileEntry) -> Result> { // chunk and embed the file ... } fn doc_embedding_schema() -> Result { TableSchema::from_row::(["id"])?.with_vector_dim("embedding", EMBED_DIM) } async fn app_main(ctx: Ctx, sourcedir: PathBuf) -> Result<()> { let table = lancedb::mount_table_target(&ctx, &DB, TABLE, doc_embedding_schema()?).await?; let files = walk_items(&sourcedir, &["**/*.md"])?; let rows_by_file = use_mount_each!(files, |file| process_file(ctx, file)).await?; for rows in &rows_by_file { for row in rows { table.declare_row(&ctx, row)?; } } Ok(()) } ``` See `examples/rust/text_embedding_lancedb` for the full program, including the `lancedb::vector_search` query.