# Turbopuffer connector > **CocoIndex v1, Rust SDK (experimental).** This page documents the CocoIndex **v1** Rust SDK (the `cocoindex` crate). The Rust SDK is experimental: any release may change or remove its API, and it is not yet published on crates.io. The Python SDK pages are at https://cocoindex.io/docs/. > > Source: https://cocoindex.io/docs/rust/connectors/turbopuffer/ · Rust docs index: https://cocoindex.io/docs/rust/llms.txt · Rust SDK tour: https://github.com/cocoindex-io/cocoindex/blob/main/rust/sdk/SHOWCASE.md · Rust examples: https://github.com/cocoindex-io/cocoindex/tree/main/examples/rust The `turbopuffer` connector provides utilities for writing rows to [Turbopuffer](https://turbopuffer.com/) namespaces, with support for both single and named vectors. ```rust use cocoindex::connectors::turbopuffer::{ self, DistanceMetric, NamespaceSchema, TurbopufferConnection, }; ``` **Note — Dependencies** The module is compiled with the `turbopuffer` Cargo feature of the `cocoindex` crate. ## Connection setup Turbopuffer uses a single client object that owns the API key and region. Construct one using `TurbopufferConnection`: ```rust fn connect() -> Result { let api_key = std::env::var("TURBOPUFFER_API_KEY") .map_err(|_| Error::engine("TURBOPUFFER_API_KEY is required"))?; let region = std::env::var("TURBOPUFFER_REGION").unwrap_or_else(|_| "gcp-us-central1".into()); Ok(TurbopufferConnection::new(®ion, &api_key)) } ``` `TurbopufferConnection::new(region, api_key)` is synchronous: it builds an HTTP client for `https://{region}.turbopuffer.com`. `with_base_url(url)` points it elsewhere, for a mock server in tests. The connector speaks the Turbopuffer HTTP API itself and uses no Turbopuffer crate, so there is no client type from a Turbopuffer SDK to construct. ## As target The `turbopuffer` connector provides target state APIs for writing rows to namespaces. CocoIndex tracks what rows should exist and automatically handles upserts and deletions. Turbopuffer creates namespaces implicitly on the first write, so there is no separate "create namespace" step — but the connector still tracks namespace-level configuration (vector schema and distance metric) and clears the namespace if it must be rebuilt. ### Declaring target states #### Setting up a connection Create a `ContextKey` to identify your client, then provide it in your lifespan: **Note** The key name is load-bearing across runs — it's the stable identity CocoIndex uses to track managed namespaces. See [ContextKey as stable identity](/docs/rust/programming_guide/context/#contextkey-as-stable-identity) before renaming. ```rust use cocoindex::connectors::turbopuffer::TurbopufferConnection; cocoindex::context_key!(static DB: TurbopufferConnection); let conn = TurbopufferConnection::new(®ion, &api_key); // Provide it on the environment builder, or in a `#[cocoindex::lifespan]`: let builder = Environment::builder().provide_key(&DB, conn); ``` `context_key!(static DB: TurbopufferConnection)` declares the key; its name is the stable identity. #### Namespaces (parent state) Declares a namespace as a target state. Returns a `NamespaceTarget` for declaring rows. ```rust pub fn declare_namespace_target( ctx: &Ctx, conn: &ContextKey, namespace: impl Into, schema: NamespaceSchema, ) -> Result pub async fn mount_namespace_target( ctx: &Ctx, conn: &ContextKey, namespace: impl Into, schema: NamespaceSchema, ) -> Result ``` `managed_by` is set with `ManagedTargetOptions::user_managed()` on the `_with_options` variants (`declare_namespace_target_with_options`, `mount_namespace_target_with_options`). `declare_namespace_target` returns a handle whose rows resolve when the current component commits; `mount_namespace_target(...).await?` returns a handle that accepts rows at once. **Parameters:** - `ctx` — The current component's `Ctx`. - `conn` — A `ContextKey` identifying the connection to use. - `namespace` — Name of the namespace: ASCII letters, digits, `-`, `_` and `.`. - `schema` — Schema definition specifying vector configuration and distance metric (see [Namespace schema](#namespace-schema)). - `options` — `ManagedTargetOptions` on the `_with_options` variants; `ManagedTargetOptions::user_managed()` assumes the namespace exists. #### Rows (child states) Once a `NamespaceTarget` is resolved, declare rows to be upserted with `declare_row` or `declare_named_row`: ```rust impl NamespaceTarget { /// A row for an unnamed-vector namespace. pub fn declare_row( &self, ctx: &Ctx, id: impl Into, vector: Vec, attributes: Map, ) -> Result<()> /// A row for a named-vector namespace. pub fn declare_named_row( &self, ctx: &Ctx, id: impl Into, vectors: I, attributes: Map, ) -> Result<()> where I: IntoIterator)>, K: Into } pub enum RowId { Str(String), Int(u64), } ``` The id, the vector(s) and the attributes are separate arguments. The id is a `RowId`, built from a `String`, a `&str` or a `u64`; the attributes are a `serde_json::Map`. A vector with a NaN or infinite element is rejected. - `id` — Document id (string or unsigned integer). - `vector` / `vectors` — For an unnamed-vector schema, pass one `Vec` to `declare_row`. For a named-vectors schema, pass `(name, vector)` pairs to `declare_named_row`. Each vector must have its field's dimension and finite elements; otherwise the declare call returns an error. - `attributes` — Non-vector attributes (text, tags, metadata, etc.). Turbopuffer infers attribute types from the data. ### Namespace schema Define vector configuration and distance metric for a namespace using `NamespaceSchema`: ```rust impl NamespaceSchema { // One unnamed f32 vector under Turbopuffer's `vector` field pub fn new(vector_size: usize, distance: DistanceMetric) -> Self pub fn from_vector_schema(schema: VectorSchema, distance: DistanceMetric) -> Result pub async fn from_vector_provider( provider: &(impl VectorSchemaProvider + ?Sized), distance: DistanceMetric, ) -> Result // Named vectors pub fn named(vectors: I, distance: DistanceMetric) -> Result where I: IntoIterator, K: Into // Named vectors from a `#[derive(SchemaFields)]` row pub fn from_row(distance: DistanceMetric) -> Result pub fn with_vector_dim(self, field_name: &str, dim: usize) -> Result } pub enum DistanceMetric { CosineDistance, EuclideanSquared } ``` `from_row` turns each `#[coco(vector)]` field of a row struct into a named vector; the other fields stay schemaless attributes. `with_vector_dim` sets the dimension of one such field. The Rust schema has one constructor per shape instead of one `create` with a union argument, and every constructor takes the distance explicitly. **Parameters:** - The vector shape — `new`, `from_vector_schema` and `from_vector_provider` declare an unnamed vector (stored under turbopuffer's default `"vector"` field); `named` and `from_row` declare named vectors. - `distance` — Distance metric applied to all vector columns in the namespace. Turbopuffer applies a single distance metric per namespace. #### VectorDef Specifies a vector field's dimension and dtype: ```rust pub struct VectorDef { pub schema: VectorSchema, } impl VectorDef { pub fn new(schema: VectorSchema) -> Result pub fn f32(size: usize) -> Result pub async fn from_provider(provider: &(impl VectorSchemaProvider + ?Sized)) -> Result } ``` The `schema` field is an explicit `VectorSchema`. Its dtype controls turbopuffer's vector type: `VectorSchema::f32(n)` gives `[N]f32` and `VectorSchema::f16(n)` gives `[N]f16`. `VectorDef::from_provider(&embedder).await?` resolves the schema from a [`VectorSchemaProvider`](/docs/rust/common_resources/vector_schema/#vectorschemaprovider), typically the embedder. #### Single (unnamed) vector For namespaces with a single unnamed vector: ```rust let target = turbopuffer::mount_namespace_target( &ctx, &DB, &namespace, NamespaceSchema::new(3, DistanceMetric::CosineDistance), ) .await?; target.declare_row(&ctx, "1", vec![1.0, 0.0, 0.0], attrs("alpha"))?; ``` `NamespaceSchema::new(size, distance)` takes the dimension as a number; `NamespaceSchema::from_vector_provider(&embedder, distance).await?` takes it from an embedder. #### Named vectors Namespaces can have multiple named vector columns (turbopuffer supports up to two per namespace). The name `"id"` is reserved for the row id and cannot be used as a vector field name. ```rust let schema = NamespaceSchema::named( [ ("text", VectorDef::f32(3)?), ("image", VectorDef::f32(2)?), ], DistanceMetric::CosineDistance, )?; ``` Rows pass `(name, vector)` pairs to `declare_named_row`: ```rust target.declare_named_row( &ctx, r.id.clone(), [("vector", r.vector.clone())], attributes, )?; ``` ### Distance metrics Turbopuffer applies a single `distance_metric` per namespace. Supported values: - `DistanceMetric::CosineDistance` — Cosine distance. - `DistanceMetric::EuclideanSquared` — Squared Euclidean distance. Every constructor takes the metric explicitly; there is no default. ### Example ```rust use std::path::PathBuf; use cocoindex::connectors::turbopuffer::{ self, DistanceMetric, NamespaceSchema, TurbopufferConnection, }; use cocoindex::ops::sentence_transformers::SentenceTransformerEmbedder; use cocoindex::prelude::*; use serde_json::json; cocoindex::context_key!(static DB: TurbopufferConnection); cocoindex::context_key!( static EMBEDDER: SentenceTransformerEmbedder, detect_change ); #[derive(Clone, Serialize, Deserialize, SchemaFields)] struct RowData { id: String, #[coco(vector)] vector: Vec, filename: String, chunk_start: i64, chunk_end: i64, text: String, } #[cocoindex::function] async fn process_file(ctx: &Ctx, file: FileEntry) -> Result> { // chunk and embed the file ... } async fn app_main(ctx: Ctx, sourcedir: PathBuf, namespace: String) -> Result<()> { let vector_dim = ctx.get_key(&EMBEDDER)?.dimension(); let target = turbopuffer::mount_namespace_target( &ctx, &DB, &namespace, NamespaceSchema::from_row::(DistanceMetric::CosineDistance)? .with_vector_dim("vector", vector_dim)?, ) .await?; let files = walk_items(&sourcedir, &["**/*.md"])?; let rows_by_file = use_mount_each!(files, |file| process_file(ctx, file)).await?; for rows in &rows_by_file { for r in rows { let attributes = json!({ "filename": r.filename, "chunk_start": r.chunk_start, "chunk_end": r.chunk_end, "text": r.text, }) .as_object() .unwrap() .clone(); target.declare_named_row( &ctx, r.id.clone(), [("vector", r.vector.clone())], attributes, )?; } } Ok(()) } ``` See `examples/rust/text_embedding_turbopuffer` for the full program. ## Row IDs Turbopuffer rows are identified by a string or an unsigned integer: the id argument is `impl Into`, so a `String` or `&str` becomes `RowId::Str` and a `u64` becomes `RowId::Int`. UUIDs should be passed as strings. An integer id must fit in an `i64` (at most `i64::MAX`); a larger one is rejected when the row is declared. ## Attributes Row attributes are schemaless; turbopuffer infers attribute types from the values you write. Supported scalar types include `string`, `int`, `uint`, `float`, `bool`, `uuid`, and `datetime`, plus their array variants. See [Turbopuffer's schema reference](https://turbopuffer.com/docs/write#schema) for the full list. The attributes are a `serde_json::Map`. Reserved attribute names depend on the schema; putting any reserved name in the attributes makes `declare_row` / `declare_named_row` return an error: - `id` is always reserved — it's the row id. - For an unnamed-vector schema, `vector` is also reserved (it's the wire-level vector field). - For a named-vectors schema, each declared vector field name is reserved instead. ## Vector search The connector focuses on writing rows. For vector search, use the turbopuffer client directly or the connector's search helpers: ```rust // Unnamed vector let hits = turbopuffer::vector_search(&conn, &namespace, vec![1.0, 0.0, 0.0], 1).await?; // Named vector field let hits = turbopuffer::vector_search_by_field(conn, namespace, "vector", query_vec, TOP_K).await?; ``` The connector has two search helpers, `vector_search` and `vector_search_by_field`; each returns `Vec` with a `distance` and the row's `attributes`.