# Sentence Transformers embeddings > **CocoIndex v1, Rust SDK (experimental).** This page documents the CocoIndex **v1** Rust SDK (the `cocoindex` crate). The Rust SDK is experimental: any release may change or remove its API, and it is not yet published on crates.io. The Python SDK pages are at https://cocoindex.io/docs/. > > Source: https://cocoindex.io/docs/rust/ops/sentence_transformers/ · Rust docs index: https://cocoindex.io/docs/rust/llms.txt · Rust SDK tour: https://github.com/cocoindex-io/cocoindex/blob/main/rust/sdk/SHOWCASE.md · Rust examples: https://github.com/cocoindex-io/cocoindex/tree/main/examples/rust The `cocoindex::ops::sentence_transformers` module runs [sentence-transformers](https://www.sbert.net/) models locally for text embeddings. It runs their ONNX exports through [`fastembed`](https://github.com/Anush008/fastembed-rs), with no Python runtime. ```rust use cocoindex::ops::sentence_transformers::SentenceTransformerEmbedder; ``` **Note — Cargo feature** The module is behind the `fastembed` feature of the `cocoindex` crate: ```toml cocoindex = { version = "...", features = ["fastembed"] } ``` ## Overview The `SentenceTransformerEmbedder` struct wraps a locally loaded `fastembed` model. It: - Implements `VectorSchemaProvider` to supply vector schema information to CocoIndex connectors - Implements the [`Embedder`](/docs/rust/common_resources/data_types/#embedder) trait, so it plugs into [entity resolution](/docs/rust/ops/entity_resolution/) directly - Provides an `embed(&ctx, text)` method, memoized and batched by the engine - Returns each embedding as a `Vec` It runs the model on the ONNX Runtime execution providers compiled into the binary (CPU by default), on a blocking thread. The Rust embedder is built on `fastembed`, not on the `sentence-transformers` library, so its options differ. It has no `prompt_name` and no `normalize_embeddings`: `fastembed` has no prompt registry and always L2-normalizes its output. It has no `device` either: [`load_with`](#execution-providers) takes ONNX Runtime execution providers instead. ## Basic usage ### Creating an embedder ```rust use cocoindex::ops::sentence_transformers::SentenceTransformerEmbedder; // Load a pre-trained model; this downloads and initializes the ONNX model. let embedder = SentenceTransformerEmbedder::load("sentence-transformers/all-MiniLM-L6-v2").await?; ``` `load` is async and returns a `Result`, because it downloads and initializes the model. `embedder.dimension()` returns the model's dimension and `embedder.model_name()` the name it was loaded with. The embedder is cheap to clone. ### Embedding text The `embed(&ctx, text)` method converts text into a `Vec`: ```rust // In a CocoIndex function: memoized per text, batched across concurrent calls. let embedding = embedder.embed(&ctx, "Hello, world!").await?; // Outside an update (a search query): one raw batch, no memoization. let query_vec = embedder.embed_batch(vec![query.to_string()]).await?.remove(0); ``` Outside an update there is no `Ctx`: use `embed_batch(texts)`, which embeds one batch directly. **Note — Built-in memoization** `embed(&ctx, text)` is memoized: inside a processing component, each distinct text's vector is cached across that component's runs, so a caller that re-executes doesn't recompute unchanged text. The tradeoff is model compute vs. cache storage (one full vector per distinct text). It also batches: concurrent cache misses in one update are coalesced into batches of up to 64 texts. The memo key holds the model name and dimension, so a different model invalidates the cached vectors. Mind the [memoization placement rule](/docs/rust/programming_guide/function/#when-to-memoize) if your components are keyed by the embedded content. ### Declaring the vector column The `SentenceTransformerEmbedder` implements [`VectorSchemaProvider`](/docs/rust/common_resources/vector_schema/#vectorschemaprovider), so a connector reads the vector dimension and element type from it when it creates a table. The row struct marks the column with `#[coco(vector)]`: ```rust use cocoindex::prelude::*; #[derive(Clone, Serialize, Deserialize, SchemaFields)] struct DocEmbedding { id: i64, filename: String, chunk_start: i32, chunk_end: i32, text: String, #[coco(vector)] embedding: Vec, } ``` A row struct cannot name the embedder in a field attribute: the embedder supplies the dimension when the table is mounted, below. When you mount the table, pass the embedder to `with_vector_schema`, and the connector takes the vector column's dimension and element type from it. For example, with Postgres: ```rust use cocoindex::connectors::postgres; // The embedder reports the dimension of the `embedding` column. let table = postgres::mount_table_target( &ctx, &DB, "doc_embeddings", postgres::TableSchema::from_row::(["id"])? .with_vector_schema("embedding", ctx.get_key(&EMBEDDER)?) .await?, Some("my_schema"), ) .await?; ``` `TableSchema::from_row::(primary_key)` builds the schema from the row struct; `with_vector_schema(field, &embedder)` resolves the `#[coco(vector)]` column from any `VectorSchemaProvider`. See [Vector schema annotations](/docs/rust/common_resources/vector_schema/). The connector automatically creates the appropriate `vector(384)` column. See the [Connectors](/docs/rust/connectors/postgres/) docs for other supported backends (LanceDB, Qdrant, SQLite). ## Example: text embedding pipeline Here's a complete example of a text embedding pipeline (based on the [text_embedding example](https://github.com/cocoindex-io/cocoindex/tree/main/examples/rust/text_embedding)): ```rust use std::path::PathBuf; use std::time::Duration; use cocoindex::connectors::postgres; use cocoindex::ops::sentence_transformers::SentenceTransformerEmbedder; use cocoindex::ops::text::{RecursiveChunkConfig, RecursiveSplitter}; use cocoindex::prelude::*; const EMBED_MODEL: &str = "sentence-transformers/all-MiniLM-L6-v2"; const PG_SCHEMA: &str = "coco_examples"; const TABLE: &str = "doc_embeddings"; const CHUNK_SIZE: usize = 2000; const CHUNK_OVERLAP: usize = 500; cocoindex::context_key!(static DB: postgres::Database); cocoindex::context_key!( static EMBEDDER: SentenceTransformerEmbedder, detect_change ); #[derive(Clone, Serialize, Deserialize, SchemaFields)] struct DocEmbedding { id: i64, filename: String, chunk_start: i32, chunk_end: i32, text: String, #[coco(vector)] embedding: Vec, } /// Chunk and embed one file. Memoized on the file, so an unchanged file /// returns its rows without re-chunking. #[cocoindex::function(memo)] async fn process_file(ctx: &Ctx, file: &FileEntry) -> Result> { let filename = file.key(); let text = file.content_str()?; let splitter = RecursiveSplitter::new()?; let chunks = splitter.split_with( &text, RecursiveChunkConfig { chunk_size: CHUNK_SIZE, min_chunk_size: None, chunk_overlap: Some(CHUNK_OVERLAP), language: Some("markdown".to_string()), }, ); if chunks.is_empty() { return Ok(Vec::new()); } let texts: Vec = chunks.iter().map(|c| c.text(&text).to_string()).collect(); let embedder = ctx.get_key(&EMBEDDER)?.clone(); let embedding_ctx = ctx.clone(); let embeddings = ctx .map(texts.clone(), move |chunk_text| { let embedder = embedder.clone(); let ctx = embedding_ctx.clone(); async move { embedder.embed(&ctx, chunk_text).await } }) .await?; let mut id_gen = IdGenerator::new(); let mut rows = Vec::with_capacity(texts.len()); for ((chunk, chunk_text), embedding) in chunks.iter().zip(texts).zip(embeddings) { let id = id_gen.next_id(&ctx, &chunk_text).await?; let id = i64::try_from(id).map_err(|_| Error::engine("generated id does not fit in BIGINT"))?; rows.push(DocEmbedding { id, filename: filename.clone(), chunk_start: chunk.start.char_offset as i32, chunk_end: chunk.end.char_offset as i32, text: chunk_text, embedding, }); } Ok(rows) } /// Environment setup: the state db, Postgres and the embedder. #[cocoindex::lifespan] async fn lifespan(builder: EnvironmentBuilder) -> Result { let db = postgres::Database::connect(&database_url()).await?; let pool = db.pool().clone(); let embedder = SentenceTransformerEmbedder::load(EMBED_MODEL).await?; Ok(builder .db_path(PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(".cocoindex_db")) .provide_key(&DB, db) .provide_key(&EMBEDDER, embedder) .on_shutdown(move || async move { pool.close().await })) } #[cocoindex::app(name = "TextEmbeddingRust")] async fn app_main(ctx: Ctx) -> Result<()> { index(ctx, source_dir()).await } async fn index(ctx: Ctx, sourcedir: PathBuf) -> Result<()> { // The embedder reports the dimension of the `embedding` column. let table = postgres::mount_table_target( &ctx, &DB, TABLE, postgres::TableSchema::from_row::(["id"])? .with_vector_schema("embedding", ctx.get_key(&EMBEDDER)?) .await?, Some(PG_SCHEMA), ) .await?; // One component per markdown file; each declares its own rows. In live // mode (`cocoindex update -L`) the walker keeps watching the directory. let files = walk_dir(sourcedir) .recursive(true) .path_matcher(PatternFilePathMatcher::new( ["**/*.md"], std::iter::empty::<&str>(), )?) .live() .poll_interval(Duration::from_secs(1)); ctx.mount_each_live(&"files", files, move |ctx, file: FileEntry| { let table = table.clone(); async move { for row in process_file(&ctx, &file).await? { table.declare_row(&ctx, &row)?; } Ok(()) } }) .await } ``` The full program, with the `database_url()` and `source_dir()` helpers, a `query` command and the `#[cocoindex::main]` entry point, is `examples/rust/text_embedding`. Run it with the [CLI](/docs/rust/cli/): `cocoindex update`, or `cocoindex update -L` to keep watching the directory. ## Configuration options ### Model selection `SentenceTransformerEmbedder::load(name_or_path)` accepts a name from the `fastembed` model registry (matched case-insensitively, and by the trailing model name, so `sentence-transformers/all-MiniLM-L6-v2` and `Xenova/all-MiniLM-L6-v2` resolve to the same model), a local directory holding a sentence-transformers ONNX export (`onnx/model.onnx` or `model.onnx` plus the tokenizer files), or the Hugging Face Hub repository of such an export. `HF_HOME` and `HF_ENDPOINT` apply to the download. A model with a pooling mode other than CLS or mean is rejected. Pooling follows the export's `1_Pooling/config.json` when it has one. ```rust let embedder = SentenceTransformerEmbedder::load("sentence-transformers/all-MiniLM-L6-v2").await?; ``` ### Normalization and prompts `fastembed` always L2-normalizes its output, so every embedding has unit length (suitable for cosine similarity). It has no prompt registry: for a model that expects a query or document prefix, prepend the prefix to the text yourself. ### Execution providers `SentenceTransformerEmbedder::load_with(name_or_path, execution_providers)` runs the model on the first of `execution_providers` that ONNX Runtime can start. A provider is registered when its `ort` feature is compiled into the binary, such as `cuda` for `CUDAExecutionProvider`; otherwise ONNX Runtime falls back to the CPU. `load(name_or_path)` is `load_with` with no providers. ```rust use cocoindex::ops::sentence_transformers::SentenceTransformerEmbedder; use ort::execution_providers::CUDAExecutionProvider; let embedder = SentenceTransformerEmbedder::load_with( "sentence-transformers/all-MiniLM-L6-v2", vec![CUDAExecutionProvider::default().build()], ) .await?; ``` Each batch runs under the GPU runner, so one inference runs at a time per GPU in the pool.