Built-in operations

Overview of CocoIndex's built-in operations: text chunking and splitting, local and hosted embeddings, audio transcription, and entity resolution, with links to each operation module's reference.

Version: v 1.0.14
Last reviewed: Nov 4, 2026

ExperimentalThe Rust SDK is experimental. It is outside CocoIndex's compatibility promise: any release, including a minor one, may change or remove its API. It is not yet published on crates.io.

CocoIndex ships a set of built-in operations under the cocoindex::ops module, plus cocoindex::entity_resolution. Each module is independently usable and composes with the rest of a pipeline.

use cocoindex::entity_resolution;
use cocoindex::ops::{api, sentence_transformers, text};

Each cocoindex::ops module sits behind a Cargo feature of the cocoindex crate, so the heavy dependencies (tree-sitter grammars, the ONNX runtime, an HTTP client) are only built when used. cocoindex::entity_resolution needs no feature.

Operation modules

Module	What it provides	Cargo feature
Text operations	Code language detection, regex-based `SeparatorSplitter`, and syntax-aware `RecursiveSplitter` (tree-sitter) returning position-tracked `Chunk`s. `cocoindex::ops::text`.	`text`
Sentence Transformers	Local text embeddings from sentence-transformers ONNX models through `fastembed`, memoized and batched, on the GPU runner. `cocoindex::ops::sentence_transformers`.	`fastembed`
Model APIs	Embeddings, audio transcription and JSON-mode chat (with typed `extract`) over any OpenAI-compatible HTTP API: `ApiEmbedder`, `ApiTranscriber`, `ApiChatClient`, `LlmPairResolver`. `cocoindex::ops::api`.	`embed_api`
Entity resolution	Deduplicate entity names via embedding similarity plus a pluggable LLM pair-resolver, with `Pinned` / `Preferred` canonical policies. `cocoindex::entity_resolution`.	none; `LlmPairResolver` needs `embed_api`

Most common: chunking code and text

The most-used built-in is RecursiveSplitter. It does syntax-aware splitting that respects language structure (functions, classes, blocks) via tree-sitter, falling back to separator-based splitting for unsupported languages.

use cocoindex::ops::text::{RecursiveChunkConfig, RecursiveSplitter};

let splitter = RecursiveSplitter::new()?;

let chunks = splitter.split_with(
    &text,
    RecursiveChunkConfig {
        chunk_size: 1000,
        min_chunk_size: Some(300),
        chunk_overlap: Some(300),
        language: Some("python".to_string()),
    },
);

It tries to keep each output chunk between min_chunk_size and chunk_size, splitting at the highest-level boundary that fits and descending to finer boundaries when a piece is still too large. See Text operations for the full parameter reference, supported languages, and custom-language configuration.

Embeddings

Two interchangeable embedding backends are available, both providing a VectorSchemaProvider so vector columns are configured automatically by connectors:

SentenceTransformerEmbedder runs locally through fastembed, no API key.
ApiEmbedder reaches any OpenAI-compatible endpoint.

Both also implement the Embedder trait, so either one plugs into entity resolution.