# CocoIndex overview

> **CocoIndex v1, Rust SDK (experimental).** This page documents the CocoIndex **v1** Rust SDK (the `cocoindex` crate). The Rust SDK is experimental: any release may change or remove its API, and it is not yet published on crates.io. The Python SDK pages are at https://cocoindex.io/docs/.
>
> Source: https://cocoindex.io/docs/rust/getting_started/overview/ · Rust docs index: https://cocoindex.io/docs/rust/llms.txt · Rust SDK tour: https://github.com/cocoindex-io/cocoindex/blob/main/rust/sdk/SHOWCASE.md · Rust examples: https://github.com/cocoindex-io/cocoindex/tree/main/examples/rust

CocoIndex is a framework for building data processing pipelines for AI workloads, with built-in incremental processing.

## Programming model

CocoIndex uses a *declarative*, state-driven programming model. You specify *what* your target should look like as a function of your source data — not *how* to incrementally update it. CocoIndex handles change detection and applies only the necessary updates automatically.

If you've used React, spreadsheets, or materialized views, this will feel familiar:
- **React**: declare UI as a function of state → React re-renders what changed
- **Spreadsheets**: declare formulas → cells recompute when inputs change
- **CocoIndex**: declare [target states](/docs/rust/programming_guide/target_state/) as a function of source → CocoIndex syncs what changed

## CocoIndex features

### Rust engine
CocoIndex executes pipelines with a Rust engine. The Rust SDK calls that engine directly, in your own binary, with no Python runtime.

### Rust transformations
Write transformations as ordinary async Rust functions with your existing crates.
Write batch-style code; CocoIndex tracks dependencies and applies incremental updates in batch and live mode.
Attribute macros add logic tracking, memoization and batching to a function; row types derive their schema.

### Incremental processing
CocoIndex tracks fine-grained dependencies and recomputes work affected by changes to source data or processing code.

### Lineage
Inspect processing steps, intermediate results, and execution paths to trace how the pipeline produced its output.

### Open integration model
Sources and targets use a standard, open interface.
Use Rust crates, models, and functions in your transformations.

### Controlled concurrency
Pipelines run work concurrently with configurable limits and batch requests to external services and models.

### Recovery
The engine retries transient failures and resumes from previous progress after interruptions.

### Managed state
CocoIndex manages change detection, cached results, target state reconciliation, execution concurrency, and backfills.

## Incremental data processing

CocoIndex stores state across runs to identify new or changed data.

What incremental processing means:
- Avoid unnecessarily recomputing work, based on multi-level change detection:
  - **Component level**: only reprocess source items with changes
  - **Function level**: within an item's processing, memoize expensive function calls and reuse when possible
  - **Target level**: apply minimum necessary changes (insertions, updates, deletions) to the target
- Support multiple mechanisms to capture source changes (CDC, poll-based) out of the box

Use the updated output for serving, training, or feature computation.

## The Rust SDK

The Rust SDK is the `cocoindex` crate. It has the same programming model as the Python SDK and runs on the same engine, with the same state database format:

- An **app** is an async function that receives a `&Ctx` and declares target states through it. `App::open` opens the app, and `app.run(...)` runs one update.
- **Functions** are `async fn`s marked with `#[cocoindex::function]`. `#[cocoindex::function(memo)]` memoizes one, and `#[cocoindex::function(batching)]` batches calls to it.
- **Processing components** are mounted with `mount!`, `use_mount!` and `mount_each!`, at stable component paths.
- **Resources**, such as a database connection or a model, are shared through typed context keys (`cocoindex::context_key!`) and read with `ctx.get_key(&KEY)`.
- **Connectors and operations** are Cargo features, so a crate builds only what it uses.
- The `cocoindex` **CLI** for Rust projects builds your crate and runs its apps.

The crate is not on crates.io yet; [Installation](/docs/rust/getting_started/installation/) shows how to depend on it from the repository.

## Next steps

- [Install CocoIndex](/docs/rust/getting_started/installation/) and follow the [Quickstart](/docs/getting_started/quickstart/) to build your first pipeline
- Read [Core Concepts](/docs/rust/programming_guide/core_concepts/) for the mental model behind CocoIndex