Declaring target state
Target states describe what should exist in an external system — files, rows, embeddings. Learn how to declare them through connector APIs, nest container targets, and rely on CocoIndex to sync minimal creates, updates, and deletes.
Read Core Concepts for target states and the declarative target state = transform(source state) model, with diagrams.
A target state represents what you want to exist in an external system. You declare target states in your code; CocoIndex keeps them in sync with your intent — creating, updating, or removing them as needed.
A target is the external system you write to — a directory, a database table, a vector store collection, etc. Targets are represented by objects like DirTarget and the connectors’ TableTarget types. A target state is what you want to exist in that target — a specific file, row, or embedding.
CocoIndex treats your declarations as the source of truth: if you stop declaring a target state, CocoIndex will remove it from the target.
Examples of target states:
- A file in a directory
- A row in a database table
- An embedding vector in a vector store
When your source data changes, CocoIndex compares the newly declared target states with those from the previous run and applies only the necessary changes.
Declaring target states
CocoIndex connectors provide targets with declare_* methods:
// Declare a file target state
target.declare_file(ctx, &output_name_for(&file), html.as_bytes())?;
// Declare a row target state
table.declare_row(ctx, &DocEmbedding { /* ... */ })?;Every declare_* method takes the component’s &Ctx first: the target state belongs to that component.
Where do targets come from?
Target states can be nested — a directory contains files, a table contains rows. The container itself is a target state you declare, and once it’s ready, you get a target to declare child target states within it.
Container target states (like a directory or table) are typically top-level — you can declare them directly. Child target states (like files or rows) require the container to be ready first.
Connectors provide convenience methods that mount the container and return a ready-to-use target in one step:
Example: writing a file to a directory
For simple cases where each processing component writes a single file, you can declare the file directly:
use cocoindex::resources::fs::{self, FileOptions};
// Declare a single file target state directly
fs::declare_file_with_options(
&ctx,
path,
b"hello",
FileOptions {
create_parent_dirs: true,
},
)?;fs::declare_file(&ctx, path, content) is the form with default options.
When you need a DirTarget to declare multiple files, use the connector’s convenience method:
// Mount a directory target, get a ready-to-use DirTarget
let target = DirTarget::mount(&ctx, &output_dir)?;
// Declare child target states (files)
target.declare_file(ctx, &output_name_for(&file), html.as_bytes())?;Example: writing a row to PostgreSQL
This example uses a ContextKey to reference the database connection — see Context for how keys are defined and provided.
use cocoindex::connectors::postgres;
// Define a ContextKey for the database connection (provided in the lifespan)
cocoindex::context_key!(static DB: postgres::Database);
// Mount a table target, get a ready-to-use TableTarget
let target_table = postgres::mount_table_target(
&ctx,
&DB,
TABLE,
postgres::TableSchema::from_row::<DocEmbedding>(["id"])?
.with_vector_schema("embedding", ctx.get_key(&EMBEDDER)?)
.await?,
Some(PG_SCHEMA),
)
.await?;
// Declare a child target state (a row)
target_table.declare_row(ctx, &DocEmbedding { /* ... */ })?;The row type derives SchemaFields, and TableSchema::from_row::<Row>(keys) builds the schema from it. See Connector rows in Serialization.
These convenience methods wrap mount_target(), which automatically derives the component path from the target’s globally unique key. See Processing Component for more on mounting APIs.
A target has no pending status: mount_table_target is async and returns the target once the container is ready, and DirTarget::mount returns a ready handle. A target handle is Clone, so one handle can be passed to every per-item component.
How CocoIndex syncs target states
Under the hood, CocoIndex compares your declared target states with the previous run and applies the minimal changes needed — the same create/update/delete sync the change-scenario diagrams in Core Concepts walk through, viewed per target state:
| Target State | CocoIndex’s Action | ||
|---|---|---|---|
| On first declaration | When declared differently | When no longer declared | |
| A database table | Create the table | Alter the table | Drop the table |
| A row in a database table | Insert the row | Update the row | Delete the row |
| A file in a directory | Create the file | Update the file | Delete the file |
| A user-managed message topic | Nothing (the topic is user-managed) | Nothing | Abandon the topic: stop producing to it; existing messages are left as-is |
| A message in a topic | Produce the message | Produce the message with its new value | Produce a deletion message (e.g. a tombstone) |
CocoIndex ensures a container exists before its contents are added, and syncs the contents individually only while the container is still declared. Once a container is no longer declared, its contents are handled by the container’s own removal, in one of two ways defined by the connector:
- Destroy — CocoIndex owns the container and removes it together with its contents: dropping a table takes its rows with it, removing a directory takes its files, dropping a graph node table takes every node carrying its label.
- Abandon — the container is user-managed (a Kafka or Iggy topic, or a table or collection declared with
ManagedTargetOptions::user_managed()). CocoIndex stops writing to it and leaves its contents untouched: no per-row deletes or per-message tombstones are issued.
App::drop_state() follows the same rule: it destroys CocoIndex-owned containers and abandons user-managed ones. To remove the contents of a user-managed container, drain it first by keeping the container declared with no contents for one update — see Kafka and Iggy for the recipe.
A target state’s identity is its key. CocoIndex doesn’t detect two different keys that reach the same physical object: two directories that overlap or alias through a symlink, two ContextKeys for one database, or two names or IDs that the backend treats as equal (for example, table names that differ only in case). CocoIndex syncs them as two target states, so syncing one can overwrite or delete the other. Keeping the objects behind different keys apart is your responsibility.
Inspecting stored target states
The state database records every tracked target state with the component that owns it. The CLI prints them with cocoindex show --target-states.
From code, cocoindex::inspect reads the same records from an App, or from an Environment and an app name with the _by_name functions:
use cocoindex::inspect;
use futures::TryStreamExt;
// Streams read in one transaction and buffer nothing.
let mut entries = std::pin::pin!(inspect::iter_target_states(&app).await);
while let Some(entry) = entries.try_next().await? {
println!("{} owner:{}", entry.readable_path, entry.owner_component_path);
}Each TargetStateEntry carries the target state’s readable path, its fingerprint path, and its owner’s component path. inspect::list_stable_paths and inspect::iter_stable_path_details list the components themselves, with the count of target states each one owns.
What happens when a target’s schema changes
When you change a container target state’s declaration (e.g., add a column to a table schema, change a primary key), CocoIndex detects the change and does its best to alter the target in place. If the change is too large to alter (e.g., changing primary keys), the target is dropped and recreated.
When a target is dropped and recreated, CocoIndex automatically reprocesses all affected components to backfill the data — you don’t need to manually trigger --full-reprocess. This is handled by the target connector’s child invalidation mechanism, which signals to CocoIndex whether the change is destructive (all children lost) or lossy (some data may be lost).
Generic target state APIs
For cases where connector-specific APIs don’t cover your needs, CocoIndex provides generic APIs:
declare_target_state()— declare a leaf target statedeclare_target_state_with_child()— declare a target state that provides child target states
These are exported from cocoindex and used internally by connectors. A leaf declaration reads declare_target_state(ctx, provider.target_state(key, value)). register_root_target_states_provider(ctx, name, handler) registers a TargetHandler and returns the provider; the tests in rust/sdk/cocoindex/tests/mount.rs show a minimal handler. For defining custom targets, see Custom Target States Connector.