CocoIndex 1.1: the highlights since 1.0

CocoIndex builds fresh, structured context for AI from agent sessions, codebases, instant messages, meeting notes, and all the other unstructured data in an enterprise, and keeps it up to date with an incremental engine.

Since the v1.0 launch, most of our work has gone into running pipelines under production limits: time budgets, provider quotas, memory, and failures that stay local. We also made pipeline logic more flexible and runs cheaper across larger corpora.

The problem

Agents act on the context they read, and that context fails in four ways.

• Fragmented Code, docs, Slack, meeting notes, tickets, and now agent sessions live across a dozen systems, and 80–90% of it is unstructured.
• Moving fast About 1% of a 100 GB codebase changes every day, so context built this morning is stale by lunch.
• Growing exponentially Google says 75% of its new code is AI-written, code per developer tripled in a year, and teams run thousands of agent sessions a month.
• Intention evaporates A session ends and its reasoning is gone, so the next agent redoes the work or contradicts a decision someone made last week.

Data readiness has been the number-one blocker to AI adoption five years running.

CocoIndex v1 set out the approach: you declare the target states your agents should read, and an incremental engine keeps them in step with every source as it changes.

Overview

Engine memory per Postgres row fell from 1,449 B to 487 B, and moving 5,000 target states went from 243 s to 1–2 s. A bad row now fails only its own component, and every connector keys each object once. There are 10 new connectors, among them Snowflake, BigQuery, Azure Blob Storage, and Neo4j, plus syntax-aware code splitting for 11 more languages.

What you can run now

7 workloads that got easier since 1.0, each with the feature behind it.

• A time budget per document. Wrap a file's processing in coco.timeout(timedelta(seconds=30)) and every call beneath it (parse, LLM summary, embedding) inherits the deadline. A file that overruns fails on its own: it applies no writes, and the next update retries it, while every other file commits. A per-file budget stays meaningful as the source grows, where a whole-run limit does not. (deadlines)
• Multi-tenant live indexing. With each tenant as a live component under one max_inflight_components, a new tenant's first full index no longer makes another tenant's 3-item refresh wait behind its whole backlog. One RateLimiter per provider, shared through context, holds one quota across every tenant's components. (Slot lending and fair sharing, rate limiter)
• 1M-row tables per component. A component that declares 1M Postgres rows costs the engine about 0.5 GB of memory instead of 1.4 GB, and an update where most of those rows are unchanged runs no per-row Python at commit (for plain-data rows, outside full_reprocess and schema changes). (cheaper commits)
• A redeploy that doesn't re-embed. Mark client handles, loggers, and debug flags as not part of the memo key, and a new process reuses every embedding and LLM result whose inputs and code did not change. (memo keys)
• Dependency upgrades that re-run only the calls that used them. A dependency version used to go in a function's deps=, which covers every call. Recorded per call with LogicDep, a bump re-runs only the calls that recorded it. Parsers are the typical case: bumping the Python grammar re-runs the Python files and the directories above them, where before it re-parsed every file in every language; every other file is reused from memo. (logic dependencies)
• Preview in a pull-request check. cocoindex update --preview lists the adds, reprocesses, and deletes a change would make, runs on pipelines with nested targets, and writes no memos, target actions, or committed state (only ID counters advance), so it can't poison the next real update. (preview)
• Warehouse-native targets. Declare rows into Snowflake or BigQuery with the same incremental reconciliation as a Postgres table: only changed rows are upserted, and rows for deleted sources are removed. (Connectors)

Upgrading from 1.0.x

Run under real limits

Each limit is set in one place, and the engine enforces it.

One deadline for a whole call tree

coco.timeout() (v1.0.17) gives a call tree one absolute deadline, so you don’t have to pass a timeout through each call. @coco.fn calls, coco.map() item tasks, and coco.use_mount() children inside the with block inherit it. This example sets a 30-second budget for processing a file:

@coco.fn(memo=True)
async def process_file(file: FileLike, target: postgres.TableTarget[DocRow]) -> None:
    # 30-second budget for this file; every call below checks it.
    with coco.timeout(timedelta(seconds=30)):
        parsed = await coco.use_mount(coco.component_subpath("parse"), parse, file)
        summaries = await coco.map(summarize, parsed.sections)
        summary = "\n".join(summaries)
        vector = await embed(summary)
    target.declare_row(
        row=DocRow(name=out_name(file), summary=summary, embedding=vector)
    )

The deadline is cooperative. CocoIndex checks it at its own checkpoints and raises coco.DeadlineExceededError, a subclass of TimeoutError. A run that times out applies no target writes and memoizes nothing, so the next update retries it. The deadline does not extend to work the caller doesn’t wait for (mount() and mount_each() children and live components). Batched function bodies and target writes do not inherit it either. The built-in retry helpers treat their own limit as a nested timeout, so the shorter budget wins.

One quota shared by every component

RateLimiter (v1.0.8) is a token bucket that components share. Every component that acquires from the same instance draws on one budget, and concurrent callers are served in arrival order. It replaces hand-rolled sleep loops at each call site.

In 1.1.0 a batched function takes two more parameters to control requests. rate_limiter takes one token per batch instead of one per call, which fits APIs where each batch is one request. max_in_flight caps how many batches run at once. A batch that isn’t full yet can start as soon as a slot frees, instead of waiting until no batch is running. This example limits batches to four concurrent requests:

from cocoindex.resources.rate_limit import RateLimiter

limiter = RateLimiter(max_rows_per_second=1.0)

@coco.fn.as_async(
    batching=True,
    max_batch_size=100,
    max_in_flight=4,
    rate_limiter=limiter,
)
async def fetch(ids: list[str]) -> list[Item]:
    return await api.fetch_many(ids)  # one request, one token

The LiteLLM embedder and transcriber also gained limits of their own in 1.0.22: timeout bounds one request end to end, retries included, and max_inflight_requests caps the requests one embedder keeps open against its backend.

A failed batch splits until the bad item stands alone

The engine can split and retry a failed batch so that one bad input does not fail the rest. A batched function body can raise coco.RetryWithSmallerBatch (v1.0.17), and the engine halves the batch and retries the halves, down to single items. This helps with a provider’s per-request token cap, a rejected input, or a GPU running out of memory. At size one the engine raises the original error for that item only. LiteLLMEmbedder and SentenceTransformerEmbedder already do this, for example when voyage-code-3 rejects a batch over 120k tokens, or on GPU out-of-memory.

Since 1.0.23 the engine applies the same rule to the batches it builds for target writes. A sink call is one connector write to the target. The engine merges writes from components that finish while a sink call is in flight into the next call. A failure in that call used to fail every component in it. In one case, one bad row in a sink call of 59 rows rolled back the writes of 58 other components. A failed sink batch is now split along component boundaries and the halves are re-applied, so the error reaches only the component that caused it. Connector code doesn’t change, and the happy path is still one sink call.

A concurrency cap that can’t deadlock a hierarchy

max_inflight_components (default 1024) caps how many processing components execute at once, and since 1.0.26 that cap holds across a hierarchy of any depth. Before, a parent waiting on its children held a slot, so the engine released it when the parent mounted its first child to avoid deadlock, which left everything after that point unbounded. One indexer ran out of memory with about 15,000 file components in flight under a limit of 256.

Now a waiting parent keeps its slot and lends it to one child at a time. A blocked parent always has one child that can run, so the hierarchy completes even with a limit of 1. In 1.1.0 the engine also shares the pool fairly between independent updates, so a small live refresh no longer waits behind a new component’s first full index.

GPU fractions and whole-card reservations

Since 1.0.15, a function run with runner=coco.GPU takes a GPU from a pool for the duration of the call. coco.GPU(0.5) takes half of one, so two such calls share a card. In 1.0.25 CocoIndex rewrote the pool in Rust and exposed it as coco.GPUPool. Requests are served in arrival order, so a stream of fractional requests can’t starve a whole-GPU one, and acquire_full(n) reserves n free GPUs at once, for example for a model sharded across cards.

More state inside the pipeline

More of what a pipeline remembers now lives in the engine, under the same reconciliation rules as the rest of the app.

Recompute only when an input that matters changes

Avoid repeating paid embedding or LLM calls when only a client, logger, config object, or debug flag changes. @coco.fn takes a per-argument memo_key (v1.0.3) to control the memo cache key. A callable transforms an argument before fingerprinting, and None leaves the argument out of the key. This example keys entry by name and version and excludes extra:

@coco.fn(memo=True, memo_key={"entry": lambda e: (e.name, e.version), "extra": None})
def transform(entry: SourceDataEntry, extra: str) -> str:
    ...

State that survives updates

coco.use_state() (v1.0.8) lets a component keep typed state across updates in the engine’s LMDB state store. Running totals, seen-entity sets, watermarks, and stateful deduplication used to need a Redis or Postgres sidecar.

A bridge between a producing and a consuming stage

LiveMap (v1.0.8) is an in-memory keyed collection that connects a producing and a consuming stage of the same live run. Producing components declare entries into it as target states. The consuming side reads it as a live source through mount_each(), which mounts one live component per entry and drops it when its producer stops declaring the key.

Polling with reconciled deletes

coco.auto_refresh() (v1.0.7) re-runs a processor on an interval as a live component. Each cycle reconciles target states against the previous one. When sync_users stops declaring a row, CocoIndex deletes it. Cycle failures go through one error-handling channel. This example refreshes users every five minutes:

await coco.mount(
    coco.auto_refresh(sync_users, interval=timedelta(minutes=5)),
    db, target,
)

Invalidate only the calls that used a changed dependency

coco.LogicDep (v1.1.0) lets a call record only the logic dependencies it uses. @coco.fn fingerprints a function’s logic, so editing the function invalidates the cached results that depend on it. Some dependencies are decided per call: a tree walk’s visit_file depends on the parser for that file’s language only. coco.LogicDep declares such a value at module import, and coco.use_logic_dep() records it for the current call. This example records the parser dependency for each file:

PARSER_DEPS = {
    name: coco.LogicDep(("parser", name, version))
    for name, version in PARSER_VERSIONS.items()
}

@coco.fn(logic_tracking="self", version=1)
async def visit_file(file: File) -> None:
    language = detect_language(file.path)
    if (dep := PARSER_DEPS.get(language)) is not None:
        coco.use_logic_dep(dep)  # this call's logic also depends on that parser
    ...

When the Python parser’s LogicDep gets a new version, only the Python files and the directories above them re-run. Every other subtree is reused from memo.

A related addition for library authors, coco.logic_propagated_to_callers() (1.0.26), carries a user callback’s logic through a library layer whose fingerprint excludes its callees (logic_tracking="self"), so an edit to the callback still invalidates the library’s memos.

See what an update will do, and what it did

Review an update before it writes anything, for example in a pull-request check, and see every target state with the component that wrote it.

A preview that writes nothing

Preview an update before writing to its targets with cocoindex update --preview. Preview arrived in 1.0.8 for target states with nothing nested under them. In 1.0.26 it reached nested states, such as rows under a Postgres table or files under a directory target, so it now runs on most real pipelines. A preview reads cached results without changing them and writes no committed state. This makes it safe to run at any time: it can’t make the next real update skip work. Run a preview for an app as follows:

cocoindex update --preview path/to/app.py

Every target state and the component that owns it

Inspect what CocoIndex wrote and which component owns it with cocoindex show (v1.0.12 to v1.0.18). CocoIndex records every declared target state together with its owner. --target-states lists every tracked target state with its owner and marks entries nothing owns anymore as [dangling]. With --db and --app-name it reads a database directly, without importing the app or installing its dependencies. This command lists target states as a tree:

$ cocoindex show --db ./cocoindex.db --app-name FilesTransform --target-states --tree
Target states:
- @cocoindex/localfs
  - [null,"output_html/data__bizarre_animals.md.html"]
    owner:/@process_file/"bizarre_animals.md"
  - [null,"output_html/data__chuck_norris.md.html"]
    owner:/@process_file/"chuck_norris.md"

Stats by tenant, project, or folder

coco.stats_group(title) (v1.0.7) breaks stats down by data slice, such as a tenant, a project, or a source folder, in addition to the existing breakdown by processor. Everything mounted inside the block reports separately under title, so you can see which tenant is growing and which one caused a reprocess storm. This example reports stats for each tenant:

for tenant in tenants:
    with coco.stats_group(f"tenant:{tenant.id}", report_to_stdout=True):
        files = localfs.walk_dir(tenant.docs_dir, ...)
        await coco.mount_each(process_doc, files.items(), target)

Commits that cost less, especially when little changed

A component can now declare 1M rows, and an update where most of them are unchanged runs no per-row Python at commit.

• Batched target writes across components (v1.0.15). The engine batches rows that many components declare into one target and applies them together. Before, a 50-row ingest into LanceDB made about 50 commits and 50 table versions.
• Large target updates (v1.0.25). The engine holds declared rows encoded instead of as Python objects and computes fingerprints natively. The SQL, warehouse, LanceDB, and graph table targets skip reconcile() for unchanged rows. Postgres upserts of scalar-only tables use unnest, 2,000 rows per statement. Tables with pgvector, enum, or domain columns stay on VALUES. Composite-key deletes match against one key list instead of an OR chain.
• Moving target states between components (v1.1.0). Commit time was quadratic in the number of moved targets. Each previous owner’s tracking record (what the engine stored about that owner’s target states) is now decoded and encoded once per commit.
• LMDB (v1.0.10, v1.0.15). The engine prefetches cached results and state in one read transaction per cycle. The state store grows its map automatically when it fills, so COCOINDEX_LMDB_MAP_SIZE needs no manual tuning.

Failures stay local, keys stay stable

One bad document no longer takes its neighbours’ writes down with it, and connector keys no longer collide or drift between runs. S3 overwrites within the same second now resolve correctly.

• Connector keys (v1.1.0). CocoIndex matches every source item and target state with its previous run by key. An audit of how every Python connector builds its keys closed several edge cases. These included two objects sharing one key (Google Drive files with the same name, S3 keys that differ only in a doubled /), one object’s key changing between runs (a case-only rename on macOS), and deletes that missed their object.
• Long updates on Linux (v1.0.24). Under sustained load, a long App.update() could hang permanently on LMDB’s writer lock. Each write transaction now stays on one OS thread from open to commit.
• Shared write transactions (v1.0.25). Concurrent commits share one LMDB write transaction. A connector rejecting one component’s declared value used to fail the commit of every component batched with it, including components of other apps in the same Environment. Only the failing component gets the error; the others commit in a transaction of their own.
• Memoization (v1.0.23 to v1.1.0). Two overlapping runs of a memoized component with the same key now execute once. A re-run notebook cell or importlib.reload no longer drops the memos of the surviving function. Memo fingerprints depend only on values, not on object identity.
• Schema changes under full_reprocess (v1.0.20, v1.0.21). The table and graph targets did not apply schema changes that arrived in a full_reprocess run. Later updates reported success while the target drifted. Fixed in Postgres, SQLite, Doris, Snowflake, BigQuery, SurrealDB, LanceDB, Neo4j, and FalkorDB.
• Security. The Postgres and SQLite connectors validate table and column identifiers before they put them into SQL (v1.0.4). litellm 1.107.0, a release published to PyPI with a credential stealer, is excluded from every litellm requirement (v1.0.26).

Connectors

10 connectors were added to v1.

Changes to existing connectors:

• LanceDB: background table compaction and new columns added in place (v1.0.7), then vector and full-text index management (v1.0.11).
• Qdrant: native sparse vectors for hybrid search (v1.0.16).
• Local file system: in live mode, a periodic rescan catches events the OS watcher missed (v1.0.15).
• SurrealDB: an overhaul with declare_fields, flexible record IDs, and typed parameters (v1.0.25). The connector raises statement errors instead of swallowing them.
• Neo4j and FalkorDB: node_target and relationship_target APIs replace the table and relation APIs (v1.1.0). The old APIs remain available but are deprecated.

Operations, SDKs, and examples

• Code splitting in 11 more languages (v1.0.3 to v1.0.26). RecursiveSplitter gained tree-sitter grammars for Svelte, Vue, Julia, Elm, Astro, Bash, CMake, HCL, Dart, Lua, and Nix.
• Speech-to-text (v1.0.1). LiteLLMTranscriber transcribes audio inside a pipeline through any LiteLLM-backed provider. In 1.0.22 transcription requests gained retries.
• Entity resolution in parallel (v1.0.7). Entity resolution resolves independent components of the candidate graph in parallel.
• Rust SDK. The core SDK is separate from the Python bindings (v1.0.5). In 1.0.25, #[cocoindex::function(memo, batching)], context_key!, and SchemaFields made batching, context keys, and connector schemas declarative. Start with the Rust quickstart.
• 17 example walkthroughs (v1.0.13), written so that a coding agent can follow them step by step.
• Meeting notes to a knowledge graph now runs on Neo4j, FalkorDB, and SurrealDB.
• Slides to speech narrates locally on the CPU without a TTS API key, and re-runs only the slides you edit.

Thanks

The four changelogs thank 50 contributors by name, more than 30 of them first-time contributors. The contributors behind the features this post leads with:

• Per-argument memo keys: @MrAnayDongre
• coco.use_state(): @hakuuww
• coco.timeout(): @tomz-alt
• Target-state inspection in cocoindex show: @junzh0u
• The Snowflake, BigQuery, and Azure Blob connectors: @sdhilip200
• The GPU pool rewrite: @YaxinCheng
• The Google Drive fix that started the connector key audit: @Sujit-1509

Each changelog thanks everyone who contributed to it. For every change, see the four changelogs linked above or the GitHub releases. If CocoIndex is useful to you, consider starring the repository.

About the author

George He Cofounder & CTO of CocoIndex · @georgehe0

Maintainer of CocoIndex, Ex-Google Infra Lead. Writes about incremental data infrastructure, Rust internals, and the engineering decisions behind the engine.