At Unity, we use CocoIndex to incrementally index critical unstructured assets. With it, we've managed to dramatically reduce unnecessary data processing computation and remote LLM embedding calls.
CocoIndex is our “Kubernetes moment” – it empowers us to index and operate data with exceptional efficiency, keeping everything always current and context-ready for AI.
Continuously fresh context for AI agents
An open-source Python framework that turns codebases, PR reviews, logs, sessions, meeting notes and instant messages into live context for your agents. Only what changed (Δ) is reprocessed.
pip install -U cocoindex
raw · incoming
Source
code · PRs · logs · messages
- repo/
- api.ts Δ
- #eng
- notes.md
- PR #482
Δ only
derived · always fresh
View
vectors · graph · tables
CocoIndex keeps this in sync
reads
reads · decides · writes
Agent
code · plans · decisions
› fix getUser null check
reads view · 4 call sites
› opens PR #483
agent output becomes the next run's source
Any source in. Any target out.
Write the transformation in plain Python. The engine reads every source with change capture and keeps every target in sync.
sources
- Codebases
- Meeting notes
- Web APIs
- Files and object storage
- Databases
- Message queues
- Images and video
- Audio and transcripts
@coco.fn(memo=True)
def index(doc, target):
row = to_row(split(doc))
target.declare_row(row=row)cocoindex engine
targets
- Relational DB
- Data warehouse
- Vector DB
- Graph DB
- Message queue
- Feature storePlus
Persistent data pipeline control plane
- Live caching
Memoized functions skip inputs that haven't changed.
- Pipeline catalogPlus
A registry of every flow, so teams find and reuse each other’s pipelines.
- Version tracking
Every target row records the code version that produced it.
- Continuous learningPlus
The engine adapts as your data and code evolve.
- Lineage
Every target row traces back to its source and code version.
- Task scheduling
Independent sources and rows run in parallel.
- Metrics collectionPlus
Throughput, freshness and cost per flow.
- Failure managementPlus
Retries, back-off and dead letters.
Plus Included in CocoIndex Plus →
Write the transform. The engine keeps it fresh.
A CocoIndex flow is plain Python. Declare what the target should hold; when a source or the code changes, only the affected rows run again.
import cocoindex as coco
from cocoindex.connectors import postgres
@coco.fn(batching=True)
async def embed(texts: list[str]) -> list[list[float]]: ...
@coco.fn(memo=True)
async def process_file(
file: FileLike,
target: postgres.TableTarget[DocRow],
):
parsed = await parse(file)
summaries = await coco.map(summarize, parsed.sections)
summary = "\n".join(summaries)
vector = await embed(summary)
target.declare_row(row=DocRow(
name=file.name,
summary=summary,
embedding=vector,
))sourcedocs/
- intro.md
- api.md Δ
- guide.md
- faq.md
- install.md
- config.md
- notes.md
Δ api.md6 unchanged, served from memo
cocoindex engineprocess_file
- parse
- summarize
- embed
- declare_row
upsert 1 row
materialized viewpostgres · doc_rows
| name | summary | embedding |
|---|---|---|
| intro.md | 0.07 0.33 … | |
| api.md Δ | 0.12 −0.08 … | |
| guide.md | −0.22 0.05 … | |
| faq.md | 0.31 −0.26 … |
+ 3 rows unchanged
Source changed or code changed. Only the delta re-runs.
Two kinds of change reach an index: the source and your code. CocoIndex handles both without a rebuild, while agents keep reading.
Source data changed
The framework watches the source, computes the delta and reconciles the target, including rows whose source is gone.
Code changed
When the code changes, only the rows it touches run again, the schema evolves in place and the target stays online for the agents reading it.
CocoIndex Code: AST-based coding context that just works.
Call graphs, hierarchies, symbol tables and semantic indexes, all kept fresh as the repo changes.
your repo
- src/
- flow.pyΔ
- pipeline.py
- embed.py
- api/
- route.ts
- tests/
- t_embed.py
Δ
ast · flow.py
reads
coding agent
› where is embed() called?
3 callers · fresh index
src/flow.py:42
src/pipeline.py:18
tests/t_embed.py:7
›
AST-aware chunks
Functions and classes stay whole, so search returns complete units, not fragments.
Index and semantic search
Find code by meaning (functions, patterns, intent) instead of string matches.
Call graphs and blast radius
Know what a change touches before it ships. Trace every caller and callee.
Global view
See duplicates and architecture across the whole repo, not one file at a time.
Built for coding, code-review and security-review agents.
Built with CocoIndex.
Working starters. Clone, plug in your source, ship. Each one is a handful of files and a flow declaration.
Real-time codebase indexing
Keep an index of your repo in sync with every commit. Feed code-review and coding agents with structure, not raw text.
Meeting notes → knowledge graph
Extract people, topics, decisions and action items from notes into a live graph. Query them with your agent.
LLM Wiki
Summarize every repo in your org with an LLM, plus one rolled-up org summary. Refreshes on every push.
Describe the flow. Claude writes the pipeline.
Vibe-coding native: a working CocoIndex flow in about 10 minutes, kept fresh by the framework from then on.
Try the Claude skillyouIndex my /docs folder into Postgres.
claudewriting flow.py: chunk → embed → postgres target…
okflow.py written · running
logdocs/ → postgres.doc_chunks · in sync
What builders say.
Your agents deserve fresh context.
Start with one source and one target. The quickstart walks through it end to end.
Running at scale? CocoIndex Plus →