Keep an AI agent's context in sync with live mode

Run a CocoIndex app in live mode so an agent's index follows every edit, add and delete as it happens, and re-embeds only what changed. Updated for v1.

FeatureUpdated 7 min read

Continuous updates

cocoindex update -L

  1. SourceChanged entries
  2. Your transformationscocoindex update -L
  3. TargetKept in sync

An AI agent that uses retrieval answers from its index. It searches the index for context, then writes code or a reply from what it finds. When the index lags behind the source, the agent works from an old copy of your code and docs, and nothing in the retrieved text tells it so.

Live mode keeps a CocoIndex app running after its first pass. It watches the source and applies each change to the index as it happens. The demo below asks one agent the same question against two indexes of the same repo, a few minutes after a teammate’s commit.

Same agent, same question, two indexes

Illustrative repo and answers
At 09:00 a teammate commits:
  1. 09:00 commit lands
  2. 09:01 commit pulled, live index updated
  3. 09:04 agent is asked
  4. 09:04 agent answers
client/auth.py
- def get_token(key: str) -> str:
+ def issue_token(key: str, *, scope: str) -> str:

User Write a script that calls our API with an auth token.

Index rebuilt nightlylast run 02:00
retrieved
  • client/auth.pydef get_token(key: str) -> str:indexed 7 h ago
agent
from client.auth import get_token
token = get_token(API_KEY)

ImportError: cannot import name 'get_token'

Index in live modecocoindex update -Lrows replaced
retrieved
  • client/auth.pydef issue_token(key: str, *, scope: str) -> str:indexed 3 min ago
agent
from client.auth import issue_token
token = issue_token(API_KEY, scope="read")

Runs against the current code

docs/api-keys.md
- # Authenticate with an API key
- Send your key in the X-API-Key header.

User How do I authenticate a request?

Index rebuilt nightlylast run 02:00
retrieved
  • docs/api-keys.md deleted from repoSend your key in the X-API-Key header.indexed 7 h ago
  • docs/tokens.mdSend a bearer token in Authorization.indexed 7 h ago
agent
Send your key in the X-API-Key header
(per docs/api-keys.md).

Cites a file that no longer exists

Index in live modecocoindex update -Lrows replaced
retrieved
  • docs/tokens.mdSend a bearer token in Authorization.indexed 2 d ago
agent
Send a bearer token in the Authorization
header (per docs/tokens.md).

The deleted guide is gone from the index

docs/rate-limits.md
+ # Rate limits
+ 600 requests per minute per token.

User What is the rate limit on the API?

Index rebuilt nightlylast run 02:00
retrieved
  • docs/tokens.mdTokens expire after 1 hour.indexed 7 h ago
agent
The docs do not state a rate limit.

The new doc is not in the index yet

Index in live modecocoindex update -Lrows replaced
retrieved
  • docs/rate-limits.md600 requests per minute per token.indexed 3 min ago
agent
600 requests per minute per token
(per docs/rate-limits.md).

Answers from the doc added at 09:00

The code below uses the CocoIndex v1 API and follows the code_embedding example on CocoIndex v1.1.1.

Why an agent needs a live index

  • Agents act on what they retrieveA stale chunk turns into code that calls a renamed function, or a reply that cites a deleted page. The agent cannot tell the chunk is old.
  • Agents read their own writesA coding agent edits a file, then searches the repo for the next step. The search has to return the edit it just made.
  • Deletes have to reach the indexA removed doc keeps being retrieved until its rows are deleted. In live mode, deleting the file deletes its rows.
  • Freshness cannot cost a full re-indexA full re-index embeds every file on each run. Live mode re-runs only the files that changed, and unchanged chunks keep their vectors.

A person who reads a stale wiki page can often notice the date. An agent gets a chunk of text and a similarity score. If the chunk is out of date, the mistake moves into the agent’s output, as in the demo above: an import that fails, or a header the API no longer accepts.

Catch-up runs exit, live runs keep watching

A CocoIndex app runs in one of two modes. The pipeline code is the same; you choose the mode when you start it.

  • Catch-up (cocoindex update main): the app scans the source, processes what changed since the last run, syncs the target and exits. Memoized functions skip unchanged work, so the processing cost follows what changed (the run still lists every item to find it). To pick up later changes, you run it again, often from a cron job.
  • Live (cocoindex update -L main): the app does the same catch-up pass, then keeps running. For sources that report their own changes, it reacts to each change without rescanning the source (see Three ways a source goes live).

Both modes cost less than a full rebuild. Live mode also removes the wait between a change and the next run. Set the interval and the repo size below and compare:

One working day, 12 edits, three ways to refresh

Model: run time of a scheduled job ignored
run every
repo size
Full rebuild on a scheduledrop the index, embed every file
average lag
–
worst lag
–
files processed
–
Catch-up run on a schedulecocoindex update main
average lag
–
worst lag
–
files processed
–
Live modecocoindex update -L main
average lag
–
worst lag
–
files processed
–

A shorter cron interval lowers the lag, but each scheduled run still has to list the whole source to find what changed. In live mode the source tells the app what changed, so a change does not wait for the next listing.

A change re-runs one component, not the app

cocoindex update -L

live mode

sourceSource entries
doc_adoc_b · modifieddoc_c · new
long-running
Your transformationsonly changed entries re-run
targetIndex, in sync
rows · doc_arows · doc_brows · doc_c

LiveMapView · scan, then watch

localfs.walk_dir(live=True): a full scan first, then file events.

LiveMapFeed · stream only

kafka.topic_as_map(): every upsert and delete arrives as a message.

coco.auto_refresh · poll

Re-runs a function on an interval; rows it stops declaring are deleted.

Live mode needs two things. The app must run with live=True (the -L flag), and a processing component (CocoIndex’s unit of work, usually one per source item) must react to changes. The usual way is to pass a live source, one that can report its changes, to coco.mount_each(). It then runs three phases:

  1. Scan current state. It lists every item and mounts one processing component per item, keyed by the item’s key (for files, the path).
  2. Ready. The initial scan is done and the target rows are synced. A parent can wait for this before it serves queries.
  3. Watch for changes. A new or modified item re-mounts its component. A deleted item removes its component, and CocoIndex deletes the rows that component declared.

Start the app below, wait for it to be ready, then change the folder:

Start it live, then change the folder

Illustrative: chunk counts made up, logic from the example
  1. 1Scan current state
  2. 2Ready
  3. 3Watch for changes
watching ./ for changes
source folder
  • client/auth.py
  • client/http.py
  • docs/tokens.md
  • docs/api-keys.md
  • README.md
  • docs/rate-limits.md
target table code_embeddings, one square per row
  • client/auth.py
  • client/http.py
  • docs/tokens.md
  • docs/api-keys.md
  • README.md
  • docs/rate-limits.md
    process_file runs
    0
    model calls
    0
    rows written
    0
    rows deleted
    0

    Two layers of memoization keep the cost of a change small:

    • process_file is memoized per file (@coco.fn(memo=True)). A file event re-runs only that file’s function. The other files are not read again.
    • process_chunk (in main.py) calls embed(), which is memoized on the model and the text. When an edited file is re-chunked, the chunks whose text did not change reuse their vectors, so only new text reaches the model.

    In a quick test with a 2-file folder and a transform with no model call, an edit reached the output about 0.1 s after the write. With an embedding model, add the time the model takes for the changed chunks.

    Make an indexing app live

    This is the indexing path of the code_embedding example: it walks a repo, splits each file along its syntax tree, embeds each chunk, and stores the vectors in Postgres with pgvector. The setup (the CodeEmbedding dataclass, the PG_DB and EMBEDDER context keys, and the lifespan that creates them) is in main.py.

    main.py
    @coco.fn(memo=True)
    async def process_file(
        file: FileLike,
        table: postgres.TableTarget[CodeEmbedding],
    ) -> None:
        text = await file.read_text()
        language = detect_code_language(filename=str(file.file_path.path.name))
        chunks = _splitter.split(
            text,
            chunk_size=1000,
            min_chunk_size=300,
            chunk_overlap=300,
            language=language,
        )
        id_gen = IdGenerator()
        await coco.map(process_chunk, chunks, file.file_path.path, id_gen, table)
    
    
    @coco.fn
    async def app_main(sourcedir: pathlib.Path) -> None:
        target_table = await postgres.mount_table_target(
            PG_DB,
            table_name=TABLE_NAME,
            table_schema=await postgres.TableSchema.from_class(
                CodeEmbedding,
                primary_key=["id"],
            ),
            pg_schema_name=PG_SCHEMA_NAME,
        )
        target_table.declare_vector_index(column="embedding")
    
        files = localfs.walk_dir(
            sourcedir,
            recursive=True,
            path_matcher=PatternFilePathMatcher(
                included_patterns=["**/*.py", "**/*.rs", "**/*.toml", "**/*.md", "**/*.mdx"],
                excluded_patterns=["**/.*", "**/target", "**/node_modules"],
            ),
            live=True,  # source supports live watch; pass -L to `cocoindex update` to actually run live
        )
        await coco.mount_each(process_file, files.items(), target_table)

    live=True on walk_dir is the only line that concerns live mode. With it, files.items() returns a LiveMapView, which mount_each() can scan and then watch. Without -L at run time, the same code does one catch-up pass and exits.

    Run it from the CLI

    sh
    cocoindex update main       # catch-up: scan, sync changes, exit
    cocoindex update -L main    # live: catch up, then keep watching for edits

    -L is short for --live. The live run keeps going until you stop it.

    Run it from Python

    python
    app.update_blocking(live=True)
    
    # or, inside async code
    handle = app.update(live=True)
    await handle.result()

    Serve the agent while the index stays live

    An agent usually queries the index from a server process. The image_search example starts the live update inside the FastAPI lifespan. It waits for the ready signal, so the first query sees a complete index, then leaves the update running in the background:

    api.py
    @asynccontextmanager
    async def lifespan(app: FastAPI) -> AsyncIterator[None]:
        async with coco.runtime():
            # (Qdrant client setup omitted.)
            # Start a live update; block startup until the initial sweep finishes so
            # the collection is queryable, then keep it running in the background.
            update_handle = pipeline.app.update(live=True)
            async for snap in update_handle.watch():
                if snap.status is coco.UpdateStatus.READY:
                    break
            update_task = asyncio.create_task(update_handle.result())
    
            try:
                yield
            finally:
                update_task.cancel()
                with contextlib.suppress(asyncio.CancelledError):
                    await update_task

    The agent’s search tool and the indexer share one process. Each file saved in the watched checkout, by the agent or by a pull of a teammate’s commit, goes through the watcher into the index. Searches see it once its changed chunks are embedded.

    Three ways a source goes live

    A source goes live in one of three ways, depending on whether it can report its own changes:

    MechanismHow changes arriveInitial scanDeletesIn v1
    LiveMapViewThe source is scanned once, then watchedYesReported by the watcherlocalfs.walk_dir(live=True); OCI Object Storage fed by bucket events
    LiveMapFeedA stream of upserts and deletes per keyNoKafka tombstones; for Iggy, an is_deletion predicate you passkafka.topic_as_map(), Apache Iggy
    coco.auto_refreshYour function re-runs on an intervalYes, each cycleRows the function stops declaringAny source: Google Drive, a Postgres table, a REST API

    LiveMapView: scan the source, then watch it

    Use this for sources that have a current state you can list, such as a directory. walk_dir(live=True) watches the folder for file events with watchdog, as in the example above. It also runs a safety rescan once an hour by default (rescan_interval=None turns it off). See the localfs connector docs.

    LiveMapFeed: consume a stream with no snapshot

    Use this for sources that only send changes. The Kafka connector treats a topic as a keyed map: each message is an upsert or a delete for its key.

    python
    from confluent_kafka.aio import AIOConsumer
    from cocoindex.connectors import kafka
    
    
    @coco.fn
    async def app_main() -> None:
        consumer = AIOConsumer({
            "bootstrap.servers": "localhost:9092",
            "group.id": "my-group",
            "enable.auto.commit": "false",
        })
        items = kafka.topic_as_map(consumer, "my-topic")
        await coco.mount_each(process_message, items)

    coco.auto_refresh: re-run a function on an interval

    Use this when the source has no change events. The v1 Google Drive source, for example, does not support live mode. process_file and DocEmbedding below are the ones from the gdrive_text_embedding example, which runs once in catch-up mode. coco.auto_refresh wraps a function as a live component. It runs the function once, signals ready, then runs it again after each interval:

    python
    import datetime
    import os
    
    import cocoindex as coco
    from cocoindex.connectors import google_drive, postgres
    
    
    async def sync_drive(table: postgres.TableTarget[DocEmbedding]) -> None:
        source = google_drive.GoogleDriveSource(
            service_account_credential_path=os.environ["GOOGLE_SERVICE_ACCOUNT_CREDENTIAL"],
            root_folder_ids=os.environ["GOOGLE_DRIVE_ROOT_FOLDER_IDS"].split(","),
        )
        await coco.mount_each(process_file, source.items(), table)
    
    
    @coco.fn
    async def app_main() -> None:
        table = await postgres.mount_table_target(
            PG_DB,
            table_name=TABLE_NAME,
            table_schema=await postgres.TableSchema.from_class(DocEmbedding, primary_key=["id"]),
            pg_schema_name=PG_SCHEMA_NAME,
        )
        await coco.mount(
            coco.auto_refresh(sync_drive, interval=datetime.timedelta(minutes=1)),
            table,
        )

    Each cycle is reconciled against the previous one:

    • A file that sync_drive no longer sees is not declared, so its rows are deleted. You do not track deletions yourself.
    • process_file is still memoized, so an unchanged file is a memo hit and makes no model call (unless the code of process_file changed).
    • Cycles never overlap. The interval is a fixed delay after each cycle ends.

    In catch-up mode, auto_refresh runs the function once and exits, so this app also works as a cron job.

    Upgrading from the v0 API

    The first version of this post used the v0 API. The v1 equivalents:

    v0v1
    refresh_interval= on add_source()coco.auto_refresh(fn, interval=...)
    recent_changes_poll_interval= on the Google Drive sourceNo v1 equivalent; use coco.auto_refresh
    cocoindex.FlowLiveUpdater(flow)app.update(live=True) and await handle.result()
    cocoindex update -L maincocoindex update -L main (unchanged)

    Where to go next

    Frequently asked questions

    What is live mode in CocoIndex?

    Live mode keeps a CocoIndex app running after its first catch-up pass. It watches the source and applies each change (new, modified or deleted items) to the target as the source reports it, instead of waiting for the next scheduled run. Start it with cocoindex update -L main or app.update(live=True).

    See Catch-up runs exit, live runs keep watching.

    Why does an AI agent need a live index?

    An agent acts on the chunks it retrieves and cannot tell when a chunk is out of date. A stale index leads to code that calls a renamed function or a reply that cites a deleted doc, and it misses docs added since the last run. A coding agent also needs to find the edits it just made. Live mode applies each change as it happens and re-embeds only what changed.

    See Why an agent needs a live index.

    What is the difference between catch-up mode and live mode?

    A catch-up run (cocoindex update main) scans the source, processes what changed since the last run, syncs the target and exits. A live run (cocoindex update -L main) does the same pass, then keeps running and reacts to changes as the source reports them. The pipeline code is the same; you choose the mode at run time.

    See Catch-up runs exit, live runs keep watching.

    How do I make a CocoIndex v1 pipeline live?

    Pass a live source to coco.mount_each(), for example localfs.walk_dir(..., live=True).items(), and run the app with -L or live=True. mount_each() scans the current state, signals ready, then re-runs the component of each changed item and removes the component and rows of each deleted item.

    See Make an indexing app live.

    How do I keep a source without change events fresh, such as Google Drive?

    Wrap the function that reads the source in coco.auto_refresh(fn, interval=...) and mount it. It runs the function once, signals ready, then runs it again after each interval. Rows for items the function no longer sees are deleted, and memoized per-item functions skip unchanged items.

    See Three ways a source goes live.

    What replaces refresh_interval and FlowLiveUpdater from the v0 API?

    refresh_interval becomes coco.auto_refresh(fn, interval=...), and cocoindex.FlowLiveUpdater(flow) becomes app.update(live=True) with await handle.result(). The CLI flag cocoindex update -L is unchanged.

    See Upgrading from the v0 API.

    Build fresh context for your agents.

    CocoIndex keeps derived data in sync with its sources. Only what changed re-runs.