# Declaring functions with @coco.fn

> **CocoIndex v1.** This page documents CocoIndex **v1** — a ground-up redesign from v0. When writing code, ignore any v0 flow-builder DSL or deprecated decorators.
>
> Source: https://cocoindex.io/docs/programming_guide/function/ · Docs index: https://cocoindex.io/docs/llms.txt · Agent skill: https://cocoindex.io/docs/skill.md
>
> v0→v1 quick map — if you reach for these v0 symbols, stop and use the v1 form: `@cocoindex.flow_def`/`FlowBuilder` → `coco.App` + a `@coco.fn` main function; `add_collector()`/`collect()`/`export()` → declare target states (`declare_row`, `declare_file`); `cocoindex.sources/functions/targets.*` → connector APIs (`localfs.walk_dir`, `coco.ops.*`, `postgres.declare_table_target`). Full mapping + API reference: https://cocoindex.io/docs/skill.md.

**Note — Prerequisite**
Read [Core Concepts](/docs/programming_guide/core_concepts/) for change detection and function memoization, with diagrams.

It's common to factor work into helper functions (for parsing, chunking, embedding, formatting, etc.). In CocoIndex, you can mark any function with `@coco.fn` when you want to add incremental capabilities to it. The marked function is still a normal function: its signature stays the same, and you can call it normally.

```python
@coco.fn
async def process_file(file: FileLike) -> str:
    return await file.read_text()

# Can be called like any normal function
result = await process_file(file)
```

`@coco.fn` preserves the sync/async nature of the underlying function. Decorating a sync function yields a sync function; decorating an async function yields an async function.

## How to think about `@coco.fn`

Decorating a function tells CocoIndex that calls to it are part of the incremental update engine. You still write normal Python, but CocoIndex can now:

- Detect when inputs or code have changed (change detection)
- Skip work when nothing has changed (memoization)

This is what lets CocoIndex avoid rerunning expensive steps on every `app.update()`. See [Processing Component](/docs/programming_guide/processing_component/) for how decorated functions are mounted at component paths.

If you don't need any of the above for a helper, keep it as a plain Python function.

## Change detection and memoization

Every `@coco.fn` function participates in CocoIndex's change detection system. With `memo=True`, the function's results are cached and reused when nothing has changed. These two mechanisms (detecting changes and acting on them) work together to enable incremental updates. For the big-picture view (how memoization skips unchanged work across both input and code changes) see the [Function memoization section in Core Concepts](/docs/programming_guide/core_concepts/#function-memoization-skip-unchanged-computations).

**Tip — Mental model**
A function's behavior is determined by its **logic** (source code, [`deps`](#deps), [`version`](#version)), its **inputs** (arguments), and the **context values** it reads via [`use_context()`](/docs/programming_guide/context/#retrieving-values). If none of those have changed, a memoized function returns its cached result without re-executing.

### Change detection

CocoIndex detects three kinds of changes:

**Logic changes** — a function's *logic* is its own source code (or, when you set [`version`](#version), that integer **instead of** the source), plus any [`deps`](#deps) values. Tracked by `@coco.fn`. Logic fingerprints **propagate transitively** up the call chain, following *runtime invocations* rather than the static call graph: every `@coco.fn` function reached during a memoized call — directly, or through plain undecorated helpers — contributes its logic to that call. So if `foo` (memoized) calls `bar` (also `@coco.fn`-decorated, with or without `memo=True`) and `bar`'s logic changes, `foo`'s memo is invalidated; undecorated helpers in between don't break the chain. What the decorator buys is having a function's *own* logic tracked: an undecorated helper is invisible to change detection, so editing it invalidates nothing. This is why `@coco.fn` matters for **any** function in the call chain, not just memoized ones — see [Common patterns](#common-patterns) and the [FAQ](/docs/faq/#how-does-logic-change-propagation-work). To stop propagation deliberately, set [`logic_tracking`](#logic_tracking) to `"self"` or `None`.

**Input changes** — the function's arguments. Tracked by `@coco.fn`. When you call a function with different arguments, the fingerprints change. Input fingerprints do not propagate transitively.

**Context changes** — [context values](/docs/programming_guide/context/) with [`detect_change=True`](/docs/programming_guide/context/#change-detection). Tracked by [`use_context()`](/docs/programming_guide/context/#retrieving-values) at the call site — independent of `@coco.fn`. Context reads propagate transitively: if a memoized `foo` calls `bar` and `bar` reads a change-detected context value, then changing that value invalidates `foo`'s memo too — even though `foo` itself never called `use_context()`. What matters is whether the key was read *anywhere* during the memoized call, not where in the call chain.

### Memoization

With `memo=True`, the function's result is cached. On subsequent calls, if no logic, input, or read context values have changed, the cached result is reused without executing the function body — it carries over [target states](/docs/programming_guide/target_state/) declared during the function's previous invocation and returns its previous return value.

```python
@coco.fn(memo=True)
def process_chunk(chunk: Chunk) -> list[float]:
    # This computation is skipped if chunk, logic, and context are unchanged
    return embed(chunk.text)
```

**Info — Type annotations**
Add a **return type annotation** to memoized functions so CocoIndex can properly reconstruct cached values. Without a type annotation, cached values may deserialize as basic Python types (`dict`, `list`, etc.) instead of their original types. See [Serialization](/docs/programming_guide/serialization/) for details on supported types.

**Caution — `memo=True` constraints**
A memoized function:

- **Must run inside a [processing component](/docs/programming_guide/processing_component/)** for memoization to take effect. Outside a component context (e.g., invoked from a script or a test), the function still executes correctly but the cache is bypassed silently — every call runs the body.
- **Cannot mount child components when called directly** (`coco.mount(...)` / `coco.use_mount(...)` inside the body of a memoized function invoked as a plain call). Mounting is a side effect the call's cache cannot replay; CocoIndex raises an error if a directly-called memoized function attempts it. The restriction applies to the call, not the function — the fix, which the error message itself suggests, is to **mount the memoized function as a component**: a [memoized component](/docs/programming_guide/processing_component/#memoized-components) may mount children freely (a cache hit carries its child mounts over along with its target states). Alternatively, drop `memo=True`, or restructure so the mount happens in a non-memoized caller.

#### Cache-hit semantics

When a memoized function's cache hits, its **body does not run** — and neither do any nested `@coco.fn` calls inside it. The cached output is replayed directly: the previous return value is returned, and any target states the function declared on its previous run are carried over.

```python
@coco.fn(memo=True)
async def inner(text: str) -> str:
    print("inner ran")
    return await call_llm(text)

@coco.fn(memo=True)
async def outer(text: str) -> str:
    print("outer ran")
    return await inner(text) + "!"

# First call: both print, both bodies execute
result = await outer("hello")

# Second call with same inputs: nothing prints — outer's cached value is returned
# directly, and inner is not invoked at all.
result = await outer("hello")
```

Propagation (logic, `deps`, context) is recorded during the **previous** invocation and replayed on cache lookup. You don't need to model the internals — change anything that affects behavior and the cache invalidates correctly.

**Note — Exceptions and the cache**
If a memoized function raises, no cache entry is written for that call. The next invocation with the same inputs sees a cache miss and re-executes the body — exceptions never poison the cache, so you don't need to wrap calls defensively.

#### When to memoize

**Cost:** A directly-called memoized function's cache entry must store everything needed to serve a hit — the return value, plus the *path* of every [target state](/docs/programming_guide/target_state/) it declared during the call (the state values themselves stay in the enclosing component's tracking; the recorded paths are what keep those states alive on a hit). Large return values, or a call that declares many target states, mean large entries. (A function *mounted* as a component avoids the per-state paths entirely: a [memoized component](/docs/programming_guide/processing_component/#memoized-components)'s entry holds fingerprints plus its return value — `None` for a typical mounted declarer.)

**Benefit:** Memoization saves more when:

- The computation is expensive
- The function's caller is reprocessed frequently (due to logic or input changes)

**Placement:** memoize at a boundary only if its key can hit while every *enclosing* memo boundary misses. Stacked levels pay off exactly when the outer boundary is keyed more coarsely than the inner one's inputs. A per-file component plus a per-chunk embedding memo composes well: an edited file re-runs its component while its unchanged chunks still skip embedding.

If an enclosing *memoized* component's identity already commits to this function's inputs (for example, the component path is keyed by a hash of the content the function transforms), the inner memo can never hit on a *content-driven* rerun. The component memo already covers those cases. The inner memo can still help when the enclosing component re-runs because its code changed or its `version` was bumped. Each entry permanently stores a value, often one the target also stores, for those reruns.

A 1536-dim float32 vector is ~6 KB. An embedding memo under content-hash-keyed components therefore stores ~600 MB per 100k chunks to cover those rare reruns. Consider that storage cost when deciding whether to keep the inner memo.

**Examples:**

- ✅ **Embedding functions** — good to memoize when callers can re-run with the same text (e.g., per-file components re-running on edits). Computation is heavy; return value is fixed-size and not too large.
- ❌ **Splitting text into fixed-size chunks** — usually not worth memoizing. Computation is light; return value can be large.
- ✅ **Processing component for files that are mostly stable between runs** — memoization skips unchanged files, saving the cost of reading and processing their contents.
- 🤔 **Chunk embedding when file-level memoization is already enabled** — still beneficial, but less so for stable files. The benefit increases for files that change frequently, or when your code evolves (e.g., adding more features per file triggers file-level reprocessing, but unchanged chunks can still skip embedding).
- ❌ **Chunk embedding inside a *memoized* component keyed by the chunk's content hash** — usually redundant by the placement rule: the component memo already commits to the text, so the inner memo serves only reruns caused by the component's own code or `version` changes — at the cost of storing every vector twice.

### Controlling change detection scope

Three parameters on `@coco.fn` let you customize how logic changes are detected:

- **`logic_tracking`** — controls the *scope* of automatic logic change detection
- **`version`** — provides explicit manual control over when dependent memos are invalidated
- **`deps`** — declares external values (e.g. a module-level prompt string) as part of the function's logic, so changing them invalidates dependent memos

These parameters control logic fingerprinting. Data fingerprinting (for arguments, `deps` values, and context values) is controlled by the objects themselves (see [Memoization Keys & States](/docs/advanced_topics/memoization_keys/)). Three more tools cover what a decorator parameter cannot: [`coco.LogicDep` / `coco.use_logic_dep`](#per-call-dependencies-logicdep-and-use_logic_dep) for a dependency decided per call, [`coco.DataDep` / `coco.use_data_dep`](#data-dependencies-datadep-and-use_data_dep) for data a body reads that is not among its arguments, and [`coco.logic_propagated_to_callers`](#callbacks-logic_propagated_to_callers) for callbacks that cross a `"self"` boundary.

#### `logic_tracking`

The `logic_tracking` parameter controls whether and how logic changes are detected:

- **`"full"` (default):** Track this function's logic AND all transitively called `@coco.fn` functions' logic. A change anywhere in the call chain invalidates dependent memos.
- **`"self"`:** Track only this function's own logic — its fingerprint still propagates to its callers, but the logic of functions it calls does not propagate through it.
- **`None`:** Don't track logic through this function at all — neither its own logic nor its callees' reaches its callers.

These settings govern nested *function calls*. A [mounted child component](/docs/programming_guide/processing_component/) is tracked through the component tree instead, so its logic reaches the parent regardless of the parent's `logic_tracking`.

**Caution — Adopting tracking later does not re-store existing memo entries**
A memo entry records only the logic fingerprints its writer offered when it was stored. Decorating a function with `@coco.fn` after memoized callers have stored entries that ran it, or raising its `logic_tracking` from `None`, leaves those entries valid: nothing in them can fail. Ship such a change together with a [`version`](#version) bump or a memoization-key change on the memoized callers, so their entries are stored again with the new fingerprints.

#### `version`

The `version` parameter lets you explicitly invalidate dependent memos by bumping an integer:

```python
@coco.fn(version=2)
def process_chunk(chunk: Chunk) -> list[float]:
    # Bumping version invalidates all memoized callers, even if code looks the same
    return embed(chunk.text)
```

**Caution — `version` replaces source tracking**
Setting `version` makes the integer the function's logic **instead of** its source code — the body is no longer fingerprinted. Edits to a versioned function take effect only when you bump the number.

#### `deps`

The `deps` parameter declares external value(s) the function logic depends on but that aren't visible in its body — for example a prompt string or a model identifier defined at module scope. When the value changes, the function's logic fingerprint changes and dependent memos are invalidated, exactly as if the function body had been edited.

```python
SYSTEM_PROMPT = "You are a helpful assistant. Be concise."

@coco.fn(memo=True, deps=SYSTEM_PROMPT)
def summarize(text: str) -> str:
    # Editing SYSTEM_PROMPT invalidates this function's memo
    # (and propagates to memoized callers) just like a logic change would.
    return call_llm(SYSTEM_PROMPT, text)
```

For multiple dependencies, pass a tuple or dict:

```python
SYSTEM_PROMPT = "..."
MODEL = "claude-haiku-4-5"

@coco.fn(memo=True, deps={"prompt": SYSTEM_PROMPT, "model": MODEL})
def summarize(text: str) -> str:
    return call_llm(SYSTEM_PROMPT, text, model=MODEL)
```

The value is canonicalized through the [memoization-key pipeline](/docs/advanced_topics/memoization_keys/), which honors `__coco_memo_key__()`, registered memo key functions, and the standard handling for primitives, dataclasses, and Pydantic models.

**Caution — Snapshotted at decoration time**
`deps` is evaluated **once** when the decorator is applied (typically at module import), not re-evaluated per call. For per-call or per-instance values (instance attributes in a bound method, request-scoped config, anything that changes at runtime) pass them as regular function arguments instead, so the memoization layer observes each new value.

`deps` requires `logic_tracking` to be enabled; combining `deps=<value>` with `logic_tracking=None` raises `ValueError`.

#### Per-call dependencies: `LogicDep` and `use_logic_dep`

`deps` describes the whole function: every call depends on the value. Some dependencies are decided per call. A tree walk's `visit_file` depends on the parser of *that file's* language, and a change to one parser should re-run the files it parsed, not every file the walk saw. `coco.LogicDep` declares such a value once, at module import; `coco.use_logic_dep` records it as part of the current call's logic:

```python
PARSER_DEPS = {
    name: coco.LogicDep(("parser", name, version))
    for name, version in PARSER_VERSIONS.items()
}

@coco.fn(logic_tracking="self", version=1)
async def visit_file(file: File) -> None:
    language = detect_language(file.path)
    if (dep := PARSER_DEPS.get(language)) is not None:
        coco.use_logic_dep(dep)  # this call's logic also depends on that parser
    ...
```

A recorded dependency travels exactly like the function's own fingerprint: into the entry of a memoized caller, onward through `"full"` callers, hidden by a `"self"` caller above unless the call is carried by [`logic_propagated_to_callers`](#callbacks-logic_propagated_to_callers), and from a [mounted component](/docs/programming_guide/processing_component/) into every ancestor component's entry. (A memoized callee that hits re-offers its stored entry to the caller whole, and that rides to the enclosing component regardless of modes, so a `"self"` caller may record more after a hit than after a miss, never less.) When a later release constructs the `LogicDep` with a new value, only the entries that recorded the old one are invalidated: in the example, a parser's new version re-runs its files and the directories above them, and every other subtree is reused. It is recorded whatever the function's own `logic_tracking` is: it describes the call, not the function's code.

**Caution — Construct at import, never lazily**
A memo entry is validated against the dependencies registered in the process at that moment. A `LogicDep` constructed on first use registers too late, so entries recorded with it are evicted when probed before that first use; one constructed per call is released too early, so they are evicted every pass. Neither is stale data, only memoization that silently stops working. Build every `LogicDep` before any component runs, at module import; constructing one inside a component context raises.

Adopting `use_logic_dep` in a function that memoized callers already stored entries for does not re-store those entries (the caution under [`logic_tracking`](#logic_tracking) applies): ship the adoption with a `version` bump or a memoization-key change on those callers.

#### Data dependencies: `DataDep` and `use_data_dep`

**Caution — Experimental**
`DataDep` and `use_data_dep` are experimental. Experimental APIs are outside CocoIndex's compatibility promise: a later release, including a minor one, may change their API or semantics, or remove them.

Logic, arguments and context values are all known before a body runs. Some bodies read things decided only while they run: an agent session opens the files the model chose, a test runner touches the files the tests import. `coco.DataDep` declares a *kind* of such a read once, at module import, with a validator; `coco.use_data_dep` records one read into the current call, as a key and the state observed. The next probe validates each record from the stored key and state alone, through the kind's validator, so the entry can be reused even though nothing in its arguments covers the read.

```python
def _validate_file(key: str, prev: tuple[int, bytes]) -> coco.MemoStateOutcome:
    try:
        st = os.stat(key)
    except FileNotFoundError:
        return coco.MemoStateOutcome(state=prev)  # memo_valid defaults to False
    if st.st_mtime_ns == prev[0]:
        return coco.MemoStateOutcome(state=prev, memo_valid=True)
    digest = hashlib.sha256(Path(key).read_bytes()).digest()
    return coco.MemoStateOutcome(state=(st.st_mtime_ns, digest), memo_valid=digest == prev[1])


# Module level, at import, after the validator it names.
FILE = coco.DataDep("file", validate=_validate_file)


@coco.fn(memo=True)
async def session(task: Task) -> Outcome:
    ...
    # Stat *before* reading, so the recorded mtime is never newer than the bytes.
    st = os.stat(path)
    data = path.read_bytes()
    coco.use_data_dep(FILE, key=str(path.resolve()), state=(st.st_mtime_ns, hashlib.sha256(data).digest()))
    ...
```

The validator takes the recorded key and the stored state and returns a [`MemoStateOutcome`](/docs/advanced_topics/memoization_keys/#memo-state-validation), sync or awaitable; its parameter annotations drive typed deserialization of the stored key and state, so annotate both. It never receives `NON_EXISTENCE`. Renaming a kind invalidates every entry that recorded it, which doubles as its version bump.

Key and state are any serializable values and are **snapshotted at the call**: identity and equality are fingerprints of the serialized bytes, and `__coco_memo_key__` is not consulted, so use plain values (the resolved path string, a tuple of mtime and digest). Recording the same kind and key again with an equal state records once; recording it with a **different** state makes the run store no entry, because output that used two states of one key has no single state to validate against. A body that writes what it then reads back should not record the read-back.

The rules for an entry that recorded data dependencies:

- The entry is reused only when its arguments, logic and context match **and** every record validates. A record whose kind is no longer registered, or whose stored key or state no longer deserializes, is a logged miss; a validator's exception propagates. A changed-but-valid state is refreshed without a run. On a miss the body's new records replace the stored set.
- A record rides up through every enclosing function frame regardless of `logic_tracking`, into every enclosing memoized function's entry and the component's entry, where it stops. Records are **not** copied into ancestor components' entries. A component reports one bit upward instead, and its parent's entry lists the children that reported it: **a component is skipped only if its own entry validates and every descendant that recorded data dependencies validates too**, which the parent's probe checks from those descendants' own stored entries, by path, without mounting them. A changed-but-valid descendant record is refreshed in place. The probe stops, and the body re-runs, at a listed descendant with no entry (a non-memoized one, whose records are dropped, or one whose last run observed a key in two states), a record that no longer decodes, a kind no longer registered, or a validator answering invalid. Closed descendants are carried over by the reuse unvalidated, as they always were. A component that records nothing is unaffected.
- Because a parent's probe may validate a record while the component that recorded it is not mounted, a validator runs with no component context and must depend on its key and stored state alone.
- `use_data_dep` raises outside a component context, and when the current frame has already closed. The check covers the current frame only: a `@coco.fn` reader started as a task that outlives the decorated function which started it keeps its own open frame, so its records merge into an already-joined frame and are lost, as its logic deps are today. Await readers inside the function that started them.

#### Callbacks: `logic_propagated_to_callers`

`logic_tracking` is per function, so `"self"` hides the logic of *everything* a function calls — including callbacks it received as arguments. A library whose `accept` is `logic_tracking="self"` therefore hides the user's `visit_file`: an edit to `visit_file` invalidates nothing, even though it is the user's input, not the library's implementation detail.

`coco.logic_propagated_to_callers(fn)` closes this gap. It is library-side; the user only puts `@coco.fn` on the callback. It does not make `fn` tracked: `fn`'s own `@coco.fn` and `logic_tracking` decide what it offers, exactly as for any function. The wrapper only carries that up to the frame that called it, through layers that would otherwise drop it. (`coco.logic_tracked` is the former name; it still works and emits a `DeprecationWarning`.)

Call it at the entry point that hands a callback down. The function that calls `logic_propagated_to_callers` is the callback's **owner**. The returned wrapper runs `fn` as usual; afterwards the logic `fn` offered (by its own `logic_tracking`) is recorded in **every** memo entry between the call site and the owner's frame (the owner's own entry included) whatever their `logic_tracking`. The owner's mode only decides what goes further up: with `"full"` (the default) the owner offers the callback's logic to its callers like any other dependency; with `"self"` or `None` it stops there.

```python
# Library
@coco.fn(memo=True, logic_tracking="self", version=1)
async def accept(file: File, visit_file: Callable[[File], Awaitable[None]]) -> None:
    await visit_file(file)


@coco.fn
async def walk(repo: Repo, visitor: RepoVisitor) -> None:
    visit_file = coco.logic_propagated_to_callers(visitor.visit_file)  # walk owns the callback
    for file in repo.files():
        await coco.use_mount(coco.component_subpath(file.path), accept, file, visit_file)
```

```python
# User
class MyVisitor(RepoVisitor):
    @coco.fn
    async def visit_file(self, file: File) -> None:
        ...


@coco.fn
async def app_main(repo: Repo) -> None:
    await walk(repo, MyVisitor())
```

The user edits `visit_file`; every `accept` memo entry misses and runs again. The user never sees `logic_propagated_to_callers`. `accept` needs no code of its own: the wrapper it receives already carries its owner.

**When a library does not do this**, wrap the callback in your own code. Your function becomes the owner:

```python
@coco.fn(memo=True)
def process(doc: str) -> str:
    return third_party_transform(doc, coco.logic_propagated_to_callers(summarize))
```

Rules, for whoever calls `logic_propagated_to_callers`:

- Decorate the callback with `@coco.fn` for its own code to be tracked. An undecorated callable contributes only the `@coco.fn` functions it calls, as always.
- The wrapper is valid only while the owner's call is running. A call after the owner has returned raises `RuntimeError` instead of silently running untracked. Mount the components that call it with `use_mount`, or `await handle.ready()` inside the owner.
- The wrapper keys by `fn`: two wrappers of the same `fn` key alike. It does not key as the bare `fn`, so passing `logic_propagated_to_callers(fn)` where `fn` was passed before moves that memo key once, when the wrapper is adopted.
- Adopting the wrapper does not re-store existing memo entries (see the caution under [`logic_tracking`](#logic_tracking)): entries stored before the wrap recorded no callback logic and stay valid until their key or the memoized caller's `version` moves. Ship the adoption with one such change.

#### Common patterns

**Tip — `@coco.fn` on non-memoized helpers**
You can (and often should) decorate helpers with `@coco.fn` even without `memo=True`. The decorator's job is to make the function's logic *visible* to the change detection system. A bare helper is invisible: editing it will not invalidate any memoized caller, leading to silently stale results. Add `@coco.fn` so its fingerprint propagates; only add `memo=True` if caching its return value is worth it.

These parameters can be set on any `@coco.fn` function — not just memoized ones.

**Fully automatic (default)** — use `logic_tracking="full"` (or omit it) without setting `version`. Any logic change in the function or its callees invalidates dependent memos.

```python
@coco.fn
async def process_file(file: FileLike) -> list[Chunk]:
    # Any change here or in called @coco.fn functions invalidates dependent memos
    text = await file.read_text()
    return split_and_embed(text)
```

**Manual, precise control** — use `logic_tracking="self"` with `version`. You decide what counts as a behavior change by bumping `version`, without being affected by implementation detail changes (performance optimizations, logging tweaks, refactoring, etc.).

```python
@coco.fn(logic_tracking="self", version=3)
async def process(data: str) -> str:
    # Bump version when behavior changes (e.g., new output format).
    # Internal refactors or logging changes won't trigger reprocessing.
    return await transform(data)
```

**Opt out of tracking** — use `logic_tracking=None` for functions with a stable contract (where logic changes don't affect output), or functions whose changes don't affect behavior (e.g., logging, performance hints). This prevents unnecessary reprocessing when only internals change.

```python
@coco.fn(logic_tracking=None)
def embed(text: str) -> list[float]:
    # Contract is stable: same input always produces the same embedding.
    # Internal changes (e.g., switching backends) are handled by version bumps.
    return model.encode(text)
```

**Note**
[Context changes](/docs/programming_guide/context/#change-detection) are independent of `@coco.fn` and `logic_tracking`. Even with `logic_tracking=None`, a change in a change-detected context value still invalidates dependent memos, because context tracking is done by `use_context()`, not by the decorator.

### Debugging unexpected re-runs

If a memoized function is re-running when you expect a cache hit, walk through the inputs in order:

1. **Logic** — did the function's source code change? Did any `@coco.fn` it transitively calls change? Was a `version` bumped? Was a `deps` value edited?
2. **Inputs** — are the arguments byte-identical to the previous call (after canonicalization)? Custom types with unstable equality are a common source of spurious invalidation — see [Memoization Keys & States](/docs/advanced_topics/memoization_keys/).
3. **Context** — did any [`detect_change=True`](/docs/programming_guide/context/#change-detection) context value read during the previous invocation change? Remember this propagates transitively through nested `@coco.fn` calls.

If none of those changed and the function still re-runs, the most common cause is a non-stable fingerprint on a custom-typed argument or context value — define `__coco_memo_key__` to make it deterministic.

Conversely, if a memo is hitting when you expect invalidation, common causes are:

- A logic change in a helper that is **not** decorated with `@coco.fn`. Add `@coco.fn` to it (no `memo=` needed) so its logic participates in propagation — see [Common patterns](#common-patterns).
- A `deps` value that you changed at runtime: `deps` is [snapshotted at decoration time](#deps), so per-instance or per-request values must be passed as regular arguments instead.
- The function uses [`logic_tracking=None`](#logic_tracking), which opts it out of code-change detection entirely.
- The change is in a callback that passes through a `logic_tracking="self"` function and nobody wrapped it with [`coco.logic_propagated_to_callers`](#callbacks-logic_propagated_to_callers).

### Customizing data fingerprinting

By default, CocoIndex fingerprints function arguments, `deps` values, and context values automatically for most types — primitives, containers, dataclasses, Pydantic models, and picklable objects. For custom types, or when you need multi-level validation (e.g., check mtime first, then content hash), see [Memoization Keys & States](/docs/advanced_topics/memoization_keys/).

For per-function overrides (excluding an argument from the memo key, or transforming it just for this function) pass `memo_key={...}` on `@coco.fn` / `@coco.fn.as_async`; see [Override at the call site with `memo_key=`](/docs/advanced_topics/memoization_keys/#override-at-the-call-site-with-memo_key).

## Execution capabilities

The following capabilities control *how* the function executes, independent of change detection and memoization.

### Async adapter

Use `@coco.fn.as_async` when you need an **async** interface for a function that has a sync underlying implementation. This is useful for compute-intensive leaf functions. The decorator also accepts an `async def` body as-is, and [batching](#batching) and [runner](#runner) (which need an async interface) are always enabled through it.

```python
@coco.fn.as_async
def embed(text: str) -> list[float]:
    return model.encode([text])[0]

# External usage: always async, even though the function body is sync
embedding = await embed("hello world")
```

`@coco.fn.as_async` is equivalent to wrapping the function in `asyncio.to_thread()` — the sync function runs on a thread pool and doesn't block the event loop.

You can also call any `@coco.fn`-decorated function asynchronously via the `.as_async()` method, without changing its primary signature:

```python
@coco.fn
def expensive_fn(data: bytes) -> bytes:
    return process(data)

# Primary call is sync:
result = expensive_fn(data)

# Async call via .as_async():
result = await expensive_fn.as_async(data)
```

### Batching

With `batching=True`, multiple concurrent calls to the function are automatically batched together. This is useful for operations that are more efficient when processing multiple inputs at once, such as embedding models.

Batching requires an async interface, so it is always enabled with `@coco.fn.as_async(batching=True)`, whether the body is a plain `def` or an `async def`. `@coco.fn(batching=True)` is not supported: it raises `ValueError` when the decorator is applied, whichever kind of body it is given.

When batching is enabled:

- The function implementation receives a `list[T]` and returns a `list[R]`
- The external signature becomes `async T -> R` (single input, single output)
- Concurrent calls are collected and processed together
- The implementation runs outside the calling components — see [Context values in a batched body](#context-values-in-a-batched-body)

```python
@coco.fn.as_async(batching=True, max_batch_size=32)
def embed(texts: list[str]) -> list[list[float]]:
    # Called with a batch of texts, returns a batch of embeddings
    return model.encode(texts)

# External usage: async, single input, single output
embedding = await embed("hello world")  # Returns list[float]

# Concurrent calls are automatically batched using asyncio.gather
embeddings = await asyncio.gather(
    embed("text1"),
    embed("text2"),
    embed("text3"),
)
```

The `max_batch_size` parameter limits how many inputs can be processed in a single batch.

The `max_in_flight` parameter bounds how many batches run at once. By default (`None`), a full batch starts as soon as it fills, and the next partial batch starts only once every running batch has finished. With `max_in_flight=n`, at most `n` batches run concurrently, and whenever one finishes the next pending batch (full or partial) starts, so a slow batch no longer holds back the rest:

```python
@coco.fn.as_async(batching=True, max_batch_size=32, max_in_flight=4)
async def embed(texts: list[str]) -> list[list[float]]:
    return await client.embed(texts)
```

#### Pacing batches with a rate limiter

When each batch is one request against a quota, pass the quota's [`RateLimiter`](/docs/common_resources/rate_limiting/) as `rate_limiter`. Every batch takes one token before it runs, so a token pays for one batch rather than one call, and the batch boundary becomes the moment a request can be sent:

- No call runs on its own: calls join the open batch, and a pump runs the oldest pending batch once a `max_in_flight` slot and a token are available. Under a scarce quota, every call that arrives while the pump waits for the token rides the next batch.
- With tokens to spare, a batch goes out at once and the next one fills during its round trip (under `max_in_flight`, when set). The burst allowance of the limiter is how many requests may go out unbatched after an idle stretch.
- Requests the body makes beyond the batch's own — a retry, a per-item fallback — take their own tokens from the same limiter.

For a method whose limiter lives on the instance, pass a callable that receives `self` and returns the limiter (or `None` for no pacing); it is resolved once per batcher, which is already keyed by `self`:

```python
class Client:
    def __init__(self, limiter: RateLimiter) -> None:
        self._limiter = limiter

    @coco.fn.as_async(
        batching=True, max_batch_size=100, rate_limiter=lambda self: self._limiter
    )
    async def fetch(self, ids: list[str]) -> list[Item]:
        return await self._api.fetch_many(ids)  # one request, one token
```

#### Context values in a batched body

One batch merges calls from many components, so the body runs outside any of them: `coco.use_context(...)` inside a `batching=True` body raises `RuntimeError: No ComponentContext available`. Resolve context values in the caller — a `@coco.fn` function or method running inside the component — and pass them to the batched function as extra arguments:

```python
EMBED_CLIENT = coco.ContextKey[EmbedClient]("embed_client")

@coco.fn.as_async(batching=True, max_batch_size=64)
async def _embed_batch(texts: list[str], client: EmbedClient) -> list[list[float]]:
    return await client.embed(texts)

@coco.fn
async def embed(text: str) -> list[float]:
    client = coco.use_context(EMBED_CLIENT)  # inside the component
    return await _embed_batch(text, client)
```

Only the first parameter is batched. Every other argument is passed through to the body unchanged and is part of the key that selects the batcher (so is `self`, for methods): calls share a batcher only when these arguments compare equal, which also means they must be hashable. Pass them the same way on every call. An explicit value and the default, or the same value given positionally and by keyword, select different batchers, and the calls are never batched together.

#### Retrying a failed batch as smaller batches

Some batch failures depend on the batch's composition rather than any single input — e.g. a provider's per-request token or payload limit, or one input the provider rejects. No fixed `max_batch_size` can rule these out. When the function body hits such a failure, raise `coco.RetryWithSmallerBatch` from it: CocoIndex halves the batch and retries each half, recursing down to single items. At size 1, the wrapped original error is raised as that item's own failure — so one bad input fails only its own caller, while the rest of the batch still succeeds. There is no need to check the batch size before raising: at size 1 the engine simply unwraps the signal and raises the original error.

```python
@coco.fn.as_async(batching=True, max_batch_size=64)
async def embed(texts: list[str]) -> list[list[float]]:
    try:
        return await provider.embed(texts)
    except ProviderError as e:
        if is_transient(e) or is_auth_error(e):
            raise  # retrying smaller can't help these
        raise coco.RetryWithSmallerBatch() from e
```

Raising it is safe for *any* deterministic batch failure, even when you can't tell whether the batch size was the cause: if the error was actually global (it would fail at any size), the recursion terminates at single items and every caller receives the underlying error, at the bounded cost of extra attempts. Only propagate directly for errors where splitting provably can't help — global errors like invalid credentials, and transient errors (rate limits, network blips) that haven't yet been retried at the same size. Once same-size retries are exhausted, splitting is a reasonable last resort: a smaller request may pass where the large one timed out.

**Tip — When to use batching**

Batching is beneficial when:

- The underlying operation has significant per-call overhead (e.g., GPU kernel launch)
- The operation can process multiple inputs more efficiently than one at a time
- You have concurrent calls from multiple coroutines

Common use cases:

- **Embedding models** — most embedding APIs and models are optimized for batch processing
- **LLM inference** — batch multiple prompts together for better GPU utilization
- **Database operations** — batch inserts or lookups

### Runner

The `runner` parameter allows functions to execute in a specific context, such as a dedicated GPU runner that serializes GPU workloads.

Like batching, a runner requires an async interface, so it is always set with `@coco.fn.as_async(runner=...)`, on a plain `def` or an `async def` body. `@coco.fn(runner=...)` raises `ValueError` when applied.

```python
@coco.fn.as_async(runner=coco.GPU)
def gpu_inference(data: bytes) -> bytes:
    # Runs with GPU serialization
    return model.predict(data)

# External usage: async
result = await gpu_inference(data)
```

The `coco.GPU` runner:

- By default, runs in-process with all functions sharing a queue for serial execution
- Sync functions run on a dedicated GPU thread to avoid blocking the event loop
- Set the environment variable `COCOINDEX_RUN_GPU_IN_SUBPROCESS=1` to run in a subprocess for GPU memory isolation

You can combine batching with a runner:

```python
@coco.fn.as_async(batching=True, max_batch_size=16, runner=coco.GPU)
def batch_gpu_embed(texts: list[str]) -> list[list[float]]:
    # Batched execution with GPU serialization
    return gpu_model.encode(texts)

# External usage: async
embedding = await batch_gpu_embed("hello world")

# Concurrent calls
embeddings = await asyncio.gather(
    batch_gpu_embed("text1"),
    batch_gpu_embed("text2"),
    batch_gpu_embed("text3"),
)
```

**Note**
By default, `coco.GPU` runs functions in-process, so no pickling is required. When using subprocess mode (`COCOINDEX_RUN_GPU_IN_SUBPROCESS=1`), the function and all its arguments must be picklable since they are serialized for subprocess execution.
