Memoization keys & states

Customize how CocoIndex fingerprints memoized inputs through the MemoInput trait, layer state validation (e.g. mtime, then content hash) on top with memo_state, and declare dependencies beyond the arguments with deps, LogicDep and DataDep.

Version

v 1.0.14

Last reviewed

Nov 4, 2026

CocoIndex detects logic, input, and context changes to decide whether it can reuse a memo. It fingerprints function arguments, deps values, and context values with detect_change through the same pipeline. Most types work automatically. You can customize fingerprinting and validation with:

• Memoization keys — how to control what CocoIndex uses as the fingerprint for your objects.
• Memo states — how to add post-fingerprint validation to check freshness beyond simple equality.

Both go through one trait, MemoInput, which keeps identity and freshness apart: its write_memo_key method is the key; its optional memo_state method is the state. The arguments of a #[cocoindex::function(memo)] function must implement MemoInput, unless memo_key(..) transforms or skips them. A context_key! value takes part only when the key is declared with detect_change.

How data fingerprinting works

For each data value (function argument, deps value, or context value), CocoIndex writes a canonical form through its MemoInput implementation. A type is a memo input when it implements MemoInput. The SDK implements it for the primitives (bool, the integer and float types, char, String, str), PathBuf / Path, Duration, SystemTime, uuid::Uuid, Fingerprint, bytes::Bytes and serde_json::Value; for Chrono date and time values with the chrono feature and for rust_decimal::Decimal with the rust_decimal feature (a connector feature turns these on). Option, arrays, slices, Vec, tuples, maps, sets, Box and Arc compose their children’s MemoInput. For your own structs and enums, derive it:

#[derive(Clone, Debug, Serialize, Deserialize, cocoindex::MemoInput)]
pub struct Utterance {
    pub speaker: String,
    pub text: String,
}

#[derive(Clone, Debug, Serialize, Deserialize, cocoindex::MemoInput)]
pub struct SessionTranscript {
    pub utterances: Vec<Utterance>,
    pub yt_channel: String,
    pub yt_title: String,
    pub yt_description: Option<String>,
}

The derive visits every field recursively, in order; the type itself does not need Serialize. Moving or renaming a derived type, or renaming one of its named fields, changes its identity and causes a one-time cache miss. A Vec<u8> is hashed element by element like any Vec<T>: for a large byte buffer, use bytes::Bytes for bulk hashing, or pass a precomputed Fingerprint (cocoindex::Fingerprint) when the content is already hashed.

The canonical forms are combined into a deterministic fingerprint. If the fingerprint matches a cached entry, the cached result is reused — unless memo states indicate it’s stale (see Memo state validation below).

Customizing the memoization key

Implement MemoInput (when you control the type)

Implement write_memo_key on your type and write a stable, deterministic value:

impl MemoInput for LlmClient {
    fn write_memo_key(&self, writer: &mut cocoindex::memo::MemoKeyWriter<'_>) -> Result<()> {
        // Write small serializable values: here, the model name.
        writer.write(&self.model)
    }
}

writer.write(&value) takes any Serialize value; writer.write_input(&value) nests another MemoInput. Add #[cocoindex::async_trait] to the impl when it also defines the async memo_state method.

Return something that uniquely identifies the semantic content your function depends on:

• Good: small tuples of primitives, e.g. (stable_id, version)
• Bad: memory addresses, unstable UUIDs, open file handles, SystemTime::now(), or large raw payloads

Example — DB row:

struct UserRow {
    user_id: i64,
    updated_at: i64,
}

impl cocoindex::MemoInput for UserRow {
    fn write_memo_key(
        &self,
        writer: &mut cocoindex::memo::MemoKeyWriter<'_>,
    ) -> cocoindex::Result<()> {
        writer.write(&("users", self.user_id, self.updated_at))
    }
}

Wrap it in a newtype (when you don’t control the type)

Rust’s orphan rules stop a crate from implementing MemoInput for a type it does not own, so wrap a third-party type in a local newtype and implement the trait on that, or transform the argument at the call site with memo_key(..) (next section):

struct ForeignId(third_party::Id);

impl cocoindex::MemoInput for ForeignId {
    fn write_memo_key(
        &self,
        writer: &mut cocoindex::memo::MemoKeyWriter<'_>,
    ) -> cocoindex::Result<()> {
        writer.write(&self.0.to_string())
    }
}

Override at the call site with memo_key(..)

The two approaches above are type-level: every memoized function sees the same fingerprint for a given value. To customize fingerprinting only for a specific function, add memo_key(..) to #[cocoindex::function(memo)]. It maps parameter names to either a transform function (transform the value before fingerprinting) or skip (exclude the parameter entirely).

fn entry_name_version(entry: &Entry) -> (String, u32) {
    (entry.name.clone(), entry.version)
}

#[cocoindex::function(memo, memo_key(entry = entry_name_version, extra = skip))]
async fn transform_entry(ctx: &Ctx, entry: &Entry, extra: &String) -> Result<String> {
    Ok(format!("{}:{extra}", entry.content))
}

memo_key(..) lists parameters by name: name = transform_fn fingerprints the function’s return value in place of the argument, name = skip leaves the argument out. A skipped parameter needs only Clone, not MemoInput. The attribute requires memo.

Each entry in memo_key(..):

• name = transform_fn — the transform’s return value is fingerprinted in place of the argument. Semantically the same as the type’s own write_memo_key, but scoped to this one function.
• name = skip — the parameter is excluded from the memo key. Changing its value never invalidates the cache. Useful for arguments that don’t affect the result (logger handles, debug flags, target handles).
• Not listed — the parameter is fingerprinted normally, through its MemoInput.

The macro checks the names at compile time. A transform also suppresses the argument type’s memo_state validation; use a newtype when the freshness check must stay.

Target handles. A target handle is where output goes, not an input of the computation, so it is not MemoInput. DirTarget, the Postgres, SQLite, Doris, SurrealDB, LanceDB, Neo4j and FalkorDB table and relation targets, the Qdrant collection, the Valkey index and the Turbopuffer namespace targets are Serialize instead: as an argument of a memoized function or of mount!, use_mount! or mount_each!, one is keyed by its stable key. The other connectors’ handles are only Clone. Skip those in the memo key; you can skip any handle the same way:

#[cocoindex::function(memo, memo_key(target = skip))]
async fn declare_nested(ctx: &Ctx, target: DirTarget, content: String) -> Result<()> {
    target.declare_file(ctx, "nested.txt", content.as_bytes())
}

Memo state validation

Sometimes fingerprint matching alone isn’t enough to decide whether a cached result is valid. For example:

• Multi-level validation: for files, check the modified time first (cheap), and only read the file for a content fingerprint when the time doesn’t match.
• Async validation: for an S3 object, send a HEAD request to check freshness — an inherently async operation.
• Stateful validation: for HTTP resources, store the last fetch time and use If-Modified-Since on the next run.

Memo state validation addresses these by letting you attach a state function to your objects. It runs after a fingerprint match, giving you a chance to check freshness before the cached result is reused.

How it works

When CocoIndex finds a fingerprint match, it calls each value’s memo_state with the stored state from the previous run:

1. First run (no previous state): previous is None.
2. Subsequent runs: previous is the state you stored last time; previous.deserialize::<T>() reads it back.

memo_state returns Ok(Some(MemoStateDecision::new(&state, memo_valid)?)):

• state — the current state value. CocoIndex stores it for the next run.
• memo_valid — whether the cached result is still valid.

This decouples “has the state changed?” from “can we reuse the memo?”:

• MemoStateDecision::new(&new_state, false) → cache is invalid. Function re-executes, new state is stored. On the first run (no previous cache), return the initial state with false.
• MemoStateDecision::new(&same_state, true) → nothing changed, cached result reused.
• MemoStateDecision::new(&new_state, true) → state changed but cached result is still valid (e.g. mtime changed but content hash unchanged). The new state is persisted so the next run uses the updated state.

Ok(None) (the default) means the value needs no state validation.

Implement memo_state (when you control the type)

Implement the async memo_state method of MemoInput beside write_memo_key, with #[cocoindex::async_trait] on the impl. The example uses a version number as the state.

struct Resource {
    id: &'static str,
    version: u64,
}

#[cocoindex::async_trait]
impl cocoindex::MemoInput for Resource {
    fn write_memo_key(
        &self,
        writer: &mut cocoindex::memo::MemoKeyWriter<'_>,
    ) -> cocoindex::Result<()> {
        // Identity only: which resource is it?
        writer.write(&self.id)
    }

    async fn memo_state(
        &self,
        previous: Option<&cocoindex::memo::MemoStateValue>,
    ) -> cocoindex::Result<Option<cocoindex::memo::MemoStateDecision>> {
        // `previous` is `None` on the first run.
        let previous_version = previous.and_then(|state| state.deserialize::<u64>().ok());
        Ok(Some(cocoindex::memo::MemoStateDecision::new(
            &self.version,
            previous_version == Some(self.version),
        )?))
    }
}

memo_state runs after an identity match. A stored state that no longer deserializes counts as a cache miss, not an error.

State for a type you don’t control

The newtype that carries write_memo_key for a foreign type carries memo_state too.

Async state methods

memo_state is always async, so a freshness check can send a network request (an S3 HEAD, an HTTP If-Modified-Since), and the state checks of nested values run concurrently. CocoIndex stores their results as one opaque state per top-level function argument or context dependency.

Dependencies beyond the arguments

A function’s memo key covers its arguments and the context values it reads. Three more ways declare what else a result depends on: deps, LogicDep / use_logic_dep and DataDep / use_data_dep.

version and deps on the attribute

version = N replaces the body hash as the function’s logic: bump it to invalidate every memo of the function, for example when a helper the macro cannot see has changed. deps = <expr> folds a value into the logic fingerprint, so every call of the function depends on it. The expression is evaluated and fingerprinted once per process, through MemoInput.

#[cocoindex::function(memo, version = 2)]
async fn my_fn(ctx: &Ctx, arg: &String) -> Result<String> { /* ... */ }

const PROMPT: &str = "summarize";

#[cocoindex::function(memo, deps = (PROMPT, DEPS_VALUE.load(Ordering::SeqCst)))]
async fn with_deps(_ctx: &Ctx, test: &str, x: &i64) -> Result<i64> {
    Ok(x * 2)
}

LogicDep: a dependency decided per call

Some dependencies are known only per call: a tree walk’s visit_file depends on the parser of that file’s language, and a new parser release should re-run only the files it parsed. LogicDep::new(&value) declares such a dependency; ctx.use_logic_dep(&dep)? records it as part of the current call’s logic. It travels like the function’s own fingerprint: into a memoized caller’s entry, and onward through "full" callers.

Construct a LogicDep before any component runs, never per call: a memo entry is validated against the dependencies registered at that moment, and one that is dropped at the end of a call evicts every entry that recorded it. A static behind LazyLock, forced in main or a lifespan, is the usual form.

use std::sync::LazyLock;

static RUST_PARSER: LazyLock<LogicDep> =
    LazyLock::new(|| LogicDep::new(&("rust-parser", 2)).unwrap());

#[cocoindex::function(memo)]
async fn leaf(ctx: &Ctx, value: &str) -> Result<String> {
    ctx.use_logic_dep(&RUST_PARSER)?;
    Ok(format!("leaf: {value}"))
}

A LogicDep is itself a MemoInput, keyed by its fingerprint, so it can also be passed as an argument.

DataDep: data the body reads while it runs (experimental)

DataDep is experimental: its API and semantics may change in a later release.

Logic, arguments and context values are known before a body runs. Some bodies read things decided only while they run: an agent session opens the files the model chose. DataDep::new(name, validate) declares such a kind of read with a validator; ctx.use_data_dep(&kind, &key, &state)? records one read as a key and the state observed. The next probe validates each record from the stored key and state alone, through the validator, so the entry can be reused even though nothing in its arguments covers the read. Construct a DataDep before any component runs, as with LogicDep; renaming a kind invalidates every entry that recorded it.

use cocoindex::Fingerprint;

type FileState = (u64, Fingerprint);

fn file_kind() -> DataDep<String, FileState> {
    DataDep::new("test-file", move |key: String, prev: FileState| async move {
        let path = PathBuf::from(&key);
        if !path.exists() {
            return MemoStateDecision::new(&prev, false);
        }
        let mtime = modified_nanos(&path);
        if mtime == prev.0 {
            return MemoStateDecision::new(&prev, true);
        }
        let digest = Fingerprint::from_bytes(&std::fs::read(&path)?);
        MemoStateDecision::new(&(mtime, digest), digest == prev.1)
    })
    .unwrap()
}

/// Read a file and record the read: stat *before* reading, so the recorded
/// mtime is never newer than the bytes it is paired with.
fn read(ctx: &Ctx, kind: &DataDep<String, FileState>, path: &Path) -> Result<Vec<u8>> {
    let mtime = modified_nanos(path);
    let data = std::fs::read(path)?;
    ctx.use_data_dep(
        kind,
        &path.to_string_lossy().into_owned(),
        &(mtime, Fingerprint::from_bytes(&data)),
    )?;
    Ok(data)
}

Recording the same kind and key with two different states in one run stores no entry: the function re-runs next time.

Memo blocks with ctx.memo

ctx.memo(&key, |ctx| async move { .. }) memoizes one block inside a function. The key is any Serialize value; the result must be Serialize + Deserialize. The closure body is not logic-tracked: when the block must invalidate after its enclosing #[cocoindex::function] changes, include that function’s generated __COCO_FN_HASH_<NAME> constant in the key.

let processed = ctx.memo(&fingerprint, |ctx| async move {
    // ... expensive computation, using `ctx` for `get_key` ...
    Ok("result".to_string())
}).await?;

Prefer #[cocoindex::function(memo)] for a whole function: it builds the key from the arguments and tracks the body’s logic for you.

Preventing memoization

Some types maintain internal state that makes memoization semantically incorrect. For example, a generator that tracks call counts would produce wrong results if memoized.

A type that does not implement MemoInput cannot be an argument of a memoized function: the compiler rejects it with a note that names the three ways out (derive, implement, or a memo_key override). Leave the trait unimplemented for a stateful type, and use memo_key(arg = skip) where it must be passed through.

Choosing keys and states

• Keep keys small and deterministic: use identifiers and versions, not full payloads. No pointer addresses, or random values.
• Separate identity from freshness: put stable identifiers (file path, URL, primary key) in the key. Put freshness checks (mtime, ETag, version) in the state.
• Use state validation for expensive checks: if freshness validation is costly (content hashing, network calls), a state function lets you do it only when the fingerprint matches, and only when a cheap pre-check (mtime) fails.
• Use MemoStateDecision::new(&new_state, true) for cheap state updates: when a cheap property changes (mtime) but the expensive check (content hash) confirms nothing meaningful changed, mark the memo valid while updating the state. This avoids re-executing the function and avoids re-checking the expensive property next time.
• Leave MemoInput unimplemented for stateful types: prevent subtle bugs from incorrect memoization of types with side effects.