Timeouts and deadlines

Use `Ctx::with_timeout` to put a time budget around a piece of processing — one file, one LLM call — with cooperative checks, deadline-aware retries, and clean time-boxing of whole updates.

Version

v 1.0.14

Last reviewed

Nov 4, 2026

Use ctx.with_timeout(duration) when a piece of CocoIndex work should stop after a time budget. The deadline lives on the Ctx: with_timeout returns a scoped Ctx, and the work you call with that Ctx sees the deadline.

The timeout is cooperative: CocoIndex checks the time at safe points and returns Error::DeadlineExceeded when the deadline has passed. Enforcement is best-effort beyond that — CocoIndex may additionally cancel an in-flight await where that is known to be safe (the built-in retry helpers do), but your own code inside an .await, such as an HTTP request or database query, is generally not interrupted. Use that library’s own timeout for individual I/O calls.

Quick start

The most common budget is per item of work inside a processing component — one file, one LLM call. Per-item budgets stay meaningful as your source grows (a whole-run limit does not), and they apply no matter how the app runs, including via the cocoindex update CLI:

use std::time::Duration;

#[cocoindex::function(memo, memo_key(target = skip))]
async fn process_file(ctx: &Ctx, file: FileEntry, target: DirTarget) -> Result<()> {
    // This file gets 30 seconds: every call made with `scoped` sees the deadline.
    let scoped = ctx.with_timeout(Duration::from_secs(30));
    let summary = summarize_with_llm(&scoped, &file.content_str()?).await?;
    target.declare_file(ctx, &out_name(&file), summary.as_bytes())
}

with_timeout returns a new Ctx; the original is unchanged. Pass the scoped Ctx to the calls that belong to the budget.

If the budget runs out, the component fails with Error::DeadlineExceeded like any other processor error: nothing external is written for it, nothing is memoized, and the next update retries it cleanly.

Nested timeouts use the shorter remaining budget:

let outer = ctx.with_timeout(Duration::from_secs(10));
assert_eq!(outer.remaining_deadline(), Some(Duration::from_secs(10)));

// Still bounded by the outer deadline: the earlier one wins.
let wider = outer.with_timeout(Duration::from_secs(20));
// Narrowed to one second.
let narrower = outer.with_timeout(Duration::from_secs(1));

// `outer` itself keeps its own deadline.

The rule

A timeout follows work the caller is directly waiting for. It does not follow background work or shared work.

Foreground child work

Use ctx.scope(..) or use_mount! when the parent depends on the child’s value. The timeout applies to both parent and child:

#[cocoindex::function]
async fn parse_file(ctx: &Ctx, file: FileEntry) -> Result<ParsedDoc> {
    let text = file.content_str()?;
    ctx.check_cancellation()?;
    Ok(parse(&text))
}

app.update(move |ctx| async move {
    let scoped = ctx.with_timeout(Duration::from_secs(30));
    // A foreground child mounted through `scope` receives the caller's deadline.
    let parsed = scoped
        .scope(&"parse", move |child| async move { parse_file(&child, file).await })
        .await?;
    index(parsed);
    Ok(())
})
.await?;

If the deadline passes while parse_file is inside an .await, CocoIndex does not interrupt it. The timeout is observed when control reaches the next CocoIndex check, such as ctx.check_cancellation()?.

Background child work

Use ctx.mount(..) or ctx.mount_each(..) when the child is independent background work. The parent’s timeout can stop the parent from waiting forever, but it does not become the child’s timeout:

let scoped = ctx.with_timeout(Duration::from_secs(30));
let handle = scoped
    .mount(&"first-file", |child| async move {
        // `child` carries no deadline: the parent's budget does not reach it.
        process_file(&child, first).await
    })
    .await?;

// This wait is bounded by the parent's deadline.
// The child processor itself remains independent.
handle.ready().await?;

If the parent times out while waiting for handle.ready(), the timeout belongs to the parent wait. It does not retroactively cancel the child processor.

What happens on timeout

The timing contract is deliberately loose:

• Any CocoIndex-managed call may return Error::DeadlineExceeded if the deadline passes before it returns; err.is_deadline_exceeded() tests for it.
• CocoIndex checkpoints — ctx.check_cancellation(), a memoized #[cocoindex::function] call, engine calls such as ctx.scope(..) and ctx.mount(..) — guarantee to return it if the deadline has already passed when they are called.

Don’t rely on the exact point where the error is returned. It can move between versions, and enforcement can get tighter where cancelling is safe (the built-in retry helpers already cancel an in-flight attempt at the deadline).

What you can rely on is the outcome:

• A processor run that times out applies no target writes and memoizes nothing; the next update retries it cleanly.
• Completed work stays completed: committed components, finished ctx.map(..) items, and in-progress target synchronization are never rolled back or interrupted by a timeout.

Add checks in long processors

CocoIndex already checks common boundaries, but a long processor loop should add explicit checks:

#[cocoindex::function]
async fn import_rows(ctx: &Ctx, rows: Vec<Row>, table: &TableTarget) -> Result<()> {
    for row in rows {
        table.declare_row(ctx, &transform(row))?;
        ctx.check_cancellation()?;
    }
    Ok(())
}

This makes the loop stop near the deadline instead of waiting until the whole function returns.

Long I/O still needs its own timeout

ctx.with_timeout(..) does not replace client-level timeouts: an .await on an HTTP client or a database driver is not interrupted. Give that client its own timeout, and call ctx.check_cancellation()? after it returns.

Think of the two budgets as complementary:

• client timeouts bound one external call
• the Ctx deadline bounds CocoIndex work between safe checks

Time-boxing a whole update (programmatic callers)

When you run an update yourself and the surrounding environment has its own clock (a CI job, a serverless handler, a shared cron window), you can put the budget around the whole update:

let result = app
    .update_with_options(
        UpdateOptions {
            timeout: Some(Duration::from_secs(10 * 60)),
            ..UpdateOptions::default()
        },
        |ctx| async move {
            // ...
            Ok(())
        },
    )
    .await;
if let Err(err) = &result {
    if err.is_deadline_exceeded() {
        println!("stopped after the 10-minute budget");
    }
}

update_blocking_with_options and start_update_with_options take the same UpdateOptions. The root processor’s Ctx carries the timeout.

Because components commit independently and the engine rolls forward, a whole-update timeout never wastes completed work: everything that finished is committed and stays committed; the timeout only stops new progress, and the next update resumes where this one left off. An update with timeout: Some(Duration::from_secs(10 * 60)) means “index as much as possible in ten minutes, cleanly.”

Retries

The built-in API operations (ApiEmbedder, ApiTranscriber, ApiChatClient and the LlmPairResolver built on it) retry transient failures inside one bound per request: the op’s own timeout (10 minutes by default, set with with_timeout), shortened to the Ctx’s remaining deadline when that is earlier. A Ctx deadline can only shorten the bound, never extend it.

Fast failures (HTTP 408, 409, 425, 429, 5xx, a refused or reset connection) are retried with exponential backoff. A request that times out is not retried, and no backoff sleep runs past the bound. When the Ctx deadline is what ran out, the op returns Error::DeadlineExceeded; when the op’s own timeout ran out, it returns an error that names the request.

The SDK has no public retry helper for your own calls. Bound them with the client’s timeout and ctx.remaining_deadline().

API

let scoped = ctx.with_timeout(Duration::from_secs(30));

Returns a clone of the Ctx with the timeout applied. An earlier deadline on the current Ctx wins.

ctx.with_timeout_scope(Duration::from_secs(30), |ctx| async move {
    // Calls in this body use the scoped deadline.
    Ok(())
})
.await?;

Runs a closure with a scoped Ctx. Name the closure argument ctx to shadow the outer context.

ctx.check_cancellation()?;

Returns Error::DeadlineExceeded if the current deadline has passed.

ctx.has_deadline();        // bool
ctx.remaining_deadline();  // Option<Duration>
err.is_deadline_exceeded() // bool, on a cocoindex::Error