Progress monitoring

Beyond report_to_stdout — read structured update stats from an UpdateHandle with detailed_stats_snapshot() and changed(), and split a run into separately-reported scopes with Ctx::stats_group().

Version

v 1.0.14

Last reviewed

Nov 4, 2026

Most runs only need the basics covered in App: awaiting app.update(..) for the result, or the built-in stdout progress display — UpdateOptions { report_to_stdout: true, .. } on app.update_with_options(..). progress_refresh_interval: Some(duration) sets the refresh interval (None uses the default). The cocoindex update command renders the same per-component lines on stderr when stderr is a terminal, and prints the final [Stats] line on stdout; -q / --quiet turns the stdout output off.

Use the structured progress APIs to stream progress from a daemon to a client, build a custom dashboard, or report pipeline phases separately.

Structured update stats

app.start_update(f) returns an UpdateHandle<T>. It starts the update at once; handle.result().await waits for the value. The handle exposes the same stats that drive the terminal display: detailed_stats_snapshot() (per-component UpdateStats), stats_snapshot() (the coarse RunStats), and changed().await to wait for the next change.

Polling

handle.detailed_stats_snapshot() returns a snapshot of the current counters as an UpdateStats. It keeps working after completion, returning the final stats:

let mut handle = app.start_update(|ctx| async move {
    // ...
    Ok(42)
})?;

loop {
    if handle.changed().await?.is_done() {
        break;
    }
}
let final_stats = handle.detailed_stats_snapshot(); // UpdateStats with the final counters
let result = handle.result().await?;                // consumes the handle

detailed_stats_snapshot() works at any time, including before the first change. Read the final stats before result(), which consumes the handle.

To read stats while an update is in flight, loop on changed() (below) and read a snapshot after each change.

Streaming

handle.changed().await waits until the stats change. It returns Progress::Changed(version) while the update runs, then Progress::Done once it terminates; result() then returns the value:

let mut handle = app.start_update(|ctx| async move {
    // ...
    Ok(())
})?;
loop {
    let progress = handle.changed().await?; // Progress::Changed(version) or Progress::Done
    let stats = handle.detailed_stats_snapshot();
    println!("{} items processed", stats.total().num_finished());
    // stats.status is UpdateStatus::Running or UpdateStatus::Ready
    if progress.is_done() {
        break;
    }
}
let result = handle.result().await?;

There is no snapshot object in Rust: changed() wakes you, and detailed_stats_snapshot() reads the current counters.

On a processing error, changed() and result() return the error.

In live mode, the update does not terminate at the first ready state: changed() keeps returning Progress::Changed as live components deliver incremental updates, and returns Progress::Done only when the app stops.

Stats types

pub struct UpdateStats {
    pub by_component: BTreeMap<String, ComponentStats>, // keyed by processor name
    pub status: UpdateStatus,                           // Running | Ready
    pub in_groups: ComponentStats,                      // what stats groups under this scope count
}
// UpdateStats::total() sums every component, not in_groups; run_stats() derives the coarse RunStats.

pub struct ComponentStats {
    pub num_execution_starts: u64,
    pub num_unchanged: u64,
    pub num_adds: u64,
    pub num_deletes: u64,
    pub num_reprocesses: u64,
    pub num_errors: u64,
}
// Derived helpers: num_processed(), num_finished(), num_in_progress(), has_errors().

pub enum Progress {
    Changed(u64), // the update advanced; carries a monotonic version
    Done,         // the update terminated
}

stats_snapshot() returns RunStats { processed, skipped, written, deleted, elapsed }, the coarse view that App::run returns.

Scoped reports with stats_group

By default every component’s stats roll up into one report for the whole update. For a large pipeline you often want to watch a part of it on its own — “indexing the docs tree” separately from “indexing the code tree”, or one expensive phase by itself.

ctx.stats_group(title, f) opens a scope: f runs with a scoped Ctx, and everything mounted from that Ctx aggregates into a separate report under title, split out of the parent (the parent report no longer counts that work — there’s no double counting). It returns the closure’s value and a StatsGroupHandle:

let (sum, handle) = ctx
    .stats_group("Indexing docs", |group_ctx, _handle| async move {
        let values = group_ctx
            .use_mount_each(
                vec![1, 2],
                |item| *item,
                |child, item| async move {
                    child
                        .memo(&item, move |_ctx| async move { Ok(item * 10) })
                        .await
                },
            )
            .await?;
        Ok::<_, cocoindex::Error>(values.into_iter().sum::<i32>())
    })
    .await?;

// The same group, with its progress printed to stdout:
let options = cocoindex::StatsGroupOptions {
    report_to_stdout: true,
    refresh_interval: Some(Duration::from_millis(50)),
};
let ((), reported_handle) = ctx
    .stats_group_with_options("Indexing docs (reported)", options, |group_ctx, _handle| async move {
        // mount from `group_ctx` ...
        Ok::<_, cocoindex::Error>(())
    })
    .await?;

With report_to_stdout: true, the group’s progress is also printed to stdout, labeled by its title, alongside the main report_to_stdout display without disrupting it. refresh_interval: Some(duration) sets the group’s refresh interval; while a terminal display runs, the group is a section of that display and redraws with it.

A failure in a group, reported or not, still counts in the final status line of the main display: the run ends in ✗ Done in 2.0s · 3 failed, even though the group’s rows stay out of the main report.

The StatsGroupHandle has the same stats_snapshot(), detailed_stats_snapshot() and changed() as UpdateHandle (the closure’s value is returned beside it, so there’s no result()):

// `group_handle` is the StatsGroupHandle returned beside the closure's value.
let mut group_handle = group_handle;
loop {
    if group_handle.changed().await?.is_done() {
        break;
    }
    send_to_dashboard(group_handle.detailed_stats_snapshot());
}
let stats = group_handle.stats_snapshot();

The closure also receives the handle as its second argument, so the body can read the group’s stats while it runs.

Semantics

• Non-blocking. Returning from the closure does not wait for background work mounted inside it (mount, mount_each) to finish — it only marks where member registration stops. The group becomes ready asynchronously once the closure has returned and every component mounted inside it is ready; observe that via handle.changed() / handle.detailed_stats_snapshot().

• Identity is unchanged. Grouping only redirects where stats are reported. Component paths, change detection, target-state ownership, and the run’s overall completion are unaffected — the grouped components are still ordinary children of the surrounding component.
• Nesting. Groups may nest. The innermost group owns a mount’s stats; an outer group’s readiness still waits for the inner group’s work to finish.
• Live members. A group containing live components becomes ready after their initial catch-up and reports their ongoing incremental stats, just like the root.