Slides-to-speech: Bring your slide decks to life with narrated audio

Presentation slide decks are information-rich, dense sources of company knowledge, containing everything from quarterly updates, technical specs, sales reports, and training materials. These decks are locked away in PDF files that get revised constantly and are generally hard to consume in their static form.

What if we could turn those decks into a continuously updated, searchable, and narrated (audio) knowledge base? Doing that would result in a few immediate benefits:

• Listen to any slide deck instead of reading every slide (useful for busy executives on the go, and for people with visual impairments).
• Extract insights from slide content as structured speaker notes by using a VLM (vision language model).
• Search slide content semantically, not just by keywords or filenames.

Producing those artifacts once is only half the task. When a deck changes, its speaker notes, audio, and embeddings also have to be kept fresh. Once everything is in sync, it can lay the foundation for more advanced user experiences later, including video generation from text.

In this post, we’ll show how to turn the slide deck’s assets into a multimodal knowledge base stored in LanceDB: the same dataset contains the slide content, speaker notes, MP3 audio narration, and embeddings for vector search, all of which are orchestrated and kept fresh by CocoIndex.

Introducing CocoIndex

CocoIndex is an ultra-performant framework that keeps context continuously fresh for AI Agents. Think “React for data processing.” You declare state transformations, and CocoIndex handles the processing logic: it continuously watches for changes, recomputes only what’s necessary, and applies minimal updates downstream for end-to-end freshness. As AI systems become increasingly autonomous, the bottleneck is no longer model capability: it’s keeping context fresh and relevant.

CocoIndex follows an incremental update philosophy:

• New slide decks appear in the source folder
• Updated files get reprocessed automatically
• Only affected rows get new embeddings and regenerated audio, if needed

Introducing LanceDB

LanceDB is a multimodal lakehouse for AI, available as an open-source, embedded, developer-friendly retrieval library that pairs well with CocoIndex. Under the hood, it’s powered by the Lance columnar format, which is designed for fast random access (great for search workloads) without sacrificing scan performance (great for large-scale aggregation queries).

For this Slides-to-Speech pipeline, that translates into a few concrete benefits:

• Multimodal in one table: store speaker notes, MP3 audio bytes, embeddings, and metadata side by side as a single dataset.
• Indexes are first-class: vector search, full-text search (FTS), and secondary indexes are stored alongside your dataset, so you don’t need to manage those using separate systems.
• Data evolution: add or refine columns over time (backfill new embeddings, quality tags, moderation fields) without touching unmodified columns, which is great for feature engineering and experimentation.
• Local-first simplicity: run it as an embedded database with minimal operational overhead.

CocoIndex’s LanceDB connector automatically appends and updates rows as new results arrive via the CocoIndex pipeline, so you don’t have to rewrite your table (or rebuild your indexes) from scratch each time content changes.

What we are building: one slide, three derived modalities

To begin building our pipeline, let’s first get the shape of the data clear. A PDF slide deck doesn’t just become one large database record. We first fan it out page by page, then turn each slide into three related outputs: speaker notes, MP3 narration (audio), and a text embedding for semantic search.

The speaker notes sit at the center of the pipeline, but they first need to be generated by a vision model that reasons over the slide content. Once they exist, audio generation and embedding can run on them concurrently. The completed outputs are then declared together as one SlideRecord, identified by its filename and page number.

Each tool has one focused job:

Most of the pipeline runs locally on CPU: PyMuPDF renders the pages as images for the VLM, Pocket TTS synthesizes audio from the speaker notes, sentence-transformers generates text embeddings, and LanceDB runs as an embedded, local retrieval library.

Start backwards from the target

CocoIndex pipelines are easiest to reason about backwards from the target, i.e., where the data sits in its final form. Put simply, target_state = transformation(source_state), where transformation can be any function, or a set of functions. For this pipeline, the target state is one clean LanceDB row for each slide.

# Declare the LanceDB table shape via a strictly typed schema
# Pydantic or PyArrow schemas work well, but dataclasses are simpler for this example
@dataclass
class SlideRecord:
    id: str
    filename: str
    page: int
    speaker_notes: str
    voice: bytes
    embedding: Annotated[NDArray, EMBEDDER]

# Mount a LanceDB table as the target
table = await lancedb.mount_table_target(
    LANCE_DB,
    table_name="slides_to_speech",
    table_schema=await lancedb.TableSchema.from_class(
        SlideRecord,
        primary_key=["id"],
    ),
)

Multimodal data is a first-class citizen in LanceDB. Each row can hold the slide’s metadata, generated notes, MP3 narration bytes, and vector embedding side by side. SlideRecord is the typed contract that keeps those values together, via a dataclass.

Each slide needs a stable primary key so CocoIndex can match it to the same LanceDB row across runs, updating or removing the correct record as the source changes. This example derives that key from the filename and page number: page 3 of quarterly_review.pdf becomes quarterly_review.pdf#3.

Functions vs. processing components

Incremental processing is only useful when its boundaries match the unit of change. We want our pipeline to run every time a change is detected in the source files, without wastefully recomputing what didn’t change.

In CocoIndex, functions define the work, while processing components group that work into independently synchronized units. Where we draw those component boundaries determines how much of the pipeline runs each time a change is detected, and which target states CocoIndex reconciles afterward.

• Functions are callable Python functions decorated with @coco.fn. Setting memo=True lets CocoIndex reuse results for unchanged inputs and code. Several functions can run inside the same processing component.
• Processing components are created by grouping functions and mounting them to the target. They manage the necessary function calls and the target states they declare. Processing components help decide the granularity at which CocoIndex reruns work and synchronizes the result.

In this example, we define the processing component at the per-slide level, through the process_file function. It takes a PDF file, renders each slide as an image, and mounts one process_slide function for each slide.

@coco.fn(memo=True)
async def process_file(
    file: FileLike,
    table: lancedb.TableTarget[SlideRecord],
) -> None:
    slides = await pdf_to_slides(await file.read())
    await coco.mount_each(
        process_slide,
        ((slide.page_number, slide) for slide in slides),
        str(file.file_path.path),
        table,
    )

process_file is memoized too, so CocoIndex can skip reading and rendering an unchanged PDF. If the file does change, process_slide provides the finer-grained memoization boundary: slide images with unchanged inputs reuse their prior results, while changed slides go through note generation, TTS, embedding, and the declaration of SlideRecord in the LanceDB target based on its stable key.

Changing a slide in a PDF still reruns page rendering for that entire file (still, PyMuPDF is a cheap CPU-based operation, so no VLM calls happen yet). Once the images are available, CocoIndex fingerprints files with a content-based hash, so it processes only slide images whose contents changed from the previous state. This is the power of memoization and the reason we mount the component at the slide level.

If a PDF is deleted from the source folder, CocoIndex unmounts its per-file component and recursively cleans up the target states it owns. In this pipeline, that means deleting the PDF’s slide rows from LanceDB.

Let’s now go over each stage of the pipeline below.

Fan a PDF out into slides

The function pdf_to_slides turns the PDF bytes into typed SlidePage values, each carrying the page number and the rendered PNG that the vision model will inspect.

@dataclass
class SlidePage:
    page_number: int
    image: bytes

@coco.fn.as_async(runner=coco.GPU, memo=True)
def pdf_to_slides(content: bytes) -> list[SlidePage]:
    import pymupdf

    slides: list[SlidePage] = []
    doc = pymupdf.open(stream=content, filetype="pdf")
    for i, page in enumerate(doc):
        pix = page.get_pixmap(matrix=pymupdf.Matrix(2, 2))
        slides.append(
            SlidePage(page_number=i + 1, image=pix.tobytes("png"))
        )
    doc.close()
    return slides

PyMuPDF renders each page at 2× scale, so the vision model sees a legible PNG rather than extracted PDF text. The image is an intermediate input: Gemini uses it to generate notes, but the pipeline discards it afterward. If you want to keep the rendered image for reuse, you can add an image field and store its bytes in the same LanceDB table.

Turn a slide image into narration

Extracting speaker notes from the slide images gives the Pocket TTS model the text it needs to turn the script into an audio file that narrates the insights from that slide. Text embeddings are also generated from the speaker notes, so they can support semantic search, allowing you to easily find slides based on their meaning rather than just keywords in their titles or filenames.

class SlideNotes(dspy.Signature):
    """Write natural spoken narration for a slide, as a presenter would say it aloud."""

    slide: dspy.Image = dspy.InputField(desc="the rendered slide image")
    speaker_notes: str = dspy.OutputField(
        desc="a few sentences of spoken narration — no markdown or bullet symbols"
    )

_speaker_notes = dspy.Predict(SlideNotes)

@functools.cache
def _get_lm(model: str) -> dspy.LM:
    return dspy.LM(model, max_tokens=8192)

@coco.fn(memo=True)
async def extract_speaker_notes(image: bytes) -> str:
    data_url = "data:image/png;base64," + base64.b64encode(image).decode()
    with dspy.context(lm=_get_lm(coco.use_context(LLM_MODEL))):
        result = await _speaker_notes.acall(slide=dspy.Image(url=data_url))
    return result.speaker_notes

SlideNotes is a DSPy signature, a typed contract that tells the VLM what we need. When a slide image comes in, speaker notes come out. The CocoIndex function extract_speaker_notes does the actual work of turning the PNG bytes into a text output.

This extraction function is memoized because speaker-note generation is an expensive dependency: it calls a capable VLM to reason over the image. Its memoization key includes the rendered image and the change-detected LLM_MODEL context, so an unchanged slide can reuse its notes if the image bytes and the model remain the same. If either of them changes, it’s tracked by CocoIndex and the work is rerun for that slide.

Turn speaker notes into audio and embeddings

We can now inspect the function process_slide and the work it does, by reading it top to bottom. The speaker notes must first exist, so they are computed and awaited via extract_speaker_notes. Once they exist, audio synthesis and embedding have no dependency on each other, so we call asyncio.gather to run them concurrently. CocoIndex commits them to the LanceDB table only after both are ready.

@coco.fn(memo=True)
async def process_slide(
    slide: SlidePage,
    filename: str,
    table: lancedb.TableTarget[SlideRecord],
) -> None:
    notes = await extract_speaker_notes(slide.image)
    voice, embedding = await asyncio.gather(
        text_to_speech(notes, coco.use_context(TTS_VOICE)),
        coco.use_context(EMBEDDER).embed(notes),
    )
    table.declare_row(
        row=SlideRecord(
            id=f"{filename}#{slide.page_number}",
            filename=filename,
            page=slide.page_number,
            speaker_notes=notes,
            voice=voice,
            embedding=embedding,
        )
    )

The target state is declared as a SlideRecord dataclass (per the LanceDB schema) keyed by its unique ID that combines the filename and page number. This is how CocoIndex reconciles the target state with the source files, updating or removing rows as needed.

The expensive nested operations are memoized as well. If only the voice changes, CocoIndex can reuse the notes and embedding while regenerating the audio. If the embedding model changes, it can reuse the notes and audio while computing a new vector. This gives us reuse at the file, slide, and individual transformation levels.

Define the app

Apps are the top-level runnable unit in CocoIndex. They name your pipeline and bind a main function with its parameters. When you call app.update(), CocoIndex runs that main function as the root processing component that can mount processing components (like our process_file and its child component process_slide) that do work and declare target states.

app_main mounts the SlideRecord table we defined earlier, uses localfs.walk_dir in live mode to discover PDFs, then calls the outer mount_each. The app is configured with a name, the main function, and the source directory to watch for changes.

@coco.fn
async def app_main(sourcedir: pathlib.Path) -> None:
    table = await lancedb.mount_table_target(
        LANCE_DB,
        table_name="slides_to_speech",
        table_schema=await lancedb.TableSchema.from_class(
            SlideRecord,
            primary_key=["id"],
        ),
    )
    files = localfs.walk_dir(
        sourcedir,
        recursive=True,
        path_matcher=PatternFilePathMatcher(
            included_patterns=["**/*.pdf"]
        ),
        live=True,
    )
    await coco.mount_each(process_file, files.items(), table)

app = coco.App(
    coco.AppConfig(name="SlidesToSpeech"),
    app_main,
    sourcedir=pathlib.Path("./slides"),
)

Run the pipeline and run semantic search

Prerequisites

• Python 3.11+
• A Gemini API key for the default vision model
• ffmpeg for MP3 export

Pocket TTS is open source and runs locally on the CPU. Its weights download on the first run, so it needs neither a TTS API key nor a GPU.

Quick start

From the example directory, copy the environment file, set GEMINI_API_KEY, and ensure LLM_MODEL=gemini/gemini-2.5-flash to match the example’s default configuration. Then install and run:

cp .env.example .env
uv sync
cocoindex update main

# Keep watching slides/ for changes instead:
cocoindex update -L main

Search the deck

python main.py "reducing latency and reliability"

The query helper embeds the question with the same sentence-transformer, then asks LanceDB for its nearest neighbors. The results already arrive ranked, so we can print the matching slide and its notes directly:

for row in table.search(vec).limit(5).to_list():
    print(f"{row['filename']} slide {row['page']}")

The slide deck we use here is an example “quarterly-review” deck, so the query about “reducing latency and reliability” brings the slide about “Engineering Priorities” to the top because its notes discuss “reducing p95 latency and multi-region resilience”.

The returned row can also fetch the voice bytes, so a downstream application can use those bytes as audio/mpeg, write them to an MP3 file for playback, and play them on demand if needed.

Here is the narration the pipeline generated for the sample deck, one MP3 per slide:

How CocoIndex tracks changes

In this demo, CocoIndex watches the ./slides directory for changed PDFs. It then identifies the changed slides, and only those slides regenerate notes, audio, and embeddings. As the table above shows, it can track much more than file changes.

However, source files changing is only one such scenario. The table below shows the other changes that CocoIndex tracks, and what they mean for other pipelines like this one.

Bring your slides to life (and beyond)

The pipeline shown here is a concrete example of how to bring slide decks to life via narration. But don’t just think of organizational data as structured tables and flat text. A large amount of company intelligence is locked away in richer artifacts like PDF reports, dashboards, and video recordings.

With CocoIndex and LanceDB, we can turn all of that raw multimodal content into a living, breathing, contextually rich knowledge base that agents can retrieve from:

• Unify multimodal content in one storage layer: text, audio, vectors, and metadata in the same table.
• Add structure that supports more questions: speaker notes, extracted fields, embeddings, tags, and other derived signals.
• Keep everything synchronized over time: incremental recomputation when source files change, with updated rows in LanceDB.
• Build user-facing experiences on top: semantic search, narrated slide playback, assistants, and video-generation pipelines.

Try the complete slides-to-speech example to bring your own deck to life! Then, star CocoIndex and LanceDB on GitHub.

CocoIndex

Fresh context for long-horizon agents.

Learn the concept → View on GitHub

About the authors.

Prashanth Rao AI Engineer @ LanceDB · @tech_optimist

AI Engineer at LanceDB. Writes about vector search, knowledge graphs, and retrieval at thedataquarry.com.

Linghua Jin Cofounder & CEO of CocoIndex · @linghuaj

CEO and cofounder of CocoIndex. Writes about incremental data infrastructure, agents, and the engineering decisions behind the engine.