Build a Knowledge Graph from YouTube Podcasts with an LLM

Build a pipeline that turns YouTube podcasts into a knowledge graph: extract speakers, statements, and entities with an LLM, then dedupe them with embeddings.

TutorialUpdated 10 min read
Turn podcasts into a knowledge graph. Three YouTube episodes flow into a graph in SurrealDB where Geoffrey Hinton makes a statement that mentions AlexNet (tech) and Google (org). Adding an episode transcribes and extracts that episode only.
Enlarge cover

Build a Knowledge Graph from YouTube Podcasts with an LLM

Zoom in to see details.

Turn podcasts into a knowledge graph. Three YouTube episodes flow into a graph in SurrealDB where Geoffrey Hinton makes a statement that mentions AlexNet (tech) and Google (org). Adding an episode transcribes and extracts that episode only.

An agent that answers questions about a podcast archive has to know who said what, about which technology, and in which episode. A transcript does not give it that structure. Search over transcript chunks returns passages. Listing every guest who discussed large language models across 50 episodes, or matching “GPT4” in one episode to “GPT-4” in another, needs structure that the transcript lacks.

This tutorial builds that structure as a knowledge graph. A CocoIndex app downloads YouTube episodes, transcribes them with speaker labels, uses an LLM to extract statements and the people, technologies and organizations they mention, merges duplicate names across episodes, and stores the result in SurrealDB as a graph.

The code follows the conversation_to_knowledge example (Python package conv_knowledge) on CocoIndex v1. If you are new to knowledge graphs, the documents-to-graph tutorial covers the basics on a smaller pipeline.

Why an agent needs a graph of the archive

  • Statements link to their speakerEach statement links to the person who made it, so an agent can say which guest made a claim.
  • Questions span episodes“Who talked about AlexNet?” is a walk over statement_mentions and person_statement edges.
  • One node per real-world thingEntity resolution merges “GPT4” and “GPT-4” into one node, so counts and joins match across episodes.
  • Adding an episode processes that episodeEach episode is its own memoized component. Adding one transcribes and extracts that episode only.

CocoIndex is an incremental data transformation framework: it records what each function produced, so a re-run processes only new or changed episodes, and it deletes the graph rows of an episode you remove. The demo further down shows what each kind of change re-runs.

What the graph holds

The graph has 5 node types and 4 relationship types:

A session is one podcast episode. A statement is one claim extracted from the conversation, for example “Scaling laws suggest that larger models will continue to improve.” Each statement links to the person who made it and to the entities it mentions.

Person, tech and org are named entities. The same entity often appears under different names across episodes (“GPT-4”, “GPT4”, “OpenAI’s GPT-4”). Entity resolution merges them into one node, as described in phase 2.

The pipeline runs in three phases

  • Phase 1 processes each episode on its own: download, transcribe, then two LLM calls that identify the speakers and extract statements. The session and its statements are written right away, because they do not depend on other episodes.
  • Phase 2 collects the raw entity names (persons, techs, orgs) from all episodes and merges duplicates with embedding similarity and LLM confirmation.
  • Phase 3 writes one node per canonical entity and the relationships that point to those nodes.

Phase 1: process each episode

process_session handles one episode. It fetches the transcript, runs the two extraction steps, and declares the session and its statements:

app.py
@coco.fn(memo=True)
async def process_session(
    youtube_id: str,
    session_table: surrealdb.TableTarget[Any],
    statement_table: surrealdb.TableTarget[Any],
    session_statement_rel: surrealdb.RelationTarget[Any],
) -> SessionRawEntities:
    """Process a single session: fetch, extract (2-step), declare session + statements."""
    transcript = await fetch_transcript(youtube_id)

    # Step 1: format with empty map (no names known yet), then extract metadata
    step1_text = format_transcript(transcript.utterances, {})
    metadata = await extract_metadata(step1_text, transcript)

    # Step 2: format with real names, then extract statements
    speaker_map = {s.label: s.name for s in metadata.speakers}
    step2_text = format_transcript(transcript.utterances, speaker_map)
    stmt_extraction = await extract_statements(step2_text)
    # ... declare the session and its statements (below)

The sections below follow it step by step.

Fetch and transcribe the audio

fetch_transcript downloads the audio with yt-dlp and transcribes it with AssemblyAI. With speaker_labels=True, AssemblyAI returns utterances labeled by speaker (“A”, “B”, …). yt-dlp also returns the YouTube metadata (channel, title, description, upload date), which step 1 uses to name the speakers:

fetch.py
@coco.fn(memo=True)
async def fetch_transcript(youtube_id: str) -> SessionTranscript:
    """Download audio via yt-dlp, transcribe with speaker diarization via AssemblyAI."""
    import yt_dlp

    url = f"https://www.youtube.com/watch?v={youtube_id}"

    with tempfile.TemporaryDirectory() as tmpdir:
        # 1. Download audio + metadata via yt-dlp
        outtmpl = os.path.join(tmpdir, "audio.%(ext)s")
        ydl_opts: dict[str, Any] = {
            "format": "bestaudio/best",
            "outtmpl": outtmpl,
            "postprocessors": [
                {
                    "key": "FFmpegExtractAudio",
                    "preferredcodec": "mp3",
                    "preferredquality": "64",
                }
            ],
            "quiet": True,
            "no_warnings": True,
        }
        with yt_dlp.YoutubeDL(ydl_opts) as ydl:
            info = ydl.extract_info(url, download=True)
        # ... read channel, title, description, upload date; find the .mp3 (actual_path)

        # 2. Transcribe with AssemblyAI (speaker diarization)
        aai.settings.api_key = os.environ["ASSEMBLYAI_API_KEY"]
        config = aai.TranscriptionConfig(
            speaker_labels=True,
        )
        # SDK 0.58.0 enum doesn't include newer models; pass raw string.
        config.speech_models = ["universal-3-pro"]  # type: ignore[assignment]
        transcript = aai.Transcriber().transcribe(actual_path, config)

        if transcript.status == aai.TranscriptStatus.error:
            raise RuntimeError(f"AssemblyAI transcription failed: {transcript.error}")

    # 3. Build structured utterances
    utterances = _extract_utterances(transcript)

    return SessionTranscript(
        utterances=utterances,
        yt_channel=yt_channel,
        yt_title=yt_title,
        yt_description=yt_description,
        yt_upload_date=yt_upload_date,
    )

@coco.fn(memo=True) memoizes the function. On a later run with the same youtube_id and the same function code, CocoIndex returns the stored transcript and does not download or transcribe the audio again. This matters while you iterate on the extraction prompts: re-running an extraction step does not repeat the transcription.

Two LLM passes: speakers first, then statements

The example splits extraction into two LLM calls. To attribute a statement, the LLM has to know who is speaking, and the diarized transcript only says “Speaker A” and “Speaker B”. So the pipeline makes two passes over the transcript. Both use format_transcript(), which replaces each diarization label with a name when one is known:

extract.py
def format_transcript(
    utterances: list[Utterance], speaker_map: Mapping[str, str | None]
) -> str:
    """Format structured utterances into text with speaker names.

    Used by both extraction steps:
    - Step 1: pass empty dict → all speakers shown as "(Speaker A)", ...
    - Step 2: pass mapping from Step 1 → recognized speakers get real names,
              unrecognized stay as "(Speaker X)"
    """
    lines: list[str] = []
    for u in utterances:
        name = speaker_map.get(u.speaker)
        if name is not None:
            lines.append(f"{name}: {u.text}")
        else:
            lines.append(f"(Speaker {u.speaker}): {u.text}")
    return "\n".join(lines)

Step 1: identify the speakers

Step 1 gives the LLM the transcript with generic labels, plus the YouTube channel, title, description and upload date as clues. It returns the session metadata and a name for each speaker it can identify:

extract.py
@coco.fn(memo=True)
async def extract_metadata(
    reformatted_transcript: str, transcript: SessionTranscript
) -> SessionMetadata:
    """Give LLM the reformatted transcript + all YouTube metadata to identify speakers."""
    client = cast(
        instructor.AsyncInstructor,
        instructor.from_litellm(litellm.acompletion, mode=instructor.Mode.JSON),
    )
    result = await client.chat.completions.create(
        model=coco.use_context(LLM_MODEL),
        response_model=SessionMetadata,
        messages=[
            {"role": "system", "content": METADATA_PROMPT},
            {
                "role": "user",
                "content": (
                    f"YouTube channel: {transcript.yt_channel}\n"
                    f"Video title: {transcript.yt_title}\n"
                    f"Description: {transcript.yt_description or 'N/A'}\n"
                    f"Upload date: {transcript.yt_upload_date or 'unknown'}\n\n"
                    f"Transcript:\n{reformatted_transcript}"
                ),
            },
        ],
    )
    # Re-validate to restore class identity for pickling.
    metadata = SessionMetadata.model_validate(result.model_dump())
    # Post-filter: drop speakers whose name is not a plausible person name
    # (e.g. LLM might output "null", "all", "unknown", single words, etc.)
    metadata.speakers = [
        s for s in metadata.speakers if _is_plausible_person_name(s.name)
    ]
    return metadata

The response is a Pydantic model, enforced by instructor. The Field descriptions are part of the JSON schema that instructor sends, so they also guide the LLM:

models.py
class SpeakerIdentification(pydantic.BaseModel):
    """Maps a diarization label to a real person name."""

    label: str = pydantic.Field(
        description='The diarization label from the transcript, e.g. "A", "B".'
    )
    name: str = pydantic.Field(
        description=(
            "Full canonical, Wikipedia-style name of the speaker "
            '(e.g. "Lex Fridman", "Sam Altman"). '
            "Only include speakers you can confidently identify — do not guess."
        )
    )


class SessionMetadata(pydantic.BaseModel):
    """LLM output from Step 1: session metadata + speaker mapping."""

    name: str = pydantic.Field(
        description="A clear, descriptive name for this session or episode."
    )
    description: str | None = pydantic.Field(
        default=None,
        description="A brief 1-2 sentence summary of the session's main topics.",
    )
    date: str | None = pydantic.Field(
        default=None,
        description=(
            "Date of the session in ISO format (YYYY-MM-DD). "
            "Use the date mentioned in the conversation if available; "
            "otherwise fall back to the upload date."
        ),
    )
    speakers: list[SpeakerIdentification] = pydantic.Field(
        description=(
            "Identified speakers: maps each diarization label to the speaker's "
            "real full name. Omit any speaker you cannot confidently identify."
        )
    )

The post-filter in extract_metadata keeps only names of at least two words, so an answer such as “unknown” or “Lex” is not used as a speaker name in step 2 or as a session participant.

Step 2: extract statements with real names

Step 2 reformats the transcript with the names from step 1 (speaker_map in process_session), so “Speaker A” becomes “Lex Fridman”, and asks the LLM for thematic statements:

extract.py
@coco.fn(memo=True)
async def extract_statements(
    reformatted_transcript: str,
) -> StatementExtraction:
    """Extract statements and involved entities from the reformatted transcript."""
    client = cast(
        instructor.AsyncInstructor,
        instructor.from_litellm(litellm.acompletion, mode=instructor.Mode.JSON),
    )
    result = await client.chat.completions.create(
        model=coco.use_context(LLM_MODEL),
        response_model=StatementExtraction,
        messages=[
            {"role": "system", "content": STATEMENTS_PROMPT},
            {
                "role": "user",
                "content": f"Transcript:\n{reformatted_transcript}",
            },
        ],
    )
    # Re-validate to restore class identity for pickling.
    extraction = StatementExtraction.model_validate(result.model_dump())
    # Post-filter: strip speaker labels that leaked through from the LLM
    for stmt in extraction.statements:
        stmt.speakers = [s for s in stmt.speakers if not is_speaker_label(s)]
        stmt.mentioned_person = [
            p for p in stmt.mentioned_person if not is_speaker_label(p)
        ]
    return extraction

Each statement carries the speakers who made it and the entities it is about:

models.py
class RawStatement(pydantic.BaseModel):
    """A thematic claim or statement made during the session."""

    statement: str = pydantic.Field(
        description=(
            "The claim or assertion, written as a clear, standalone sentence. "
            "Do NOT include the speaker's name or attribution "
            '(avoid "X says...", "X believes...", "According to X...") — '
            "the speaker is captured separately."
        )
    )
    speakers: list[str] = pydantic.Field(
        description=(
            "Full canonical names of the speakers who made this statement, "
            "exactly as they appear in the transcript."
        )
    )
    mentioned_person: list[str] = pydantic.Field(
        default=[],
        description=(
            "Full canonical names of people this statement is ABOUT. "
            "Do NOT include the speakers themselves unless the statement is "
            "specifically about them (e.g. their background or credentials)."
        ),
    )
    mentioned_tech: list[str] = pydantic.Field(
        default=[],
        description="Technologies, tools, frameworks, or technical concepts this statement is about.",
    )
    mentioned_org: list[str] = pydantic.Field(
        default=[],
        description="Organizations or companies this statement is about, using their canonical names.",
    )

Every entity name has to be self-contained. The step 2 prompt (STATEMENTS_PROMPT) tells the LLM: “Never use pronouns (“he”, “she”, “they”), speaker labels (“Speaker A”), or contextual references (“the host”, “the guest”, “the interviewer”). Every name must be a clear, unambiguous identifier that stands on its own.” Statements from different episodes are joined later, and a name like “he” or “the host” means nothing outside its own transcript.

Declare the session and its statements

After extraction, process_session declares the session and its statements as records in SurrealDB. CocoIndex uses a declarative model: you describe the target state, and the framework works out the inserts, updates and deletes.

IDs come from IdGenerator, whose IDs are stable across runs: the same calls in the same component return the same IDs, so a re-run does not create duplicates. next_id() with no argument gives the next ID in sequence. next_id(stmt.statement) ties the ID to the statement text, so a statement keeps its ID when the order of statements changes:

app.py
    id_gen = IdGenerator(youtube_id)

    # Declare session node (store the fully-resolved transcript)
    session_id = await id_gen.next_id()
    session = Session(
        id=session_id,
        youtube_id=youtube_id,
        name=metadata.name or transcript.yt_title,
        description=metadata.description,
        transcript=step2_text,
        date=metadata.date or transcript.yt_upload_date,
    )
    session_table.declare_record(row=session)

    # Declare statements + session_statement edges
    identified_stmts: list[IdentifiedStatement] = []
    for stmt in stmt_extraction.statements:
        stmt_id = await id_gen.next_id(stmt.statement)
        statement_table.declare_record(
            row=Statement(id=stmt_id, statement=stmt.statement)
        )
        session_statement_rel.declare_relation(from_id=session_id, to_id=stmt_id)
        identified_stmts.append(IdentifiedStatement(id=stmt_id, raw=stmt))
    # ... return SessionRawEntities(...) with the speakers and statements

Because a statement’s ID comes from its text, a statement that comes back worded differently after a prompt change is a new row, and CocoIndex deletes the old one.

One component per episode

app_main reads the URLs from every .txt file under INPUT_DIR (default ./input) and mounts process_session once per YouTube ID with coco.use_mount(). Each mount is a processing component: a unit of work that CocoIndex tracks on its own, together with the rows it declared:

app.py
    session_coros = []
    async for _key, file in files.items():
        text = await file.read_text()
        for line in text.strip().splitlines():
            line = line.strip()
            if not line or line.startswith("#"):
                continue
            youtube_id = extract_video_id(line)
            session_coros.append(
                coco.use_mount(
                    coco.component_subpath("session", youtube_id),
                    process_session,
                    youtube_id,
                    session_table,
                    statement_table,
                    session_statement_rel,
                )
            )
    all_session_raw = list(await asyncio.gather(*session_coros))

process_session is memoized too, so on a re-run an unchanged episode is skipped as a whole: its body does not run, and the rows it declared last time are kept. A URL you delete from the input files is no longer mounted, and CocoIndex deletes its session, statements and session_statement edges.

Each component returns a SessionRawEntities, the raw names and statements that phases 2 and 3 need:

models.py
@dataclass
class SessionRawEntities:
    """Raw entities from a single session, for entity resolution."""

    session_id: int
    # Maps entity type name → session-level participants (e.g. "person" → identified speakers).
    # Statement-level entities are accessed directly via IdentifiedStatement.raw.
    raw_entities: dict[str, list[str]]
    statements: list[IdentifiedStatement]

Phase 2: merge duplicate entities across episodes

After phase 1, the raw names from all episodes contain duplicates: “GPT-4” and “GPT4”, “Google” and “Google LLC”, “Sam Altman” and “Samuel Altman”. The example merges them with resolve_entities from cocoindex.ops.entity_resolution and its built-in LLM resolver, LlmPairResolver:

app.py
from cocoindex.ops.entity_resolution import resolve_entities
from cocoindex.ops.entity_resolution.llm_resolver import LlmPairResolver
from cocoindex.ops.sentence_transformers import SentenceTransformerEmbedder

# ...
EMBEDDER = coco.ContextKey[SentenceTransformerEmbedder]("embedder", detect_change=True)


@coco.fn(memo=True)
async def _resolve_entities(
    all_raw_entities: set[str],
) -> dict[str, str | None]:
    result = await resolve_entities(
        entities=all_raw_entities,
        embedder=coco.use_context(EMBEDDER),
        resolve_pair=LlmPairResolver(
            model=coco.use_context(RESOLUTION_LLM_MODEL),
        ),
    )
    return result.to_dict()

resolve_entities works in two stages:

  1. It embeds every name (the app provides Snowflake/snowflake-arctic-embed-xs as the embedder) and uses a FAISS index to compare each name, in sorted order, with the canonical names before it. Up to 5 candidates within cosine distance 0.3 (similarity of at least 0.7) count as near-matches.
  2. For each name with candidates, it asks the pair resolver whether the name matches one of them. LlmPairResolver sends the name and its candidates to the LLM, which returns the match and which side should be the canonical name.

A name with no candidate within the threshold never reaches the LLM, so the number of LLM calls follows the number of near-matches. The embeddings and each decision (one name and its candidate list) are memoized. On a later run, only new names are embedded, and LLM calls go to new names and to names whose candidate list changed. The resolver model is set separately from the extraction model, with RESOLUTION_LLM_MODEL (default openai/gpt-5-mini). The result maps each name to the name it merged into, or to None when the name is itself canonical (phase 3 follows the chain to the root), for example {"Google": None, "Google LLC": "Google"}.

The meeting notes graph tutorial uses the same entity resolution with a Neo4j target.

Resolution runs once per entity type (person, tech, org), each as its own component, and the three run concurrently:

app.py
    entity_dedup: dict[str, dict[str, str | None]] = dict(
        zip(
            [cfg.name for cfg in ENTITY_TYPES],
            await asyncio.gather(
                *(
                    coco.use_mount(
                        coco.component_subpath("resolve", cfg.name),
                        _resolve_entities,
                        _collect_all_raw(all_session_raw, cfg.name),
                    )
                    for cfg in ENTITY_TYPES
                )
            ),
        )
    )

Phase 3: write the knowledge graph

With the dedup maps ready, create_knowledge_base declares one node per canonical entity, then every relationship with canonical names. A helper chases the dedup chain to the root name:

models.py
def resolve_canonical(name: str, dedup: dict[str, str | None]) -> str:
    """Chase dedup chains to find the canonical name."""
    while dedup.get(name) is not None:
        name = dedup[name]  # type: ignore[assignment]
    return name
app.py
@coco.fn
async def create_knowledge_base(
    all_session_raw: list[SessionRawEntities],
    entity_dedup: dict[str, dict[str, str | None]],
    entity_tables: dict[str, surrealdb.TableTarget[Any]],
    person_session_rel: surrealdb.RelationTarget[Any],
    person_statement_rel: surrealdb.RelationTarget[Any],
    statement_mentions_rel: surrealdb.RelationTarget[Any],
) -> None:
    """Declare canonical entity nodes and all relationships."""
    # Declare canonical nodes for each entity type (name is the ID)
    for cfg in ENTITY_TYPES:
        dedup = entity_dedup[cfg.name]
        table = entity_tables[cfg.name]
        for name, upstream in dedup.items():
            if upstream is None:
                table.declare_record(row=Entity(id=name, name=name))

    person_dedup = entity_dedup[PERSON_ENTITY_NAME]

    # Declare relationships
    for session_raw in all_session_raw:
        # person_session: person attended session
        for person_name in session_raw.raw_entities.get(PERSON_ENTITY_NAME, []):
            canonical = resolve_canonical(person_name, person_dedup)
            person_session_rel.declare_relation(
                from_id=canonical,
                to_id=session_raw.session_id,
            )

        # person_statement + statement_mentions
        for identified in session_raw.statements:
            stmt = identified.raw
            stmt_id = identified.id
            # person_statement: person made the statement
            seen_speakers: set[str] = set()
            for speaker in stmt.speakers:
                canonical = resolve_canonical(speaker, person_dedup)
                if canonical not in seen_speakers:
                    seen_speakers.add(canonical)
                    person_statement_rel.declare_relation(
                        from_id=canonical, to_id=stmt_id
                    )
            # statement_mentions: deduplicate after resolution
            for cfg in ENTITY_TYPES:
                dedup = entity_dedup[cfg.name]
                table = entity_tables[cfg.name]
                for canonical in {
                    resolve_canonical(e, dedup)
                    for e in getattr(stmt, f"mentioned_{cfg.name}")
                }:
                    statement_mentions_rel.declare_relation(
                        from_id=stmt_id,
                        to_id=canonical,
                        to_table=table,
                    )

statement_mentions is polymorphic: its target can be a person, a tech or an org. app_main mounts it with a list of target tables, and each declare_relation call names the table with to_table:

app.py
    statement_mentions_rel = await surrealdb.mount_relation_target(
        SURREAL_DB,
        "statement_mentions",
        statement_table,
        [entity_tables[cfg.name] for cfg in ENTITY_TYPES],  # polymorphic TO
    )

create_knowledge_base is not memoized, so it runs on every update. It makes no LLM calls, and CocoIndex compares what it declares with the previous run and writes only the difference to SurrealDB.

What re-runs when you change the pipeline

Typical changes are adding or removing episodes, adding an entity type such as Product, or editing a prompt. A memoized function re-runs when its logic (its code, the @coco.fn functions it calls, declared deps), its arguments, or a change-detected context value it reads has changed; otherwise CocoIndex reuses its result. Pick a change below and run the pipeline again:

Change the pipeline, then run it again

Illustrative: episode lengths and counts made up, reuse logic from the example

cocoindex update conv_knowledge.app run 1: first run

First run: every stage runs for episodes A, B and C. 287 minutes of audio are transcribed, 6 extraction calls and 9 entity-resolution calls go to the LLM.

LLM calls, this run
15
total 15
transcription min
287
total 287
graph rows written
228
total 228
graph rows deleted
0
total 0
  1. fetch_transcriptdownload audio, transcribeA ranB ranC ranD ran
  2. extract_metadatastep 1: who is speaking (LLM)A ranB ranC ranD ran
  3. extract_statementsstep 2: statements, entities (LLM)A ranB ranC ranD ran
  4. resolve_entitiesone component per entity typeperson rantech ranorg ranproduct ran
  5. create_knowledge_basedeclares nodes and edgesran +228 rows
SurrealDB yt_conversations: one square per statement
  • session A52 edges
  • session B67 edges
  • session C33 edges
  • session D44 edges

32 entities: person 9, tech 16, org 7

The demo shows these cases:

  • Add an episode. Only the new episode is fetched, transcribed and extracted. Resolution re-runs for each entity type because its set of names changed, but decisions whose name and candidate list are unchanged are memo hits, so LLM calls go to new names and to names whose candidates changed.
  • Remove an episode. Delete its URL from input/sample.txt. Its component is not mounted on the next run, so CocoIndex deletes its session, statements and edges, and the entities that no other episode mentions. Nothing is transcribed or extracted.
  • Add an entity type or edit the step 2 prompt. Step 2 re-runs for every episode; the audio download, transcription and step 1 are memo hits. For 50 episodes that is 50 step 2 calls, plus resolution calls for new name pairs, and no transcription. Statements the LLM words differently get new IDs, so their old rows and edges are replaced.

To add Product, add an EntityTypeConfig to ENTITY_TYPES in models.py and a mentioned_product: list[str] field to RawStatement. STATEMENTS_PROMPT is built from ENTITY_TYPES, so the prompt lists the new type. app_main and create_knowledge_base loop over ENTITY_TYPES, so the new table, its resolution component and its mention edges need no other change.

Declare module-level prompts as dependencies

CocoIndex fingerprints a function’s source code, not the module-level values it reads. STATEMENTS_PROMPT is a module-level string, so editing it, or adding an entity type, does not invalidate extract_statements by itself: the next run returns the memoized statements. Declare the prompt with deps so that a change to it re-runs step 2 (the demo assumes this). The same applies to METADATA_PROMPT and extract_metadata.

python
@coco.fn(memo=True, deps=STATEMENTS_PROMPT)
async def extract_statements(
    reformatted_transcript: str,
) -> StatementExtraction:
    ...

A model change needs no extra step: LLM_MODEL is a context key with detect_change=True, so changing it re-runs both extraction steps.

Run it

Prerequisites

Setup

Clone the example and start SurrealDB:

bash
git clone https://github.com/cocoindex-io/cocoindex.git
cd cocoindex/examples/conversation_to_knowledge

docker run -d --name surrealdb --user root -p 8787:8000 \
  -v surrealdb-data:/data surrealdb/surrealdb:latest \
  start --user root --pass root surrealkv:/data/database

Set the API keys. The other variables are optional and shown with their defaults:

bash
export ASSEMBLYAI_API_KEY="..."   # speaker-diarized transcription
export OPENAI_API_KEY="sk-..."    # LLM extraction via LiteLLM

# Optional (shown with defaults)
export SURREALDB_URL="ws://localhost:8787/rpc"
export SURREALDB_NS="cocoindex"
export SURREALDB_DB="yt_conversations"
export SURREALDB_USER="root"
export SURREALDB_PASS="root"
export INPUT_DIR="./input"
export LLM_MODEL="openai/gpt-5-mini"
export RESOLUTION_LLM_MODEL="openai/gpt-5-mini"

The lifespan reads the SurrealDB and model variables, and provides the embedder and the two model names as context values:

app.py
@coco.lifespan
async def coco_lifespan(
    builder: coco.EnvironmentBuilder,
) -> AsyncIterator[None]:
    surreal_db = surrealdb.ConnectionFactory(
        url=os.environ.get("SURREALDB_URL", "ws://localhost:8787/rpc"),
        namespace=os.environ.get("SURREALDB_NS", "cocoindex"),
        database=os.environ.get("SURREALDB_DB", "yt_conversations"),
        credentials={
            "username": os.environ.get("SURREALDB_USER", "root"),
            "password": os.environ.get("SURREALDB_PASS", "root"),
        },
    )
    builder.provide(SURREAL_DB, surreal_db)
    builder.provide(
        EMBEDDER,
        SentenceTransformerEmbedder("Snowflake/snowflake-arctic-embed-xs"),
    )
    builder.provide(
        LLM_MODEL,
        os.environ.get("LLM_MODEL", "openai/gpt-5-mini"),
    )
    builder.provide(
        RESOLUTION_LLM_MODEL,
        os.environ.get("RESOLUTION_LLM_MODEL", "openai/gpt-5-mini"),
    )
    yield
    await surreal_db.close()

Install the package:

bash
pip install -e .

Add YouTube URLs to input/sample.txt, one per line (# starts a comment):

text
# AI podcasts
https://www.youtube.com/watch?v=VIDEO_ID_1
https://www.youtube.com/watch?v=VIDEO_ID_2

Build the knowledge graph

bash
cocoindex update conv_knowledge.app

Run the same command after you change the URL list or the code. It processes only what changed, as in the demo above.

Explore the graph in SurrealDB

Open Surrealist, SurrealDB’s query and graph UI, and connect to ws://localhost:8787 with namespace cocoindex and database yt_conversations. The graph view of SELECT * FROM statement, person; shows persons in blue and the statements they made or are mentioned in, in pink:

Surrealist graph view of SELECT * FROM statement, person in database yt_conversations: 1054 records and 1154 edges. Person nodes in blue are linked to statement nodes in pink by person_statement and statement_mentions edges.
Surrealist graph view: persons (blue) and statements (pink), joined by person_statement and statement_mentions edgesEnlarge

Surrealist graph view: persons (blue) and statements (pink), joined by person_statement and statement_mentions edges

Zoom in to see details.

Surrealist graph view of SELECT * FROM statement, person in database yt_conversations: 1054 records and 1154 edges. Person nodes in blue are linked to statement nodes in pink by person_statement and statement_mentions edges.

You can also run analytical queries. This one ranks technologies by the number of distinct people whose statements mention them:

surql
SELECT
  name,
  array::distinct(
    <-statement_mentions<-statement<-person_statement<-person.name
  ) AS persons,
  array::len(array::distinct(
    <-statement_mentions<-statement<-person_statement<-person.id
  )) AS person_count
FROM tech
ORDER BY person_count DESC
LIMIT 15;
Surrealist table of technologies mentioned by the most people: Artificial intelligence (12), Language model (9), Machine learning (7), then AI agent, Algorithm, Artificial neural network, Computation, Software engineering and Superintelligence (6 each), then AI safety, AlexNet, Artificial general intelligence and ChatGPT (5 each).
Result of the query above: technologies ranked by how many people mention themEnlarge

Result of the query above: technologies ranked by how many people mention them

Zoom in to see details.

Surrealist table of technologies mentioned by the most people: Artificial intelligence (12), Language model (9), Machine learning (7), then AI agent, Algorithm, Artificial neural network, Computation, Software engineering and Superintelligence (6 each), then AI safety, AlexNet, Artificial general intelligence and ChatGPT (5 each).

More queries to start from:

surql
-- All statements a person made
SELECT <-person_statement<-person.name AS speaker, statement FROM statement;

-- Full graph around a person
SELECT name,
  ->person_session->session.name AS sessions,
  ->person_statement->statement.statement AS statements
FROM person;

-- All entities involved in statements
SELECT statement,
  ->statement_mentions->person.name AS persons,
  ->statement_mentions->tech.name AS techs,
  ->statement_mentions->org.name AS orgs
FROM statement;

Summary

The pipeline turns YouTube episodes into a graph an agent can query:

  1. Fetch: download the audio and transcribe it with speaker labels.
  2. Extract in two LLM steps: name the speakers from the YouTube metadata, then extract statements from the transcript with real names.
  3. Resolve: merge duplicate entity names across episodes with resolve_entities, where embeddings find candidates and an LLM confirms near-matches.
  4. Store: declare the nodes and edges in SurrealDB; CocoIndex writes only the difference from the previous run.

Each episode is a memoized component and each expensive step is memoized, so adding an episode or changing a prompt (once declared with deps) re-runs only the work it affects. The full code is in examples/conversation_to_knowledge, and the docs version of this tutorial has the same walkthrough.

Frequently asked questions

How do I turn a podcast into a knowledge graph?

Build a CocoIndex pipeline that downloads the episode audio, transcribes it with speaker diarization, uses an LLM to extract structured statements and entities, resolves duplicate entities across episodes, and stores everything in SurrealDB as a graph. The graph uses five node types (session, statement, person, tech, org) connected by four relationship types. See What the graph holds.

Why use a two-step LLM extraction for speaker attribution?

To attribute a statement, the LLM needs to know who each speaker is, but the diarized transcript only has generic labels like "Speaker A". Step 1 identifies the speakers and extracts session metadata, using the YouTube channel, title, description and upload date as clues. Step 2 reformats the transcript with the real names and extracts thematic statements. See Two LLM passes.

How do I deduplicate entities across documents in a knowledge graph?

The example calls resolve_entities from cocoindex.ops.entity_resolution with an embedder and LlmPairResolver. It embeds each entity name, uses a FAISS index to find up to 5 candidates within cosine distance 0.3, and asks the LLM to confirm a match and pick the canonical name only for names that have candidates. The embeddings and each decision are memoized across runs. See Phase 2: merge duplicate entities.

How does CocoIndex make a knowledge-graph pipeline incremental?

Two mechanisms. @coco.fn(memo=True) memoizes expensive functions by their code and inputs, so fetching, transcribing and extraction are skipped for unchanged episodes. coco.use_mount() processes each episode as an independent component, so adding a YouTube URL processes only that episode, and removing one deletes its session, statements and relationships. See What re-runs when you change the pipeline.

What happens to a knowledge-graph pipeline when I change the extraction schema?

Only the affected steps re-run. If you add an entity type like Product, audio download, transcription and speaker identification are reused, statement extraction re-runs for every episode, and entity resolution runs fresh for the new type while the other types reuse every decision whose name and candidate list did not change. The prompt is a module-level string, so declare it on the function with deps=STATEMENTS_PROMPT; otherwise CocoIndex does not see the change. See What re-runs when you change the pipeline.

How do I generate stable IDs so re-running the pipeline doesn't create duplicates?

Use CocoIndex's IdGenerator, whose IDs are stable across runs: the same calls in the same component return the same IDs. next_id() gives the next ID in sequence, and next_id(content) ties the ID to the content, so a statement keeps its ID when the order of statements changes. See Declare the session and its statements.

How do I query the resulting knowledge graph in SurrealDB?

Connect SurrealDB's Surrealist UI (namespace cocoindex, database yt_conversations) to view the graph, or run SurrealQL queries, for example ranking technologies by how many people mention them by traversing the statement_mentions and person_statement edges. See Explore the graph in SurrealDB.

Build fresh context for your agents.

CocoIndex keeps derived data in sync with its sources. Only what changed re-runs.