Linghua Jin
Cofounder & CEO of CocoIndex
35 posts
How Tree-sitter Works: Parsing Code as You Type
Grammars, concrete syntax trees, GLR parsing, error recovery and queries, then how CocoIndex splits code into chunks on function and class boundaries.
Codebase Indexing for AI Agents with CocoIndex
Index a codebase for RAG and AI coding agents with CocoIndex V1 and Tree-sitter: language-aware chunking, embedding, and a live vector index in async Python.
CocoIndex V1 is Live!
CocoIndex V1 is live: a ground-up redesign of incremental data pipelines, built for AI engineers and agent builders shipping RAG, memory, and knowledge graphs.
CocoIndex joins the GitHub Secure Open Source Fund
CocoIndex joined the GitHub Secure Open Source Fund, hardening the AI data infrastructure developers depend on with threat modeling, CodeQL, and audits.
Hybrid search over SEC EDGAR filings with Apache Doris
Index SEC 10-K text and XBRL facts into one Apache Doris table with CocoIndex v1, then rank each chunk by meaning and by keywords, fused with RRF.
Generate Codebase Documentation with an LLM, Kept Current
Generate a Markdown wiki page for every Python project in a folder with an LLM and CocoIndex v1. On each run, the LLM re-reads only the files that changed.
Turn PDF Slides into Narrated Audio with Gemini and TTS
Build a CocoIndex v1 pipeline that turns PDF slides into vision-generated speaker notes, local Pocket TTS narration, and searchable LanceDB records.
Extract Patient Intake Forms with DSPy and CocoIndex
Extract Pydantic-typed patient records from PDF intake forms with DSPy and Gemini vision into Postgres with pgvector, and see which edits re-run extraction.
Build a Knowledge Graph from Meeting Notes with Neo4j
Build a self-updating Neo4j knowledge graph from meeting notes with CocoIndex v1: LLM-extracted decisions, tasks, and owners, with embedding-based dedupe.
Normalize and rank Hacker News topics with an LLM
Normalize free-text topics with an LLM prompt and rank them with a weighted SQL score in Postgres, in an incremental CocoIndex v1 pipeline over Hacker News.
Turn a polling API into an incremental source: Hacker News
Turn a polling API into an incremental CocoIndex v1 source in plain Python, using Hacker News: list keys, fetch by key, own rows, survive failed fetches.
Extracting Intake Forms with BAML and CocoIndex
Use BAML and CocoIndex v1 to extract typed patient records from PDF intake forms into Postgres with pgvector, and avoid the role("user") PDF gotcha on OpenAI.
AI-native data pipelines: why we built CocoIndex
Why AI applications need open, AI-native data pipelines that keep context fresh as sources and code change, and the design choices behind CocoIndex.
Multimodal PDF Search: Embed Text and Images with CLIP
Split PDFs into text chunks and images, embed text with SentenceTransformers and images with CLIP, and search both in Qdrant with the page number on every hit.
Write a Custom Source in Python: Index Any API
Index an internal API with CocoIndex v1: plain async Python, deletes propagate, and a memo on updated_at skips unchanged items.
Trace search results back to source data
Define query handlers in CocoIndex and trace search results back to source data in CocoInsight to close the loop on indexing strategy.
Add Semantic Search to a Postgres Table with pgvector
Use an existing PostgreSQL table as a CocoIndex source: derive fields, embed each row, and store the vectors in Postgres with pgvector, incrementally.
Search PDFs and Images Without OCR Using ColPali and Qdrant
Index PDFs and images into one Qdrant collection with CocoIndex v1 and ColPali multi-vector embeddings. Each page is embedded as an image, so no OCR is needed.
CocoIndex Changelog 2025-08-18
CocoIndex updates: production readiness, scalability, and reliability, plus more customization, native integrations, and multi-modal pipeline features.
Multi-vector image search with ColPali and Qdrant
Index images with ColPali multi-vector patch embeddings and a Qdrant MaxSim collection using CocoIndex v1: incremental, live, in plain async Python.
Multi-Dimensional Vector Support in CocoIndex
CocoIndex v1 describes vector and multi-vector columns with VectorSchema and MultiVectorSchema, so Qdrant gets dense vectors and MaxSim multi-vectors.
Write a Custom Target: Export Indexed Data Anywhere
Export CocoIndex data anywhere: implement a TargetHandler with a tracking record and an action sink; the engine handles diffing, syncing, and cleanup.
Search your photos by face with CocoIndex
Detect every face in a photo folder, embed each with dlib, and index one Qdrant point per face with CocoIndex v1, so a face finds the photos it appears in.
Extract Metadata from Research Paper PDFs with an LLM
Index academic papers with CocoIndex: read the first page of each PDF, extract title, authors, and abstract with an LLM, and embed them for semantic search.
Introducing CocoInsight
CocoInsight, the CocoIndex v0 tool for inspecting pipeline data and lineage step by step. It is not part of CocoIndex v1; v1 apps use cocoindex show.
Schema inference for Qdrant collections
CocoIndex v1 creates your Qdrant collection from the code: the vector size comes from the embedding model, and a schema change rebuilds the collection.
CocoIndex + Kuzu: Real-time knowledge graph with Kuzu
Build a real-time knowledge graph with Kuzu as a native CocoIndex target: incremental updates, high-performance graph queries.
Incremental ETL from Amazon S3 to Postgres pgvector
Index Markdown from Amazon S3 into Postgres pgvector with CocoIndex v1. A re-run reads only the objects that changed and embeds only the chunks with new text.
Build Text-to-Image Search with CLIP and Qdrant
Build live image search with CocoIndex v1, CLIP, FastAPI, and Qdrant. Embed photos and text in one vector space and search by meaning.
What Are Text Embeddings? Build Semantic Search in Python
Text embeddings 101: what an embedding is, how similarity works, why you chunk, and how to build a semantic search index with CocoIndex v1 and pgvector.
Story of CocoIndex, at 1k stars
The story of CocoIndex at 1,000 GitHub stars: the open-source engine that combines custom transformation logic with incremental processing for data indexing.
Build a Product Recommendation Engine with an LLM and Neo4j
An LLM extracts each product's taxonomy and complementary taxonomy; CocoIndex turns the labels into a Neo4j knowledge graph that answers 'bought this, also need…'.
Build a Knowledge Graph from Documents with an LLM and Neo4j
Build a Neo4j knowledge graph from Markdown docs with an LLM and CocoIndex v1. An edit re-extracts only that file, and unsupported nodes and edges are deleted.
What Is Incremental Processing? Keep Indexes Fresh
Incremental processing re-runs only the work a change affects. See how CocoIndex v1 finds changes, reuses memoized results and deletes stale rows.
Welcome to CocoIndex
CocoIndex keeps derived data (chunks, embeddings, graphs, summaries) in sync with its sources and code, so AI agents read current context.