Index a codebase for RAG and AI coding agents with CocoIndex V1 and Tree-sitter: language-aware chunking, embedding, and a live vector index in async Python.
CocoIndex V1 is live: a ground-up redesign of incremental data pipelines, built for AI engineers and agent builders shipping RAG, memory, and knowledge graphs.
CocoIndex joined the GitHub Secure Open Source Fund, hardening the AI data infrastructure developers depend on with threat modeling, CodeQL, and audits.
A multi-format CocoIndex v1 pipeline that ingests SEC filings (TXT, JSON), scrubs PII, extracts topics, and powers hybrid search with Apache Doris.
Auto-generate documentation for every project in your codebase: a CocoIndex pipeline writes a wiki page per repo with an LLM, kept fresh as code changes.
Build a CocoIndex v1 pipeline that turns PDF slides into vision-generated speaker notes, local Pocket TTS narration, and searchable LanceDB records.
Extract Pydantic-typed structured data from patient intake forms using DSPy and CocoIndex: OCR vision models with incremental processing.
Build a self-updating Neo4j knowledge graph from meeting notes with CocoIndex v1: LLM-extracted decisions, tasks, and owners, with embedding-based dedupe.
Rank trending HackerNews topics in Postgres with an incremental CocoIndex v1 pipeline: scrape threads and comments, extract topics with an LLM.
Build a custom CocoIndex source for the HackerNews API: fetch threads and comments in async Python, extract topics with an LLM, keep Postgres in sync.
How to use BAML and CocoIndex to extract structured data from patient intake forms in PDF/Word with LLMs continuously for production.
Why the next wave of AI needs open-source, scalable, AI-native data infrastructure, and how CocoIndex is building the foundation for intelligent data pipelines.
Extract, embed, and store multimodal PDF elements (text with SentenceTransformers, images with CLIP) for unified semantic search with traceable metadata.
Read data from any system with CocoIndex: in v1 a custom source is plain async Python plus processing components. No connector API to implement.
Define query handlers in CocoIndex and trace search results back to source data in CocoInsight to close the loop on indexing strategy.
Use an existing PostgreSQL table as a CocoIndex source: derive fields, embed each row, and store the vectors in Postgres with pgvector, incrementally.
Build a unified visual document index from multiple file formats (including PDFs, images, and slides) using CocoIndex and ColPali. No OCR needed.
CocoIndex updates: production readiness, scalability, and reliability, plus more customization, native integrations, and multi-modal pipeline features.
Index images with ColPali multi-vector patch embeddings and a Qdrant MaxSim collection using CocoIndex v1: incremental, live, in plain async Python.
CocoIndex natively handles typed multi-dimensional vectors, from simple arrays to multi-vector embeddings, unlocking multimodal AI pipelines at scale.
Export CocoIndex data anywhere: implement a TargetHandler with a tracking record and an action sink; the engine handles diffing, syncing, and cleanup.
Build a scalable face detection and recognition pipeline with CocoIndex: embed faces, structure for search, and export to a vector DB.
Index academic papers with CocoIndex: read the first page of each PDF, extract title, authors, and abstract with an LLM, and embed them for semantic search.
Introducing CocoInsight, a data lineage and observability tool that lets you inspect, trace, and debug every step of a CocoIndex pipeline in real time.
CocoIndex sets up Qdrant collections automatically by inferring the target schema from your indexing flow: no manual config, vector sizes kept in sync.
Build a real-time knowledge graph with Kuzu as a native CocoIndex target: incremental updates, high-performance graph queries.
Index Markdown from Amazon S3 into Postgres pgvector with CocoIndex v1: incremental processing re-embeds only the files and chunks that changed.
Search a folder of photos by meaning with CocoIndex v1: CLIP embeds images and text into one vector space, the index runs live inside a FastAPI app, and vectors live in Qdrant.
Text embeddings 101: what they are, why you chunk and embed, and how to build a semantic search index with CocoIndex v1 and Postgres pgvector.
The story of CocoIndex at 1,000 GitHub stars: the open-source engine that combines custom transformation logic with incremental processing for data indexing.
An LLM extracts each product's taxonomy and complements; CocoIndex turns the labels into a Neo4j knowledge graph that answers 'bought this, also need…'.
Turn a folder of Markdown docs into a Neo4j knowledge graph: an LLM extracts subject-predicate-object triples, and CocoIndex keeps the graph in sync as the docs change.
What incremental processing is, who needs it, and how CocoIndex keeps an index in sync with source changes through caching, lineage tracking, and change data capture.
Welcome to the official CocoIndex blog! We're excited to share our journey in building high-performance indexing infrastructure for AI applications.