New features and improvements in CocoIndex.
Walk through a live CocoIndex pipeline that watches a folder of CSV files and publishes each row as JSON to a Kafka topic incrementally, with no glue code.
CocoIndex V1 is live: a ground-up redesign of incremental data pipelines, built for AI engineers and agent builders shipping RAG, memory, and knowledge graphs.
Featuring batching support for CocoIndex functions, execution robustness, schema & type system improvements, custom source support, and more.
CocoIndex now batches GPU and ML workloads automatically: 5x throughput on text embeddings and AI ops, with zero configuration required.
Extract, embed, and store multimodal PDF elements (text with SentenceTransformers, images with CLIP) for unified semantic search with traceable metadata.
Read data from any system with CocoIndex: in v1 a custom source is plain async Python plus processing components. No connector API to implement.
Define query handlers in CocoIndex and trace search results back to source data in CocoInsight to close the loop on indexing strategy.
How CocoIndex's layered concurrency controls optimize data-processing performance, prevent system overload, and keep pipelines stable and efficient at scale.
Index images with ColPali multi-vector patch embeddings and a Qdrant MaxSim collection using CocoIndex v1: incremental, live, in plain async Python.
CocoIndex natively handles typed multi-dimensional vectors, from simple arrays to multi-vector embeddings, unlocking multimodal AI pipelines at scale.
Export CocoIndex data anywhere: implement a TargetHandler with a tracking record and an action sink; the engine handles diffing, syncing, and cleanup.
Introducing CocoInsight, a data lineage and observability tool that lets you inspect, trace, and debug every step of a CocoIndex pipeline in real time.
CocoIndex sets up Qdrant collections automatically by inferring the target schema from your indexing flow: no manual config, vector sizes kept in sync.
Build a real-time knowledge graph with Kuzu as a native CocoIndex target: incremental updates, high-performance graph queries.
CocoIndex continuously watches source changes and applies incremental updates to keep derived data in sync, with low latency and no full reindexing.
What incremental processing is, who needs it, and how CocoIndex keeps an index in sync with source changes through caching, lineage tracking, and change data capture.
How CocoIndex handles system updates in indexing flows: automatic schema inference and managing data + logic evolution without downtime.