---
title: "Index PDFs, images, and slides with ColPali, no OCR"
description: "Build a unified visual document index from multiple file formats (including PDFs, images, and slides) using CocoIndex and ColPali. No OCR needed."
last_updated: 2025-08-20
doc_version: "2025-08-20"
canonical: https://cocoindex.io/blogs/multi-format-indexing/
---
# Index PDFs, images, and slides with ColPali, no OCR

> Build a unified visual document index from multiple file formats (including PDFs, images, and slides) using CocoIndex and ColPali. No OCR needed.

Published: 2025-08-20 · Canonical: https://cocoindex.io/blogs/multi-format-indexing/

Do you have a messy collection of scanned documents, PDFs, academic papers, presentation slides, and standalone images, all mixed together with charts, tables, and figures, that you want to process into the same vector space for semantic search or to power an AI agent?

In this example, we’ll walk through how to build a visual document indexing pipeline using ColPali for embedding both PDFs and images, and then query the index using natural language.  
We’ll skip OCR entirely: ColPali can directly understand document layouts, tables, and figures from images, making it perfect for semantic search across visual-heavy content.

If this helps you, ⭐ Star [CocoIndex GitHub](https://github.com/cocoindex-io/cocoindex)!

## Use cases

- Semantic search across scanned documents and visual-heavy reports
- Searching tables, charts, or diagrams without OCR
- Cross-modal search (e.g., “find the page showing a bar chart about Q2 revenue”)
- Research paper figure indexing
- Historical archive search without text extraction

## Flow overview

We’ll build a pipeline that:

- **Ingests PDFs and images** from a local directory
    - **Converts PDF pages** into high-resolution images (300 DPI)
    - **Generates visual embeddings** for each page/image using ColPali
- **Stores embeddings + metadata** in a Qdrant vector database
- **Supports natural language queries** directly against the visual index

Example queries:

- *"handwritten lab notes about physics"*
- *"architectural floor plan with annotations"*
- *"pie chart of Q3 revenue"*

Full code is open source and available [here](https://github.com/cocoindex-io/cocoindex/tree/main/examples/multi_format_indexing). 
:rocket: Only ~70 lines of Python on the indexing path (super simple!)

## Core components

### Image ingestion

We use CocoIndex’s [`LocalFile` source](https://cocoindex.io/docs/connectors/localfs/) to read PDFs and images:

```python
data_scope["documents"] = flow_builder.add_source(
    cocoindex.sources.LocalFile(path="source_files", binary=True)
)
```

### Convert files to pages

We classify files by MIME type and process accordingly. 

Define a dataclass:

- `page_number`: The page number (if applicable, only for PDFs)
- `image`: The binary content of that page as a PNG image

```python
@dataclass
class Page:
  page_number: int | None
  image: bytes
```

This normalizes different file formats into a **list of page images** so the rest of the pipeline can process them uniformly. This `file_to_pages` [custom function](https://cocoindex.io/docs/programming_guide/function/) takes a **filename** and its **raw binary content** (`bytes`) and returns a list of `Page` objects, where each `Page` contains:

```python
@cocoindex.op.function()
def file_to_pages(filename: str, content: bytes) -> list[Page]:
    mime_type, _ = mimetypes.guess_type(filename)

    if mime_type == "application/pdf":
        images = convert_from_bytes(content, dpi=300)
        pages = []
        for i, image in enumerate(images):
            with BytesIO() as buffer:
                image.save(buffer, format="PNG")
                pages.append(Page(page_number=i + 1, image=buffer.getvalue()))
        return pages
    elif mime_type and mime_type.startswith("image/"):
        return [Page(page_number=None, image=content)]
    else:
        return []
```

For each document:
- If the file is an image → `file_to_pages` returns a single `Page` where `page["image"]` is just the original image binary.
- If the file is a PDF → `file_to_pages` converts each page to a PNG, so `page["image"]` contains that page’s PNG binary.

In the flow we convert all the files to pages. This makes each page and all images in the output data: pages.

```jsx
 output_embeddings = data_scope.add_collector()

 with data_scope["documents"].row() as doc:
    doc["pages"] = flow_builder.transform(
        file_to_pages, filename=doc["filename"], content=doc["content"]
    )
```

### Generate visual embeddings

We use ColPali to generate embeddings for images on each page. 

```python
with doc["pages"].row() as page:
    page["embedding"] = page["image"].transform(
        cocoindex.functions.ColPaliEmbedImage(model=COLPALI_MODEL_NAME)
    )
    output_embeddings.collect(
                id=cocoindex.GeneratedField.UUID,
                filename=doc["filename"],
                page=page["page_number"],
                embedding=page["embedding"],
            )
```

ColPali Architecture fundamentally rethinks how documents, especially visually complex or image-rich ones, are represented and searched.
Instead of reducing each image or page to a single dense vector (as in traditional bi-encoders), ColPali breaks an image into many smaller patches, preserving local spatial and semantic structure. 

Each patch receives its own embedding, which together form a [multi-vector representation](https://cocoindex.io/docs/common_resources/vector_schema/) of the complete document.

For a detailed explanation of ColPali Architecture, please refer to [our previous blog](https://cocoindex.io/blogs/colpali) with image search examples.

## Collect & export to Qdrant

Note the ways to embed an image and a query are different, as they’re two different types of data. 

Create a [transform flow](https://cocoindex.io/docs/programming_guide/processing_component/) to embed the query:

```python
@cocoindex.transform_flow()
def query_to_colpali_embedding(
    text: cocoindex.DataSlice[str],
) -> cocoindex.DataSlice[list[list[float]]]:
    return text.transform(
        cocoindex.functions.ColPaliEmbedQuery(model=COLPALI_MODEL_NAME)
    )
```

We store metadata and embeddings in [Qdrant](https://cocoindex.io/docs/connectors/qdrant/):

```jsx
output_embeddings.export(
    "multi_format_indexings",
    cocoindex.targets.Qdrant(
        connection=qdrant_connection,
        collection_name=QDRANT_COLLECTION,
    ),
    primary_key_fields=["id"],
)
```

## Query the index

ColPali supports **text-to-visual embeddings**, so we can search using natural language:

```python
query_embedding = query_to_colpali_embedding.eval(query)

search_results = client.query_points(
    collection_name=QDRANT_COLLECTION,
    query=query_embedding,
    using="embedding",
    limit=5,
    with_payload=True,
)
```

Check out the full code [here](https://github.com/cocoindex-io/cocoindex/tree/main/examples/multi_format_indexing).

## Debugging with CocoInsight

Run CocoInsight locally:

```sh
cocoindex server -ci main
```

Open [https://cocoindex.io/cocoinsight](https://cocoindex.io/cocoinsight) to:

- View extracted pages
- See embedding vectors and metadata

## Support us

We’re constantly adding more examples and improving our runtime.

⭐ Star CocoIndex on [GitHub](https://github.com/cocoindex-io/cocoindex) and share the love :heart:!

And let us know what you are building with CocoIndex. We’d love to feature them.

## Sitemap

- [Blog index](https://cocoindex.io/blogs/)
- [Site index (llms.txt)](https://cocoindex.io/llms.txt)
- [Full blog corpus](https://cocoindex.io/llms-full.txt)
- [Markdown sitemap](https://cocoindex.io/sitemap.md)
- [XML sitemap](https://cocoindex.io/sitemap.xml)
- [RSS feed](https://cocoindex.io/blogs/rss.xml)