---
title: "CocoIndex Changelog 2025-05-31"
description: "CocoIndex updates: Amazon S3 as a data source, improved query handling, a standalone runtime mode, and more connector and performance improvements."
last_updated: 2025-05-31
doc_version: "2025-05-31"
canonical: https://cocoindex.io/blogs/cocoindex-changelog-2025-05-31/
---
# CocoIndex Changelog 2025-05-31

> CocoIndex updates: Amazon S3 as a data source, improved query handling, a standalone runtime mode, and more connector and performance improvements.

Published: 2025-05-31 · Canonical: https://cocoindex.io/blogs/cocoindex-changelog-2025-05-31/

In the past weeks, we've added support for Amazon S3 as a native data source, updated query handling, 
and made many core improvements in performance and stability over 15 releases. 
We're all in on building the best real-time incremental data framework, and we couldn’t be more excited.

Full changelog: [v0.1.30...v0.1.44](https://github.com/cocoindex-io/cocoindex/compare/v0.1.30...v0.1.44).

## Amazon S3 and Amazon Simple Queue Service (SQS) as native data source

CocoIndex now natively supports Amazon S3 as a data source, allowing seamless integration with your storage.
Combined with support for AWS Simple Queue Service (SQS), CocoIndex enables true real-time, 
incremental processing of data as it's updated in S3.

CocoIndex processes only S3 files that have been newly added or updated, 
reducing unnecessary compute cycles and improving overall system efficiency. 
Incremental processing focuses on handling only new or modified data since the previous run, rather than reprocessing the entire dataset every time. 
It becomes essential when working with large-scale data or when up-to-date information is critical.

- Read more about this integration in the [blog post](https://cocoindex.io/blogs/s3-incremental-etl).
- Get started now with the [example](https://github.com/cocoindex-io/cocoindex/tree/main/examples/text_embedding_s3).
- Documentation: [Amazon S3 as a data source](https://cocoindex.io/docs/connectors/amazon_s3/)

## Query stack
On the query side, CocoIndex has removed support for `SimpleQueryHandler`. 
We made this decision because:
- Flexibility - users directly talk to the database to have maximum control. Users often have specific requirements for query handling, such as custom filtering and search logic.
- Many databases already have optimized query implementations with their own best practices
- The query space has excellent solutions for querying, reranking, and other search-related functionality.

And we’re doubling down on what we do best: real-time indexing + transformation.

Here are some examples with query server:
- [Postgres with psycopg](https://github.com/cocoindex-io/cocoindex/blob/b2d6de588b89067f90027637d29a625f3131c442/examples/text_embedding/main.py#L65)
- [Qdrant](https://github.com/cocoindex-io/cocoindex/blob/b2d6de588b89067f90027637d29a625f3131c442/examples/text_embedding_qdrant/main.py#L81)

## Transform flow
When building vector indexes and querying against them, you often need to share transformation logic between indexing and querying operations. 
For instance, the embedding computation must remain consistent across both processes. 

In this case, you can
1.  Extract a sub-flow with the shared transformation logic into a standalone function. E.g.,

    ```python
    def text_to_embedding(text: cocoindex.DataSlice[str]) -> cocoindex.DataSlice[list[float]]:
        return text.transform(
            cocoindex.functions.SentenceTransformerEmbed(
                model="sentence-transformers/all-MiniLM-L6-v2"))
    ```

2.  When you're defining your indexing flow, you can directly call the function. E.g.,

    ```python
    with doc["chunks"].row() as chunk:
        chunk["embedding"] = text_to_embedding(chunk["text"])
    ```
3. At query time, you usually want to directly run the function with specific input data, instead of letting it be called as part of a long-lived indexing flow.
To do this, declare the function as a transform flow, by decorating it with `@cocoindex.transform_flow()`. 
    ```python
    @cocoindex.transform_flow()
    def text_to_embedding(text: cocoindex.DataSlice[str]) -> cocoindex.DataSlice[list[float]]:
        return text.transform(
            cocoindex.functions.SentenceTransformerEmbed(
                model="sentence-transformers/all-MiniLM-L6-v2"))
        
    ```
    So that you can directly call it with specific input data.
    ```python
    print(text_to_embedding.eval("Hello, world!"))
    ```

Learn more about transform flow in the [documentation](https://cocoindex.io/docs/programming_guide/processing_component/).

## New standalone CLI
CocoIndex now supports a standalone CLI, with cleaner CLI commands and tons of improvements. 

You have two ways to launch CocoIndex, see the [Documentation to launch CocoIndex](https://cocoindex.io/docs/getting_started/installation/) for more details.

- Use CocoIndex CLI. It's handy for most routine indexing building and management tasks. 
  It will load settings from environment variables, either already set in your environment, or specified in a `.env` file. 
  Check out the [CLI documentation](https://cocoindex.io/docs/cli/) for more details.

  For example:

    ```sh
    cocoindex setup main
    ```

    ```sh
    cocoindex update main
    ```

-   Call CocoIndex functionality from your own Python application or library. 
    For example:
    ```python
    from dotenv import load_dotenv
    import cocoindex

    load_dotenv()
    cocoindex.init()
    ```
    It's needed when you want to leverage CocoIndex support for query, or have your custom logic to trigger indexing, etc.
    Check out the full example [here](https://github.com/cocoindex-io/cocoindex/blob/f0c5618d2aaeec999a4e57c38486858247cb23db/examples/text_embedding/main.py#L63-L86).

## Support app namespace 

CocoIndex now supports application namespaces, making it easier to manage flows across different environments. 
This feature allows you to prefix flow names with a namespace, helping to organize and separate flows between environments like staging and production.

You can set the namespace using the `COCOINDEX_APP_NAMESPACE` environment variable. 
For example, if you set `COCOINDEX_APP_NAMESPACE=Staging` and define a flow with `@cocoindex.flow_def(name="TextEmbedding")`, the final flow name will be `Staging.TextEmbedding`.

This is particularly useful when:
- Managing flows across different environments
- Sharing flows within a team
- Organizing flows by project or application

You can also programmatically access the current namespace using `cocoindex.get_app_namespace()` to use it in your own code.

Learn more about this feature in the [documentation](https://cocoindex.io/docs/programming_guide/app/).

## Pretty print flow spec / schema 

CocoIndex now supports showing the flow spec and schema in a more readable format with `cocoindex show`.

For example, to print out the flow spec in this [document to knowledge graph project](https://github.com/cocoindex-io/cocoindex/tree/main/examples/docs_to_knowledge_graph), 
you can run:

```sh
cocoindex show main
```

It prints out
- the lineage of the flow from data source to the final output. 
- the global data schema of the flow

## Type system updates
### TimeDelta type 

CocoIndex now supports the `TimeDelta` type, which is a duration of time.
Learn more about the type system in the [documentation](https://cocoindex.io/docs/common_resources/data_types/).

## Examples and tutorials

### Real-time semantic search from files in S3
- Tutorial: [Real-time semantic search from files in S3](https://cocoindex.io/blogs/s3-incremental-etl)
- Code example: [Amazon S3 Embedding](https://github.com/cocoindex-io/cocoindex/tree/main/examples/amazon_s3_embedding)

### Image search with vision model
- Tutorial: [Image search with Vision model](https://cocoindex.io/blogs/live-image-search)
- Code example: [Image Search](https://github.com/cocoindex-io/cocoindex/tree/main/examples/image_search)

### Text search with FastAPI server on Docker Compose
- Code example: [Docker Compose for fast api server](https://github.com/cocoindex-io/cocoindex/tree/main/examples/fastapi_server_docker)

## Thanks to the community!
Welcome new contributors to the CocoIndex community! We are so excited to have you!

### [@lemorage](https://github.com/lemorage)

Thanks to [@lemorage](https://github.com/lemorage) for the contributions! 
CocoIndex has received a list of high quality PRs from him, and we really appreciate his work.

#### CLI: 
- feat: create standalone CLI for CocoIndex [#485](https://github.com/cocoindex-io/cocoindex/pull/485)   
- feat(pyo3): implement flow spec pretty print and add verbose mode [#442](https://github.com/cocoindex-io/cocoindex/pull/442)
- feat(cli): enhance flow display with schema rendering support[#427](https://github.com/cocoindex-io/cocoindex/pull/427)
- feat(cli): make flow name retrieval efficient [#414](https://github.com/cocoindex-io/cocoindex/pull/414)

#### DataTypes
- feat: add support for TimeDelta type in Python and Rust [#497](https://github.com/cocoindex-io/cocoindex/pull/497)
- feat: support namedtuple for struct types [#462](https://github.com/cocoindex-io/cocoindex/pull/462)
- feat(yaml): store variant name in tuple and struct variants[#444](https://github.com/cocoindex-io/cocoindex/pull/444)

### [@AbdelRahmanYaghi](https://github.com/AbdelRahmanYaghi)

Created an example of docker compose which runs pg17 along with a simple fastapi endpoint for querying text embeddings. [432](https://github.com/cocoindex-io/cocoindex/pull/432)

### [@par4m](https://github.com/par4m)

Created an example of image search with Vision model [447](https://github.com/cocoindex-io/cocoindex/pull/447)

## Full changelog
Review the detailed changelog [here](https://github.com/cocoindex-io/cocoindex/compare/v0.1.30...v0.1.44).

We are constantly improving CocoIndex, and more features are coming soon!
Stay tuned and follow us by starring our [GitHub repo](https://github.com/cocoindex-io/cocoindex).

## Sitemap

- [Blog index](https://cocoindex.io/blogs/)
- [Site index (llms.txt)](https://cocoindex.io/llms.txt)
- [Full blog corpus](https://cocoindex.io/llms-full.txt)
- [Markdown sitemap](https://cocoindex.io/sitemap.md)
- [XML sitemap](https://cocoindex.io/sitemap.xml)
- [RSS feed](https://cocoindex.io/blogs/rss.xml)