# Azure Blob Storage connector > **CocoIndex v1, Rust SDK (experimental).** This page documents the CocoIndex **v1** Rust SDK (the `cocoindex` crate). The Rust SDK is experimental: any release may change or remove its API, and it is not yet published on crates.io. The Python SDK pages are at https://cocoindex.io/docs/. > > Source: https://cocoindex.io/docs/rust/connectors/azure_blob/ · Rust docs index: https://cocoindex.io/docs/rust/llms.txt · Rust SDK tour: https://github.com/cocoindex-io/cocoindex/blob/main/rust/sdk/SHOWCASE.md · Rust examples: https://github.com/cocoindex-io/cocoindex/tree/main/examples/rust The `azure_blob` connector provides utilities for reading blobs from Azure Blob Storage containers. ```rust use cocoindex::connectors::azure_blob::{ self, AzureBlobFile, ContainerClient, Credential, ListOptions, }; ``` **Note — Installation** Enable the `azure_blob` feature of the `cocoindex` crate: ```toml cocoindex = { git = "https://github.com/cocoindex-io/cocoindex", features = ["azure_blob"] } ``` The connector speaks the Blob service REST API directly; it does not depend on an Azure SDK crate. ## As source The connector provides three read helpers: - `list_blobs()` lists blobs in a container, with optional prefix and filtering. - `get_blob()` fetches a single blob by name. - `read()` reads blob content directly by name. All helpers take the connector's own `azure_blob::ContainerClient`, a handle to one container built with `ContainerClient::new(account_url, container_name, credential)`. ### Connection setup The `credential` is a `Credential`: an Entra ID token the client gets and renews itself, a SAS token, a storage account access key, or a bearer token you obtained. For local development, `TokenCredential::Default` can use your Azure CLI login: ```bash az login az account set --subscription "" ``` ```rust use cocoindex::connectors::azure_blob::{ContainerClient, Credential, TokenCredential}; let client = ContainerClient::new( "https://myaccount.blob.core.windows.net", "docs", Credential::TokenCredential(TokenCredential::Default), )?; ``` `TokenCredential::Default` takes the sources of the Azure SDK's `DefaultAzureCredential` that need no interactive sign-in. It uses the first source that is configured where the process runs, a service principal from `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` and `AZURE_CLIENT_SECRET`, the managed identity of the Azure resource the process runs on, or the account signed in with `az login`. `TokenCredential::ClientSecret { tenant_id, client_id, client_secret }`, `TokenCredential::ManagedIdentity { client_id }` (`None` for the system-assigned identity) and `TokenCredential::AzureCli` name one source. The client gets the token on the first request and renews it shortly before it expires. `Credential::BearerToken` takes an access token for the `https://storage.azure.com/` resource that you obtained yourself, such as the one `az account get-access-token --resource https://storage.azure.com/` prints, and does not renew it. To use a SAS token: ```rust let client = ContainerClient::new( "https://myaccount.blob.core.windows.net", "docs", Credential::SasToken(sas_token), )?; ``` The SAS token is the query string Azure generates, with or without the leading `?`. To use a storage account access key: ```rust let client = ContainerClient::new( "https://myaccount.blob.core.windows.net", "docs", Credential::AccountKey(account_key), )?; ``` A `ContainerClient` is `Clone` and has no `close()`; declare a key with `cocoindex::context_key!(static CONTAINER: ContainerClient)` and provide the client with `Environment::builder().provide_key(&CONTAINER, client)`. An account URL with a path, such as the Azurite emulator's `http://127.0.0.1:10000/devstoreaccount1`, names the account by the first path segment. The identity must have permission to list and read blobs. With Azure role-based access control, `Storage Blob Data Reader` is enough for read-only indexing. ### list_blobs List blobs in an Azure Blob Storage container. Returns an `AzureBlobWalker`. ```rust pub fn list_blobs(client: &ContainerClient, options: ListOptions) -> AzureBlobWalker pub struct ListOptions { /// Only list blobs in this folder. pub prefix: String, /// Filter by relative path (after prefix stripping). Defaults to match-all. pub path_matcher: Option>, /// Skip blobs larger than this many bytes. pub max_file_size: Option, } pub async fn AzureBlobWalker::list(&self) -> Result> pub async fn AzureBlobWalker::items(&self) -> Result> ``` `ListOptions::default()` lists the whole container. The walker is not an iterator: `list()` returns every matching blob, and `items()` the `(key, file)` pairs for `mount_each!`. **Parameters:** - `container_client` - The `ContainerClient` (the argument is named `client`). - `prefix` - Only list blobs in this folder. A prefix names a folder: `"docs"` and `"docs/"` both list the blobs under `docs/`, and neither lists `docsa.md`. The folder is stripped from relative paths in returned files. To select blobs by the start of their name (for example `2024-*.csv`), use `path_matcher`. - `path_matcher` - Optional filter for files. Patterns are matched against the relative path after prefix stripping. See [PatternFilePathMatcher](/docs/rust/common_resources/data_types/#patternfilepathmatcher). - `max_file_size` - Skip blobs larger than this size in bytes. **Returns:** An `AzureBlobWalker` whose `list()` and `items()` list the container once each time you call them. ### Iterating files `list_blobs()` lists `AzureBlobFile` objects implementing [`FileLike`](/docs/rust/common_resources/data_types/#filelike): ```rust let files = azure_blob::list_blobs( client, ListOptions { prefix: "docs/".to_string(), ..Default::default() }, ) .list() .await?; for file in files { let text = file.read_text().await?; // ... } ``` ### Keyed iteration with `items()` `AzureBlobWalker::items()` returns `(String, AzureBlobFile)` pairs. The key is the blob name relative to the prefix, exactly as stored. Keys are not normalized, so `a//b.txt` and `a/b.txt` are two items: ```rust let files = azure_blob::list_blobs( &client, ListOptions { prefix: "docs".to_string(), ..Default::default() }, ) .items() .await?; mount_each!(files, |file| index_blob(ctx, file)).await?; ``` `file.key()` is the same relative blob name. ### Filtering files Use `PatternFilePathMatcher` to include or exclude blobs by glob pattern. Patterns are matched against the relative path after prefix stripping: ```rust use cocoindex::resources::file::PatternFilePathMatcher; let files = azure_blob::list_blobs( client, ListOptions { prefix, path_matcher: Some(Arc::new( PatternFilePathMatcher::include(["**/*.md"]) .map_err(|e| Error::engine(format!("matcher: {e}")))?, )), ..Default::default() }, ) .list() .await?; ``` `PatternFilePathMatcher::new(included, excluded)` takes exclude patterns too. ### Limiting file size Use `max_file_size` to skip blobs above a size threshold: ```rust let small = azure_blob::list_blobs( client, ListOptions { max_file_size: Some(10 * 1024 * 1024), ..Default::default() }, ) .list() .await?; ``` ### get_blob Fetch a single blob by name. ```rust pub async fn get_blob(client: &ContainerClient, blob_name: &str) -> Result ``` **Parameters:** - `container_client` - The `ContainerClient` (the argument is named `client`). - `blob_name` - The full blob name in the container. **Returns:** An `AzureBlobFile` for the specified blob. **Example:** ```rust let file = azure_blob::get_blob(client, "docs/readme.md").await?; let text = file.read_text().await?; ``` ### read Read blob content directly by name, without fetching metadata first. ```rust pub async fn read( client: &ContainerClient, blob_name: &str, size: Option, ) -> Result> ``` **Parameters:** - `container_client` - The `ContainerClient` (the argument is named `client`). - `blob_name` - The full blob name in the container. - `size` - Number of bytes to read. `None` reads the entire blob. **Returns:** The blob content as bytes. **Example:** ```rust let data = azure_blob::read(client, "docs/readme.md", None).await?; ``` ### AzureBlobFilePath Each file returned by the connector has an `AzureBlobFilePath`, a [`FilePath`](/docs/rust/common_resources/data_types/#filepath) specialized for Azure Blob Storage: - **Relative path** (`file.file_path().path()`, a `&str`) - The blob name relative to the walker prefix, or the full blob name if no prefix was used. `file.key()` returns it as a `String`, the key for `mount_each!`. - **Resolved path** (`file.file_path().resolve()`, also `blob_name()`) - The full blob name in the container. - **Account and container** (`account_name()`, `container_name()`). - **Identity** - The memo key is `(account_name, container_name, relative_path)`. For example, with `prefix: "docs/"` and blob name `"docs/readme.md"`: - `file.file_path().path()` returns `"readme.md"` - `file.file_path().resolve()` returns `"docs/readme.md"` `AzureBlobFile` has the public fields `size`, `modified_secs` and `etag`, plus `key()`, `read()`, `read_size(n)` and `read_text()`; it implements `FileLike` and `MemoInput`. Its memo key holds only stable metadata, never the client. `AzureBlobFilePath` has no derived-path methods (`parent`, `/`, `with_name()`, `with_suffix()`). ### Example ```rust use std::sync::Arc; use cocoindex::connectors::azure_blob::{self, AzureBlobFile, ContainerClient, ListOptions}; use cocoindex::prelude::*; use cocoindex::resources::file::PatternFilePathMatcher; cocoindex::context_key!(static CONTAINER: ContainerClient); #[cocoindex::function(memo)] async fn process_file(_ctx: &Ctx, file: AzureBlobFile) -> Result<()> { let text = file.read_text().await?; // Process the file content. Ok(()) } async fn app_main(ctx: Ctx, prefix: String) -> Result<()> { let client = ctx.get_key(&CONTAINER)?; let files = azure_blob::list_blobs( client, ListOptions { prefix, path_matcher: Some(Arc::new( PatternFilePathMatcher::include(["**/*.md"]) .map_err(|e| Error::engine(format!("matcher: {e}")))?, )), ..Default::default() }, ) .list() .await?; mount_each!( files.into_iter().map(|f| (f.key(), f)), |file| process_file(ctx, file) ) .await?; Ok(()) } ``` See `examples/rust/azure_blob_embedding` for a runnable project.