# BigQuery connector

> **CocoIndex v1, Rust SDK (experimental).** This page documents the CocoIndex **v1** Rust SDK (the `cocoindex` crate). The Rust SDK is experimental: any release may change or remove its API, and it is not yet published on crates.io. The Python SDK pages are at https://cocoindex.io/docs/.
>
> Source: https://cocoindex.io/docs/rust/connectors/bigquery/ · Rust docs index: https://cocoindex.io/docs/rust/llms.txt · Rust SDK tour: https://github.com/cocoindex-io/cocoindex/blob/main/rust/sdk/SHOWCASE.md · Rust examples: https://github.com/cocoindex-io/cocoindex/tree/main/examples/rust

The `bigquery` connector provides target state APIs for writing rows to BigQuery tables. CocoIndex tracks the rows that should exist and applies table creation, upserts, updates, and deletes incrementally.

```rust
use cocoindex::connectors::bigquery::{self, BigQueryConfig, BigQueryConnection, TableSchema};
```

**Note — Install**
Enable the `bigquery` Cargo feature of the `cocoindex` crate. The connector speaks the BigQuery v2 REST API directly.

## Connection setup

Declare a context key for a `BigQueryConnection` with `cocoindex::context_key!`, then provide the connection when you build the environment:

**Note**
The key name is load-bearing across runs. It is the stable identity CocoIndex uses to track managed rows. See [ContextKey as stable identity](/docs/rust/programming_guide/context/#contextkey-as-stable-identity) before renaming.

```rust
use cocoindex::connectors::bigquery::{BigQueryConfig, BigQueryConnection};

cocoindex::context_key!(static BIGQUERY: BigQueryConnection, key = "bigquery_demo");

let optional = |name: &str| std::env::var(name).ok().filter(|v| !v.is_empty());
let conn = BigQueryConnection::connect(BigQueryConfig {
    project: optional("BIGQUERY_PROJECT"),
    credentials_path: optional("GOOGLE_APPLICATION_CREDENTIALS").map(Into::into),
    location: optional("BIGQUERY_LOCATION"),
})?;
let app = cocoindex::Environment::builder()
    .db_path(".cocoindex_db")
    .provide_key(&BIGQUERY, conn)
    .build()
    .await?
    .app("BigQueryTarget")
    .await?;
```

`BigQueryConnection::connect(config)?` resolves the credentials and makes no request until the first query. The connection is `Clone`; `conn.query(sql).await?` runs a query and returns its rows as JSON objects.

If `credentials_path` is omitted, Application Default Credentials are used.

### BigQueryConfig

```rust
pub struct BigQueryConfig {
    pub project: Option<String>,
    pub credentials_path: Option<PathBuf>,
    pub location: Option<String>,
}
```

`BigQueryConfig` implements `Default`. When `project` is unset, the `GOOGLE_CLOUD_PROJECT` / `GCLOUD_PROJECT` environment variable, then the credentials file's project, is used.

**Parameters:**

- `project` - Optional Google Cloud project that runs the query jobs.
- `credentials_path` - Optional path to a service account JSON key file. If omitted, Application Default Credentials are used.
- `location` - Optional BigQuery job location, for example `US` or `EU`.

## As target

The `bigquery` connector provides target state APIs for writing rows to tables.

### Tables

Declares a table as a target state. Returns a `TableTarget` for declaring rows.

```rust
pub async fn mount_table_target(
    ctx: &Ctx,
    conn: &ContextKey<BigQueryConnection>,
    dataset: impl Into<String>,
    table_name: impl Into<String>,
    table_schema: TableSchema,
) -> Result<TableTarget>

pub async fn mount_table_target_with_options(
    ctx: &Ctx,
    conn: &ContextKey<BigQueryConnection>,
    dataset: impl Into<String>,
    table_name: impl Into<String>,
    table_schema: TableSchema,
    options: BigQueryTableOptions,
) -> Result<TableTarget>

pub struct BigQueryTableOptions {
    /// Project that owns the table. `None` means the connection's project.
    pub project: Option<String>,
    pub managed_by: ManagedBy,
}
```

`declare_table_target` and `declare_table_target_with_options` take the same arguments and declare the table in the current component.

**Parameters:**

- `conn` - The context key of the `BigQueryConnection` to use.
- `dataset` - BigQuery dataset name.
- `table_name` - Name of the table.
- `table_schema` - Schema definition including columns and primary key.
- `options.project` - Optional project that owns the target table. If `None`, the connection's project is used. `None` and naming the connection's project explicitly are two different keys for the same table (see the caution below).
- `options.managed_by` - Whether CocoIndex manages the table lifecycle (`ManagedBy::System`, the default) or assumes it exists (`ManagedBy::User`).

With `ManagedBy::System`, CocoIndex creates the dataset and table if needed. Table changes use BigQuery DDL, and row changes use BigQuery `MERGE` for upserts.

**Note**
BigQuery primary key constraints are not enforced by BigQuery. CocoIndex still uses the primary key list to reconcile rows and generate `MERGE` and `DELETE` statements.

**Caution — One table, one key**
CocoIndex identifies a table by its key: the connection's context key, the project, `dataset` and `table_name`. `project: None` and `project: Some("<connection project>")` reach the same table through two different keys. Declare each table through one form only. Switching a table from one form to the other changes its key: CocoIndex then cleans up the table it tracked under the old key, which is the same table. See [ContextKey as stable identity](/docs/rust/programming_guide/context/#contextkey-as-stable-identity).

### Rows

Once a `TableTarget` is resolved, declare rows to be upserted:

```rust
pub fn TableTarget::declare_row<R: serde::Serialize>(&self, ctx: &Ctx, row: &R) -> Result<()>
```

The row serializes to an object whose keys are column names; keys outside the schema are ignored.

Every primary-key column must be present in the row.

## Table schema: from a Rust struct

Define the table structure with a struct that derives `SchemaFields`:

```rust
use cocoindex::prelude::*;
use serde::{Deserialize, Serialize};
use serde_json::Value as JsonValue;

#[derive(Clone, Serialize, Deserialize, SchemaFields)]
struct BigQueryOrder {
    order_id: String,
    customer: String,
    quantity: i64,
    unit_price: f64,
    #[coco(json)]
    attributes: JsonValue,
}

let schema = TableSchema::from_row::<BigQueryOrder>(["order_id"])?;
```

`TableSchema::from_row::<Row>(primary_key)` takes the primary-key columns as an iterator of names. A `serde_json::Value` field needs `#[coco(json)]`; a `Vec<T>`, map or nested struct maps to `JSON` on its own.

Rust field types are automatically mapped to BigQuery column types:

| Rust Type | BigQuery Type |
|-----------|---------------|
| `bool` | `BOOL` |
| `i16`, `i32`, `i64` (and the unsigned widths) | `INT64` |
| `f32`, `f64` | `FLOAT64` |
| `rust_decimal::Decimal` | `NUMERIC` |
| `String` | `STRING` |
| `Vec<u8>` | `BYTES` |
| `uuid::Uuid` | `STRING` |
| `chrono::NaiveDate` | `DATE` |
| `chrono::NaiveTime` | `TIME` |
| `chrono::DateTime`, `chrono::NaiveDateTime` | `TIMESTAMP` |
| `std::time::Duration` | `FLOAT64` |
| other `Vec<T>`, maps, nested structs, `#[coco(json)]` | `JSON` |
| `#[coco(vector)] Vec<f32>` | `ARRAY<FLOAT64>` |

`JSON` values are JSON-serialized and written with `PARSE_JSON`. An `Option<T>` field is a nullable column; every other field is `NOT NULL`.

### Custom column types

Use `#[coco(type = "…")]` on a field to specify a custom BigQuery type:

```rust
#[derive(Serialize, SchemaFields)]
struct ProductRow {
    #[coco(type = "NUMERIC")]
    id: i64,
    #[coco(type = "ARRAY<FLOAT64>")]
    embedding: Vec<f64>,
}
```

`#[coco(type = "…")]` sets the column type verbatim. Explicit columns are also possible: `TableSchema::new([("id", ColumnDef::new("INT64").not_null()), ("payload", ColumnDef::new("JSON").parse_json())], ["id"])?`. A `ColumnDef` is nullable unless `.not_null()` is called, and `.parse_json()` writes the value through `PARSE_JSON`.

There is no per-column `encoder`: a value is written as it serializes with serde.

You can also override one column of a derived schema with `with_column_type`, which replaces the column's type and its `parse_json` flag and keeps its derived nullability:

```rust
let schema = TableSchema::from_row::<Row>(["id"])?
    .with_column_type("payload", ColumnDef::new("JSON").parse_json())?;
```

## Example

```rust
let schema = TableSchema::from_row::<BigQueryOrder>(["order_id"])?;
let table = bigquery::mount_table_target(
    &ctx,
    &BIGQUERY,
    "analytics",
    "product_index",
    schema,
)
.await?;
let rows = use_mount_each!(orders, |order| process_order(ctx, order)).await?;
for row in &rows {
    table.declare_row(&ctx, row)?;
}
```

To name the project that owns the table, use `mount_table_target_with_options` with `BigQueryTableOptions { project: Some("my-project".to_string()), ..Default::default() }`.

See `examples/rust/bigquery_target` for a runnable project.

## Identifier handling

Dataset, table, and column names must be simple BigQuery identifiers containing letters, numbers, and underscores, and must not start with a number. The connector quotes identifiers when generating SQL.