# Google Drive connector > **CocoIndex v1, Rust SDK (experimental).** This page documents the CocoIndex **v1** Rust SDK (the `cocoindex` crate). The Rust SDK is experimental: any release may change or remove its API, and it is not yet published on crates.io. The Python SDK pages are at https://cocoindex.io/docs/. > > Source: https://cocoindex.io/docs/rust/connectors/google_drive/ · Rust docs index: https://cocoindex.io/docs/rust/llms.txt · Rust SDK tour: https://github.com/cocoindex-io/cocoindex/blob/main/rust/sdk/SHOWCASE.md · Rust examples: https://github.com/cocoindex-io/cocoindex/tree/main/examples/rust The `google_drive` connector provides utilities for reading files from Google Drive, authenticated with a service account or with a personal Google account signed in through OAuth. ```rust use cocoindex::connectors::gdrive::{DriveFile, GoogleDriveClient, GoogleDriveSource}; ``` **Note — Dependencies** The module is `cocoindex::connectors::gdrive`, behind the `google_drive` feature of the `cocoindex` crate: ```toml cocoindex = { git = "https://github.com/cocoindex-io/cocoindex", features = ["google_drive"] } ``` ## As source The connector provides two ways to read from Google Drive: - `GoogleDriveSource` — the source, with `list_files()` and `items()` - `list_files()` — a function over a `GoogleDriveSourceSpec` Both read through a `GoogleDriveClient`, which authenticates with one of: - a **service account** with access to the target Drive folders (`GoogleDriveClient::from_service_account_file(path)?`), the fit for a shared or automated setup; - a **personal Google account** signed in once through OAuth (`GoogleDriveClient::from_user_account(&account)?`, experimental), the fit for your own Drive: see [Signing in with a personal account](#signing-in-with-a-personal-account). The client is `Clone`; provide it under a context key so that functions can read files through it. ### Setting up a service account 1. Create a service account in the [Google Cloud Console](https://console.cloud.google.com/iam-admin/serviceaccounts) 2. Download the JSON credential file 3. Share the target Drive folders with the service account's email address **Note — Google Workspace CLI** [`gws`](https://github.com/googleworkspace/cli) is an optional, unofficial Google Workspace CLI. It is actively developed and subject to change, but can be useful for exploring or validating Drive API access before configuring CocoIndex's service-account flow. For example: ```bash gws auth setup gws auth login gws drive files list ``` In headless or agent workflows, `gws` can also read credentials from `GOOGLE_WORKSPACE_CLI_CREDENTIALS_FILE`. CocoIndex still expects the service account JSON path in `GoogleDriveClient::from_service_account_file`; use the `gws` credentials setting for `gws` commands themselves. ### GoogleDriveSource The primary source class for iterating over Google Drive files. ```rust pub fn GoogleDriveSource::new(client: GoogleDriveClient, root_folder_ids: Vec) -> Self pub fn GoogleDriveSource::mime_types(self, mime_types: Vec) -> Self ``` For example, with the client provided under the `GDRIVE` key: ```rust cocoindex::context_key!(static GDRIVE: GoogleDriveClient); let source = GoogleDriveSource::new(ctx.get_key(&GDRIVE)?.clone(), root_folder_ids); ``` **Parameters:** - `client` — The `GoogleDriveClient`, signed in with a service account or a [`UserAccount`](#useraccount). - `root_folder_ids` — Google Drive folder IDs to scan. Subfolders are traversed recursively. - `mime_types` — Optional MIME types to include, set with `.mime_types(...)`. Without it, all file types are included. ### Iterating files `GoogleDriveSource::list_files()` lists `DriveFile` objects (implementing the [`FileLike`](/docs/rust/common_resources/data_types/#filelike) base class): ```rust let client = GoogleDriveClient::from_service_account_file("./credentials.json")?; let source = GoogleDriveSource::new(client, vec!["1abc...xyz".to_string()]); for file in source.list_files().await? { let text = file.read_text().await?; // ... } ``` `list_files()` returns the whole listing as a `Vec`; there is no async iterator. A listed file carries the client, so `file.read_text()` works; `client.read_text(&file)` reads through a client you hold. ### Keyed iteration with `items()` `items()` returns `(String, DriveFile)` pairs keyed by the file's Google Drive ID. IDs remain stable across renames and avoid collisions when files share a name, including within the same folder. This is useful with `mount_each!`: ```rust let files = source.items().await?; mount_each!(files, |file| process_file(ctx, file, table)).await?; ``` `file.key()` is the same Google Drive ID. ### Filtering by MIME type Use `mime_types` to restrict which files are returned: ```rust let source = GoogleDriveSource::new(client, vec!["1abc...xyz".to_string()]) .mime_types(vec!["application/pdf".to_string(), "text/plain".to_string()]); ``` Google Workspace files (Docs, Sheets, Slides) are automatically exported: | Google Workspace type | Exported as | |---|---| | Google Docs | Plain text | | Google Sheets | CSV | | Google Slides | Plain text | ### list_files A lower-level sync iterator for listing files: ```rust pub struct GoogleDriveSourceSpec { pub client: GoogleDriveClient, pub root_folder_ids: Vec, pub mime_types: Option>, } pub async fn list_files(spec: GoogleDriveSourceSpec) -> Result> ``` **Parameters:** - `spec` — A `GoogleDriveSourceSpec` with the same fields as `GoogleDriveSource`: the client, the root folder IDs and the optional MIME types. **Returns:** Every listed `DriveFile`, as a `Vec`. ### DriveFile `DriveFile` implements [`FileLike`](/docs/rust/common_resources/data_types/#filelike) with Google Drive-specific behavior: - Public fields `file_id`, `name`, `path`, `mime_type`, `size`, `modified_time` (RFC 3339, as Drive returns it) and `md5_checksum` (binary files only); `info()` returns them as a `DriveFileInfo`, and `key()` returns the file ID. - `file_path()` — A `DriveFilePath` with `path()`, the path relative to the configured root folder, and `file_id()`; `resolve()` returns the Google Drive file ID. If roots overlap or a file is reachable more than once, it is emitted once using the first path encountered in breadth-first traversal. Each folder's children are visited in order of name, then ID, so the same path is kept on every run. - `read()` / `read_text()` — Downloads file content via the Google Drive API. A listed file carries the client; `client.read(&file)` and `client.read_text(&file)` read through a client you hold. - Every Drive API request, listing and download alike, is retried with backoff (up to five more attempts) when Google answers with a server error, HTTP 429, or 403 with a rate-limit reason, or when the connection fails. - Change detection uses the Drive MD5 checksum of binary files, so an unchanged file is not downloaded again. Google Docs, Sheets and Slides have no checksum; their exported content is hashed instead. `DriveFile` implements `MemoInput`: a memoized function that takes it runs again only for a file whose content changed. Its memo key holds only the file's metadata, never the client. ## Signing in with a personal account **Caution — Experimental** `UserAccount` and the `cocoindex auth` commands are experimental. Experimental APIs are outside CocoIndex's compatibility promise: a later release, including a minor one, may change their API or semantics, or remove them. A service account only sees folders shared with it. To read your own Drive, sign in with your Google account instead. The sign-in is a one-time interactive step, run from the shell, outside any CocoIndex run; it uses the OAuth flow Google recommends for command-line applications (an authorization code with PKCE, redirected to a listener on this machine's loopback interface), from a Desktop app OAuth client. CocoIndex asks for read-only access to Drive (`drive.readonly`) and nothing else. CocoIndex's use and transfer of information received from Google APIs adheres to the [Google API Services User Data Policy](https://developers.google.com/terms/api-services-user-data-policy), including the Limited Use requirements; see [how CocoIndex handles Google user data](https://cocoindex.io/privacy-policy/#google-user-data). CocoIndex's own OAuth client is not in a release yet: Google has to verify it first. Until a release includes it, sign in with an OAuth client you create in your own Google Cloud project, as below. A release that includes it needs nothing but your Google account: `cocoindex auth sign-in google-drive` without a client file. The built-in client is compiled in only when `COCOINDEX_GOOGLE_OAUTH_CLIENT_ID` and `COCOINDEX_GOOGLE_OAUTH_CLIENT_SECRET` are set at build time, as a release build sets them. A `cocoindex` crate or `cocoindex` CLI built from the git repository has none, so its sign-in takes your own client file. ### Setting up the OAuth client Once per Google Cloud project: 1. In the [Google Cloud Console](https://console.cloud.google.com/), create a project (or pick one) and enable the **Google Drive API** under *APIs & Services > Library*. 2. Under *APIs & Services > OAuth consent screen* (called *Google Auth Platform* in newer consoles), configure the consent screen: external user type, any app name, your email as the contact. Leave the publishing status at **Testing** and add your own Google account under **Test users**. 3. Under *APIs & Services > Credentials*, create an **OAuth client ID** of type **Desktop app** and download its JSON file. What to expect from a client you own, as Google documents it: - `drive.readonly` is a *restricted* scope, so Google would require app verification (and a security assessment) before the public could use the client. Neither applies to a client in **Testing** status used by its listed test users, which is the setup for reading your own Drive. - In Testing status, Google expires the consent **seven days** after it was given: a run then fails with `SignInRequiredError` and you sign in again. Publishing the client to *In production* without verification removes the seven-day limit; the consent screen then shows an "unverified app" warning you pass through *Advanced*, and the client is limited to 100 users over its lifetime. - The downloaded JSON contains a client secret. For a Desktop app Google treats it as not confidential, but keep the file out of git all the same; CocoIndex copies what it needs into the token file and never writes either into the state database, a memo key or a log. ### Signing in ```bash cocoindex auth sign-in google-drive ~/Downloads/client_secret_XXXX.json ``` Without the file argument, the command uses CocoIndex's own OAuth client, and a release that doesn't include one says so. The command opens the consent page in your browser. Approve the request, including the Google Drive permission; the browser lands on a page that says the sign-in completed, and the terminal prints the signed-in address. On a machine with no browser (over SSH, or with no display), the command prints the link instead. Open it in a browser on another machine and sign in; the browser then lands on `http://127.0.0.1:/?code=...&state=...` and fails to load, since that address is the other machine. Copy the whole address from the address bar and paste it into the terminal. (If you forward the port with `ssh -L :127.0.0.1: ` in a second session, the page loads instead and nothing is left to paste.) The tokens are stored at `~/.config/cocoindex/google_drive_token.json` on Linux and macOS (under `$XDG_CONFIG_HOME` when set) and under `%APPDATA%\cocoindex` on Windows; `--token-path` picks another location. **Caution — Treat the token file like a password** The token file holds a long-lived refresh token with read access to your whole Drive, in plain text, next to the OAuth client's secret. On macOS and Linux, CocoIndex creates it with mode `0600` (readable by your user only), and the connector refuses a token file that other users can access. On Windows there is no such check: the file is as private as its folder, which for the default location is your user profile. Either way, any process that runs as your user can read it. Don't commit it to git, sync it to cloud storage or copy it into a container image. If it may have leaked, revoke the sign-in: run `cocoindex auth sign-out google-drive`, or remove the app's access at [myaccount.google.com/permissions](https://myaccount.google.com/permissions). ```bash cocoindex auth sign-out google-drive # revokes the sign-in at Google and deletes the token file ``` These are commands of the [`cocoindex` CLI for Rust projects](/docs/rust/cli/#auth). The Python CLI has the same two commands over the same token file, at the same default location, so one sign-in serves both SDKs. From code, `UserAccount::new()?.sign_in(Some(client_secrets_path)).await?` signs in with your own client and `sign_in(None)` with the client built into the release, when it has one; `sign_out().await?` revokes the sign-in and deletes the file. See [Signing in from your own program](#signing-in-from-your-own-program). ### Signing in from your own program A program can sign in by itself instead of asking you to run the command first: check `is_signed_in()`, and call `sign_in()` when it is false, before the app runs. - `sign_in` waits for a person, up to five minutes, to finish the consent in the browser. So call it before the run starts, never from inside the run. - On a host with no browser, it prints the link and reads the address the browser lands on, pasted back into the terminal, as the command does. - The command and the library use the same token file, at the same default location, so a sign-in with one serves the other. - `is_signed_in()` reads only the token file. A stored sign-in that Google has since refused (the consent expired or was revoked) still counts; the run then fails with `SignInError::SignInRequired`, and you sign in again. ```rust use std::path::{Path, PathBuf}; use cocoindex::connectors::gdrive::{GoogleDriveClient, UserAccount}; use cocoindex::prelude::*; #[tokio::main] async fn main() -> Result<()> { let account = UserAccount::new()?; if !account.is_signed_in() { // A client file of your own; `None` for CocoIndex's own client. let email = account.sign_in(Some(Path::new("client_secret.json"))).await?; println!("Signed in as {email}."); } let gdrive = GoogleDriveClient::from_user_account(&account)?; let root_folder_ids = vec!["1abc...xyz".to_string()]; let app = Environment::builder() .db_path(PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(".cocoindex_db")) .provide_key(&GDRIVE, gdrive) .build() .await? .app("GoogleDriveTextEmbeddingRust") .await?; let stats = app.run(move |ctx| app_main(ctx, root_folder_ids)).await?; println!("{stats}"); Ok(()) } ``` `GDRIVE` and `app_main` are those of the [example](#example) below. ### UserAccount ```rust pub fn UserAccount::new() -> Result pub fn UserAccount::with_token_path(token_path: impl Into) -> Self pub fn UserAccount::token_path(&self) -> &Path pub async fn UserAccount::sign_in(&self, client_secrets_path: Option<&Path>) -> Result pub async fn UserAccount::sign_out(&self) -> Result pub fn UserAccount::email(&self) -> Result pub fn UserAccount::is_signed_in(&self) -> bool ``` `new()` uses the default token path; `with_token_path` picks another location. `sign_in` returns the signed-in address. `sign_out` deletes the token file even when Google can't be reached, and returns whether the file existed. `email()` fails with `SignInError::SignInRequired` when no sign-in is stored. `is_signed_in()` is `false` as well when the file exists but can't be used, for example because other users can access it. The handle a `GoogleDriveClient` signs with. `GoogleDriveClient::from_user_account(&account)` reads the token file at `token_path` (default: the path `sign-in` writes to) when it builds the client: - the access token is refreshed with the stored refresh token as it expires, and the refreshed token is written back to the file; - `email()` is the signed-in account's address, and the client's identity is that address: a refreshed token does not invalidate memoized work, a different account does; - when no sign-in is stored, or Google refuses the stored one (the consent expired or was revoked, or the OAuth client was deleted, disabled or given a new secret), the call fails with `SignInError::SignInRequired`, whose message says which and to sign in again. Other sign-in failures are `SignInError::Failed`. `SignInError::of(&error)` recovers the `SignInError` from the SDK's `Error`. The message never contains a token. ```rust let account = UserAccount::new()?; let client = GoogleDriveClient::from_user_account(&account)?; let files = GoogleDriveSource::new(client, folder_ids).list_files().await?; mount_each!(files.into_iter().map(|file| (file.key(), file)), |file| { process_file(ctx, file) }) .await?; ``` ### Example ```rust use std::path::PathBuf; use cocoindex::connectors::gdrive::{DriveFile, GoogleDriveClient, GoogleDriveSource}; use cocoindex::prelude::*; cocoindex::context_key!(static GDRIVE: GoogleDriveClient); #[cocoindex::function(memo)] async fn process_file(_ctx: &Ctx, file: DriveFile) -> Result<()> { let text = file.read_text().await?; // ... process the file content ... Ok(()) } async fn app_main(ctx: Ctx, root_folder_ids: Vec) -> Result<()> { let source = GoogleDriveSource::new(ctx.get_key(&GDRIVE)?.clone(), root_folder_ids); let files = source.items().await?; println!("indexing {} Google Drive files", files.len()); mount_each!(files, |file| process_file(ctx, file)).await?; Ok(()) } #[tokio::main] async fn main() -> Result<()> { let gdrive = GoogleDriveClient::from_service_account_file("./credentials.json")?; let root_folder_ids = vec!["1abc...xyz".to_string()]; let app = Environment::builder() .db_path(PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(".cocoindex_db")) .provide_key(&GDRIVE, gdrive) .build() .await? .app("GoogleDriveTextEmbeddingRust") .await?; let stats = app.run(move |ctx| app_main(ctx, root_folder_ids)).await?; println!("{stats}"); Ok(()) } ``` The full program, which chunks and embeds each file into Postgres, is `examples/rust/gdrive_text_embedding`.