> ## Documentation Index
> Fetch the complete documentation index at: https://docs.elementary-data.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Connect to Bigquery

This guide contains the necessary steps to connect a BigQuery environment to your Elementary account.

## Choose an authentication method

Elementary supports two authentication methods for BigQuery. Pick the one that fits your security model:

* **Service account** — create a service account, download its JSON key, and upload the key to Elementary. Simplest to set up.
* **Workload Identity Federation (WIF)** — Elementary authenticates from its AWS role through a federated identity. No long-lived credentials are stored in Elementary.

Select a tab below and follow the steps for your chosen method.

> [tab] Service account
>
> ### Create service account:
>
> 1. In the Cloud Console, go to: IAM & Admin > Service Accounts
>
> 2. Click on 'CREATE SERVICE ACCOUNT'
>
> ![Create service account](https://res.cloudinary.com/mintlify/image/upload/v1659304875/elementary/create-service-account_birda0.png)
>
> 3. Fill in the service account name ('elementary') and account description ('Elementary Data') and click 'CREATE AND
>    CONTINUE':
>
> ![Create and Continue](https://res.cloudinary.com/mintlify/image/upload/v1659304876/elementary/create-continue_ai6ta9.png)
>
> 4. Now we need to configure the relevant permissions for this new service account.
>    Select the following role: BigQuery Job User (you will need to grant read access to specific datasets later).
>
> 5. The last step is optional, skip it and press done.
>
> 6) Press on the dots icon to the right of your screen for your new service account and select 'Manage keys':
>
> ![Manage keys](https://res.cloudinary.com/mintlify/image/upload/v1659304878/elementary/dots-icon_aafqkv.png)
>
> 7. Press on 'ADD KEY' and select 'Create new key':
>
> ![Add Key](https://res.cloudinary.com/mintlify/image/upload/v1659304875/elementary/add-key_v4hztr.png)
>
> 8. Use the 'JSON' option radio button and press 'CREATE':
>
> ![JSON](https://res.cloudinary.com/mintlify/image/upload/v1659304877/elementary/json_mybzda.png)
>
> 9. This will automatically generate and download a JSON file with your private key information for this service account.
>    This JSON file provides the credentials to programmatically connect and work with your BigQuery environment.
>
> ### Grant service user access
>
> In order for a service user to work with Elementary Cloud, it requires the following permissions:
>
> * Role **"BigQuery Data Viewer"** on your Elementary dataset.
>
> * Roles **"BigQuery Metadata Viewer"** and **"BigQuery Resource Viewer"** at the **project level** for the project containing your dbt models and any external projects it references.
>
> > [!NOTE]
> >
> > "BigQuery Metadata Viewer" and "BigQuery Resource Viewer" **must be granted at the project level** (via IAM & Admin), not at the dataset level. These roles include permissions such as `bigquery.jobs.listAll` which are required for reading query history and metadata from `INFORMATION_SCHEMA` — they cannot be granted on individual datasets.
>
> #### Step 1: Grant project-level roles
>
> These roles are required for Elementary to read query history (`INFORMATION_SCHEMA.JOBS`) and table metadata. Without them, BI integrations (such as Hex) that rely on query history will not function.
>
> 1. Go to [IAM & Admin](https://console.cloud.google.com/iam-admin/iam) in the Google Cloud Console.
>
> 2. Select the project that contains your dbt models.
>
> 3. Click **"Grant Access"** at the top.
>
> 4. Fill out the form:
>    * In the "New principals" field, enter the **email address** of the Elementary service account.
>    * Add the role **"BigQuery Metadata Viewer"** (`roles/bigquery.metadataViewer`).
>    * Click **"Add another role"** and add **"BigQuery Resource Viewer"** (`roles/bigquery.resourceViewer`).
>    * Click **"Save"**.
>
> 5. **Repeat for any additional GCP projects** referenced in your dbt sources or used by BI tools (e.g., Hex, Looker) for query execution.
>
> #### Step 2: Grant dataset-level role on the Elementary dataset
>
> This role provides read access to your Elementary schema data.
>
> 1. Go to your project in [BigQuery console](https://console.cloud.google.com/bigquery).
>
> 2. In the "Explorer" tab, find your Elementary dataset.
>
> 3. Click on the three dots icon next to the dataset name, then Share.
>
> ![Dataset share](https://res.cloudinary.com/diuctyblm/image/upload/f_auto,q_auto/v1/dwh/bigquery/dataset_share)
>
> 4. Click the "ADD PRINCIPAL" button on the top right corner.
>
> ![Add principal](https://res.cloudinary.com/diuctyblm/image/upload/f_auto,q_auto/v1/dwh/bigquery/add_principal_button)
>
> 5. Fill out the form:
>    * In the "New principals" textbox, write the **email address** of your service account.
>    * In the "Select a role" dropdown menu, choose **"BigQuery Data Viewer"**.
>    * Click "Save".
>
> ![Grant access](https://res.cloudinary.com/diuctyblm/image/upload/f_auto,q_auto/v1/dwh/bigquery/grant_access_form)

> [tab] Workload Identity Federation
>
> ### Create a Workload Identity pool and provider:
>
> 1. In the Cloud Console, go to: IAM & Admin > Workload Identity Federation. Click 'CREATE POOL' (or select an existing pool), give it a name (e.g. `elementary-identity-pool`), and continue.
>
> 2. On the pool, click 'ADD A PROVIDER' and choose 'AWS'. Set the provider name to `Elementary AWS` and the AWS account ID to `743289191656` (Elementary's AWS account), then continue.
>
> ![Add AWS provider](https://mintcdn.com/elementary/GjTveqGnRvVPZM5I/pics/bigquery-wif/add_aws_provider.png?fit=max\&auto=format\&n=GjTveqGnRvVPZM5I\&q=85\&s=b95272cb04ed989a4d38ee526ecb99be)
>
> 3. In the provider's attribute conditions, add the following condition. This is the Elementary role for GCP — only Elementary can authenticate through this provider. Then save the provider:
>
>    ```
>    attribute.aws_role == "arn:aws:sts::743289191656:assumed-role/elementary-gcp-wif-role-prod-elementary-cloud"
>    ```
>
> ![Attribute condition](https://mintcdn.com/elementary/GjTveqGnRvVPZM5I/pics/bigquery-wif/attribute_condition.png?fit=max\&auto=format\&n=GjTveqGnRvVPZM5I\&q=85\&s=a3c51e152a34dc50eae19b5259d5a12e)
>
> ### Grant BigQuery permissions to the identity pool principal:
>
> 4. Go to IAM & Admin > IAM and click 'GRANT ACCESS'. In 'New principals', paste the following principalSet. Replace `PROJECT_NUMBER` with your GCP project number and `POOL_NAME` with the pool you created above:
>
>    ```
>    principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_NAME/attribute.aws_role/arn:aws:sts::743289191656:assumed-role/elementary-gcp-wif-role-prod-elementary-cloud
>    ```
>
> 5. Assign the following project-level roles to the principalSet and save. You will grant `BigQuery Data Viewer` at the dataset level in the next step:
>    * **BigQuery Job User**
>    * **BigQuery Metadata Viewer**
>    * **BigQuery Resource Viewer**
>
> ### Grant principal access to specific datasets:
>
> 6. In the [BigQuery console](https://console.cloud.google.com/bigquery), find your Elementary dataset in the 'Explorer' tab. Click the three dots next to it > Share > ADD PRINCIPAL. Paste the same principalSet from step 4, select `BigQuery Data Viewer`, and save. If your dbt dataset is in a different project, also grant it `BigQuery Metadata Viewer` and `BigQuery Resource Viewer`.
>
> ### Download the credential configuration file:
>
> 7. Back in IAM & Admin > Workload Identity Federation, select your pool and click 'GRANT ACCESS' at the top.
>
> 8. Choose 'Grant access using federated identities (Recommended)' and click 'DOWNLOAD CONFIG'.
>
> ![Grant access using federated identities](https://mintcdn.com/elementary/GjTveqGnRvVPZM5I/pics/bigquery-wif/grant_access_federated_identities.png?fit=max\&auto=format\&n=GjTveqGnRvVPZM5I\&q=85\&s=6f5f8e9236576b42386a8469734722fa)
>
> 9. In the 'Configure your application' panel, select the provider you created (e.g. `Elementary AWS`) from the Provider dropdown, then click 'DOWNLOAD CONFIG' to download the JSON configuration file.
>
> ![Select provider and download config](https://mintcdn.com/elementary/GjTveqGnRvVPZM5I/pics/bigquery-wif/configure_application_download_config.png?fit=max\&auto=format\&n=GjTveqGnRvVPZM5I\&q=85\&s=6dc856a5b313b82228aa3c51d62925a8)
>
> 10. Upload this JSON file to Elementary Cloud in the connection form below.

#### Permissions and security

Elementary Cloud doesn't require read permissions to your tables and schemas, but only the following:

* Read-only access to the Elementary schema.
* Access to read metadata in `INFORMATION_SCHEMA` (table metadata and query history), related to the tables in your dbt project.

It is recommended to create a user using the instructions specified above to avoid granting excess privileges.
For more details, refer to [security and privacy](https://docs.elementary-data.com/cloud/general/security-and-privacy.md).

For BigQuery, this translates to:

* **BigQuery Data Viewer** on the Elementary dataset (for reading Elementary schema data).
* **BigQuery Metadata Viewer** and **BigQuery Resource Viewer** at the **project level** (for reading `INFORMATION_SCHEMA.JOBS` query history and table metadata). These project-level roles are required for BI integrations like Hex that rely on query history.

### Fill the connection form

Use the **Authentication method** toggle at the top of the form to select either **Service account** or **Workload Identity Federation**, matching the method you set up above. The credentials upload field changes based on your selection:

* **Service account file** (Service account method): the service account JSON key file you downloaded.
* **WIF credential file** (Workload Identity Federation method): the external account JSON configuration file you downloaded from the identity pool.

Then provide the remaining fields:

* **Project**: The name of your BigQuery project.
* **Elementary dataset**: The name of your Elementary dataset. Usually `[dataset name]_elementary`.
* **Location**: Use this field to configure the location of BigQuery datasets as per [the BigQuery documentation](https://cloud.google.com/bigquery/docs/locations).

### Add the Elementary IP to allowlist

Elementary IP for allowlist: `3.126.156.226`

### Need help with onboarding?

We can provide [support on Slack](https://elementary-data.com/community) or hop on an [onboarding call](https://savvycal.com/MaayanSa/df29881c).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.