> ## Documentation Index
> Fetch the complete documentation index at: https://support.configview.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Google Cloud setup

ConfigView pulls Google Cloud data through **one organization-scoped service account** plus a few `*_ID` secrets. The service account uses the Cloud Asset API to enumerate resources across every project in your organization in a single call per asset type — no per-project iteration, no per-region loops.

You will end up with a handful of secrets in ConfigView — the 3 core IDs (`GCP_ORG_ID`, `GCP_CUSTOMER_ID`, `GCP_WORKSPACE_ADMIN_EMAIL`), plus the 3 `GCP_BILLING_*` secrets if you enable billing detail — plus **one uploaded JSON file** (`google_cloud.json`, the service account key).

**Prerequisite — enable the required APIs** on the project that hosts the service account (the one you pick in Step 2). Every call is billed/quoted against that project, and on a fresh project several of these are **off by default**:

* `cloudasset.googleapis.com` — all Asset-API endpoints (compute, storage, IAM bindings, …); the bulk of the 27
* `cloudresourcemanager.googleapis.com` — Projects, Organizations, Folders
* `cloudbilling.googleapis.com` — Billing Accounts
* `iam.googleapis.com` — Service Accounts, Service Account Keys
* `serviceusage.googleapis.com` — APIs Enabled (per Project); usually already on
* `logging.googleapis.com` — Audit Logs (Admin Activity)
* `bigquery.googleapis.com` — billing cost + pricing (queries the BigQuery export)
* `billingbudgets.googleapis.com` — Billing Budgets
* `admin.googleapis.com` — **only** for Cloud Identity Users (optional, Step 6)

Enable them in one command (swap in your SA's host project):

```bash theme={null}
gcloud services enable cloudasset.googleapis.com cloudresourcemanager.googleapis.com \
  cloudbilling.googleapis.com iam.googleapis.com serviceusage.googleapis.com \
  logging.googleapis.com bigquery.googleapis.com billingbudgets.googleapis.com \
  --project=YOUR_SA_HOST_PROJECT
```

> **Propagation.** After enabling, `SERVICE_DISABLED` 403s can persist **intermittently for \~10–15 minutes** — some calls succeed while others still fail as the change rolls out across Google's systems. Wait it out before assuming a misconfiguration.

***

## Step 1: Identify your IDs

You need three identifiers up-front:

1. **`GCP_ORG_ID`** — Open [https://console.cloud.google.com/iam-admin/settings](https://console.cloud.google.com/iam-admin/settings), switch to your organization in the picker, copy the **Organization ID** (e.g. `123456789012`).
2. **`GCP_CUSTOMER_ID`** — In the same Organization Settings page, copy the **Directory customer ID** (begins with `C`, e.g. `C03az79cb`). This is your Cloud Identity / Workspace customer ID.
3. **`GCP_WORKSPACE_ADMIN_EMAIL`** — Only required for the **Cloud Identity Users** endpoint. The email of a Workspace admin the service account will impersonate via DWD. Skip this if you're not enabling Cloud Identity Users.

***

## Step 2: Create the service account

1. Open [https://console.cloud.google.com/iam-admin/serviceaccounts](https://console.cloud.google.com/iam-admin/serviceaccounts)
2. Pick (or create) a project to host the service account — `configview-integration` is conventional, but any project the SA can use as its "home" works
3. Click **Create Service Account**
4. **Name:** `configview`
5. Click **Create and Continue** — we'll add roles in the next step
6. Click **Done** (you can skip the optional grant-users step)

***

## Step 3: Grant the service account org-level roles

Switch to **IAM** at the **organization** level (not project): [https://console.cloud.google.com/iam-admin/iam](https://console.cloud.google.com/iam-admin/iam) → pick your organization in the picker → click **Grant Access**.

Add the service account email as a principal and assign:

| Role                                                               | Why                                                | Endpoints it unlocks                                                                                                          |
| ------------------------------------------------------------------ | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| **Browser** (`roles/browser`)                                      | Read project/folder/org metadata                   | Projects, Organizations, Folders                                                                                              |
| **Cloud Asset Viewer** (`roles/cloudasset.viewer`)                 | Bulk inventory via Asset API                       | Compute (all), Storage, SQL, BigQuery, Run, GKE, Functions, Pub/Sub, DNS, KMS, Secret Manager, Service Accounts, IAM Bindings |
| **Service Usage Viewer** (`roles/serviceusage.serviceUsageViewer`) | List enabled APIs per project                      | APIs Enabled (per Project)                                                                                                    |
| **Billing Account Viewer** (`roles/billing.viewer`)                | Read billing account list                          | Billing Accounts                                                                                                              |
| **Security Reviewer** (`roles/iam.securityReviewer`)               | Read IAM policies + service account keys           | IAM Bindings, Service Account Keys                                                                                            |
| **Logging Viewer** (`roles/logging.viewer`) — at the **org**       | Read Admin Activity audit logs across all projects | Audit Logs — Admin Activity                                                                                                   |

For the **billing detail** endpoints (Cost Summary/Detailed, Pricing, Budgets) the service account also needs, **not at the org but on the project that hosts the BigQuery billing export**:

| Role                                                   | Where                                          | Why                              |
| ------------------------------------------------------ | ---------------------------------------------- | -------------------------------- |
| **BigQuery Data Viewer** (`roles/bigquery.dataViewer`) | the billing-export project (e.g. `my-billing`) | Read the export + pricing tables |
| **BigQuery Job User** (`roles/bigquery.jobUser`)       | the SA's own host project                      | Run the aggregation queries      |

> **No billing export, no cost data.** GCP has no cost API — detailed spend only exists if you've enabled **Billing → BigQuery export** (Standard + Detailed usage cost, and optionally Pricing). If you haven't, set it up first ([console](https://console.cloud.google.com/billing) → Billing export), then point ConfigView at it via the secrets in Step 5. Budgets and the account→project map work without the export.

Click **Save**.

> **About `Cloud Asset Viewer`.** This role alone covers *most* of the heavy lifting. If you want a single-role install and don't mind slightly less granular reads, you can grant **`roles/cloudasset.viewer` + `roles/browser`** and the majority of the 27 endpoints will function. The other roles fix the gaps for billing, service-usage, IAM policy, and SA keys.

> **Cloud Identity Users needs no org role.** That endpoint reads the user directory through the Admin Directory API via **domain-wide delegation** (Step 6), not a GCP IAM grant — so there's nothing to add here for it. (There is no `roles/cloudidentity.viewer`; the predefined Cloud Identity roles are `roles/cloudidentity.groups.readonly` and friends, none of which this integration uses.)

***

## Step 4: Create + upload the JSON key

1. From the service account list, click your `configview` service account
2. Open the **Keys** tab → **Add Key** → **Create new key** → **JSON**
3. Click **Create** — the JSON file downloads automatically
4. Upload the JSON to your ConfigView satellite at `<satellite-root>/json/google_cloud.json` (your ConfigView administrator can do this — it's NOT a Secret Manager secret because the file is bigger and is referenced directly by all scripts)

> **Where the file lives.** The scripts resolve `<project_root>/json/google_cloud.json`, where `<project_root>` is the dashboard-backend root. On a satellite this is `/opt/configview-dashboard-backend/json/google_cloud.json` — the same directory that already holds `secret-manager.json` and `google_admin.json`. Permissions: `chmod 400`, owner `configview:configview`.

***

## Step 5: Add the secrets to ConfigView

1. Go to your ConfigView dashboard: `https://{companyname}.configview.com/admin/secret/`
2. Click **Add Secret** and create:
   * `GCP_ORG_ID`: from Step 1.1 (numeric, e.g. `123456789012`)
   * `GCP_CUSTOMER_ID`: from Step 1.2 (starts with `C`)
   * `GCP_WORKSPACE_ADMIN_EMAIL`: from Step 1.3 (admin email to impersonate, only needed for Cloud Identity Users)
3. For the **billing detail** endpoints, also add (skip if you're not enabling cost/pricing/budgets):
   * `GCP_BILLING_BQ_PROJECT`: the project that holds the BigQuery billing export (e.g. `my-billing`)
   * `GCP_BILLING_BQ_DATASET`: the dataset name within it (e.g. `billing`)
   * `GCP_BILLING_ACCOUNT_ID`: the billing account ID with dashes (e.g. `01ABCD-23EFGH-45IJKL`)
   * *(optional)* `GCP_BILLING_DETAILED_DAYS` (default `7`), `GCP_AUDIT_LOOKBACK_HOURS` (default `24`), `GCP_AUDIT_MAX_ENTRIES` (default `100000`)
4. Click **Save**

***

## Step 6: Domain-Wide Delegation (only for Cloud Identity Users)

Skip this step if you're not enabling the **Cloud Identity Users** endpoint. The endpoint duplicates google-workspace data, so most customers already running google-workspace can leave it disabled.

If you do want it:

1. From the service account list, click your `configview` SA → **Details** → expand **Advanced settings** → copy the **Client ID**
2. Sign in to your Google Workspace admin console as a super admin: [https://admin.google.com](https://admin.google.com)
3. Go to **Security** → **Access and data control** → **API controls** → **Manage Domain Wide Delegation**
4. Click **Add new** and paste:
   * **Client ID:** (from above)
   * **OAuth scopes:** `https://www.googleapis.com/auth/admin.directory.user.readonly`
5. Click **Authorize**

The SA can now impersonate the admin email you set in `GCP_WORKSPACE_ADMIN_EMAIL` to read the user directory.

***

## Step 7: Enable the Google Cloud scripts

1. Go to: `https://{companyname}.configview.com/admin/cron/`
2. You should see **google-cloud** in the list of available apps
3. Enable scripts by tier — turn on the parents first; children will run automatically after them:

### Identity & governance

| Script                                    | Notes                                                                                                                       |
| ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| **Projects**                              | All projects across the org. Most other scripts implicitly depend on this list.                                             |
| **Organizations**                         | Usually one row. Enable for completeness.                                                                                   |
| **Folders**                               | Folder hierarchy. **Depends on Organizations.**                                                                             |
| **Billing Accounts**                      | All accounts the SA can view                                                                                                |
| **APIs Enabled (per Project)**            | One row per (project × enabled service). **Depends on Projects.** Can be large — hundreds to thousands of rows.             |
| **IAM Bindings (Org + Folder + Project)** | Flattened binding × member rows across every resource the SA can see                                                        |
| **Service Accounts**                      | All SAs across the org                                                                                                      |
| **Service Account Keys**                  | One row per (SA × key). **Depends on Service Accounts.** Lists `keyType` (`USER_MANAGED` is the audit target)               |
| **IAM Principals (Derived)**              | One row per unique principal aggregated from IAM Bindings. **Depends on IAM Bindings.** No API call — pure SQL aggregation. |
| **Cloud Identity Users**                  | Same data as google-workspace users. Enable only if google-workspace isn't running. Requires DWD (Step 6).                  |

### Compute

| Script                                | Notes                                                                                            |
| ------------------------------------- | ------------------------------------------------------------------------------------------------ |
| **Compute Instances (VMs)**           | Renamed from `gcp_get_org_vm`; historical data preserved via migration if you ran the old script |
| **Compute Disks**                     | All persistent disks                                                                             |
| **Compute Networks (VPCs)**           | All VPC networks                                                                                 |
| **Compute Subnetworks**               | All subnetworks                                                                                  |
| **Compute Firewalls**                 | All firewall rules — useful for security audits                                                  |
| **Compute External IPs**              | Static + ephemeral external IPs (`Address` resources)                                            |
| **Load Balancers (Forwarding Rules)** | All forwarding rules across global + regional LBs                                                |

### Storage & data

| Script                    | Notes                                           |
| ------------------------- | ----------------------------------------------- |
| **Cloud Storage Buckets** | All GCS buckets                                 |
| **Cloud SQL Instances**   | All SQL instances (MySQL, Postgres, SQL Server) |
| **BigQuery Datasets**     | All BQ datasets                                 |
| **Cloud Run Services**    | All Cloud Run services                          |
| **GKE Clusters**          | All GKE clusters                                |
| **Cloud Functions**       | Gen-1 (`CloudFunction`) + Gen-2 (`Function`)    |
| **Pub/Sub Topics**        | All Pub/Sub topics                              |
| **Cloud DNS Zones**       | All managed DNS zones                           |

### Security

| Script                                | Notes                                                                                                                                                                |
| ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **KMS Crypto Keys**                   | All keys across all key rings                                                                                                                                        |
| **Secret Manager Secrets (Metadata)** | Names + replication settings only — **never the secret payload**                                                                                                     |
| **Audit Logs — Admin Activity**       | Who created/changed/deleted resources, across every project + the org, for the last 24h (rolling). **Depends on Projects.** Needs `roles/logging.viewer` at the org. |

### Cost & billing

These query the **BigQuery billing export** (see the export note in Step 3) and need the `GCP_BILLING_*` secrets.

| Script                                               | Notes                                                                                                                       |
| ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| **Billing Cost — Summary (Monthly)**                 | Net cost (cost + credits) by invoice month × project × service, current + previous month. The lightweight history table.    |
| **Billing Cost — Detailed (Daily / SKU / Resource)** | Per resource × SKU × day for the last 7 days (`GCP_BILLING_DETAILED_DAYS`). High row count; keeps only the latest snapshot. |
| **Billing Pricing (Rate Card)**                      | List + negotiated SKU prices, restricted to SKUs you've used in the last 60 days.                                           |
| **Billing Budgets**                                  | Budget amounts, threshold alert rules, and scope filters. Needs `billingbudgets.googleapis.com`.                            |
| **Billing — Projects on Account**                    | Which projects bill to the account and whether billing is enabled — catches drift.                                          |

4. Click **Save**

***

## Step 8: Verify

1. Go to: `https://{companyname}.configview.com/admin/cron/`
2. Expand **google-cloud** and click the **Health check** button on the app header
3. A passing run reports `overall: ok`. On failure, the toast lists the failing checks — and the same result is written to the **Activity** log and emailed to your alert address. The health check also runs automatically right after any google-cloud script fails, so failures are captured even if you don't trigger it by hand.
4. Projects, Organizations, and at least one Asset-API endpoint (e.g. Compute Instances) should pass

If a check fails:

* **`PERMISSION_DENIED` on Asset API** — `roles/cloudasset.viewer` not granted at the **organization** level (project-level isn't enough). Re-do Step 3 against the org.
* **`SERVICE_DISABLED: <API> has not been used in project X`** — That API isn't enabled on the SA's host project (`X` is the project *number*). Applies to `cloudasset`, `cloudresourcemanager`, `cloudbilling`, and `iam` — enable the full set from the Prerequisite at the top. Calls are billed/quoted against the SA's host project, not the org. Expect intermittent `SERVICE_DISABLED` for \~10–15 min after enabling while it propagates.
* **`PERMISSION_DENIED` on Service Usage** — Missing `roles/serviceusage.serviceUsageViewer`.
* **`PERMISSION_DENIED` on billing** — Missing `roles/billing.viewer`. Billing is granted at the **billing account** level, not org level, depending on your account structure.
* **`PERMISSION_DENIED` on Service Account Keys** — Missing `roles/iam.securityReviewer` OR the customer turned off SA key listing via org policy (`constraints/iam.disableServiceAccountKeyCreation` is unrelated; the relevant constraint is on read).
* **`unauthorized_client: Client is unauthorized to retrieve access tokens`** on Cloud Identity Users — DWD client ID not authorized, or wrong scope. Re-do Step 6.
* **Billing cost/pricing returns `Access Denied` or `Not found: Table`** — the SA is missing `roles/bigquery.dataViewer` on the billing-export project or `roles/bigquery.jobUser` on its own project, or a `GCP_BILLING_*` secret is wrong (project/dataset/account ID). Confirm the export table exists: `bq ls <project>:<dataset>`.
* **Audit Logs returns few/no rows** — `roles/logging.viewer` must be at the **org** so it covers all projects; Admin Activity is always on, but check the `GCP_AUDIT_LOOKBACK_HOURS` window. Transient `500 Internal error` on `entries.list` is retried and bad projects are skipped automatically.

***

## Data Tables

Selected highlights. See `manifest.json` for the full 33-table list.

| Table                          | Source                                          | Key Columns                                                                               |
| ------------------------------ | ----------------------------------------------- | ----------------------------------------------------------------------------------------- |
| `gcp_projects`                 | Cloud Resource Manager v3                       | project\_id, parent, state, display\_name, labels                                         |
| `gcp_billing_cost_summary`     | BigQuery standard export                        | invoice\_month, project\_id, service, cost, credits, net\_cost                            |
| `gcp_billing_cost_detailed`    | BigQuery detailed export                        | usage\_date, project\_id, service, sku\_id, resource\_name, net\_cost                     |
| `gcp_billing_pricing`          | BigQuery pricing export                         | sku\_id, service, pricing\_unit, list\_price\_usd, account\_price, discount\_percent      |
| `gcp_billing_budgets`          | Billing Budgets v1                              | name, display\_name, budget\_amount, currency\_code, threshold\_rules\_json               |
| `gcp_billing_account_projects` | Cloud Billing v1                                | billing\_account, project\_id, billing\_enabled                                           |
| `gcp_audit_admin_activity`     | Cloud Logging entries.list                      | log\_timestamp, principal\_email, method\_name, resource\_name, project\_id, status\_code |
| `gcp_organizations`            | Cloud Resource Manager v3                       | organization\_id, display\_name, directory\_customer\_id                                  |
| `gcp_folders`                  | Cloud Resource Manager v3                       | folder\_id, parent, display\_name                                                         |
| `gcp_billing_accounts`         | Cloud Billing v1                                | name, display\_name, open\_account, currency\_code                                        |
| `gcp_project_services`         | Service Usage v1                                | project\_id, service\_name, state                                                         |
| `gcp_iam_bindings`             | Cloud Asset SearchAllIamPolicies                | resource, project, role, member, member\_type                                             |
| `gcp_service_accounts`         | Cloud Asset (iam.googleapis.com/ServiceAccount) | email, project\_id, disabled, oauth2\_client\_id                                          |
| `gcp_service_account_keys`     | IAM v1 keys.list                                | service\_account\_email, key\_id, key\_type, valid\_before\_time                          |
| `gcp_iam_principals`           | Derived from gcp\_iam\_bindings                 | member, member\_type, email\_domain, binding\_count, roles\_json                          |
| `gcp_cloud_identity_users`     | Admin Directory v1 (impersonated)               | primary\_email, suspended, archived, is\_admin                                            |
| `gcp_compute_instances`        | Cloud Asset (Compute Instance)                  | name, project, location, state, labels\_json                                              |
| `gcp_compute_firewalls`        | Cloud Asset (Firewall)                          | name, project, raw\_json                                                                  |
| `gcp_gcs_buckets`              | Cloud Asset (Bucket)                            | name, project, location, labels\_json                                                     |
| `gcp_kms_keys`                 | Cloud Asset (CryptoKey)                         | name, project, location, additional\_attributes\_json                                     |
| `gcp_secret_manager_secrets`   | Cloud Asset (Secret)                            | name, project, labels\_json                                                               |

***

## What you can't ingest with this setup

* **Data Access / System Event / Policy Denied audit logs.** Only **Admin Activity** audit logs are ingested (config changes). Data Access logs are off by default, very high volume, and not pulled in this version.
* **Per-resource detailed metrics** (CPU, network throughput, etc.). The Asset API gives configuration + metadata. Real-time metrics live in Cloud Monitoring.
* **GCS bucket object listings.** Buckets yes; objects no — that would multiply the row count by many orders of magnitude.
* **Secret Manager secret payloads.** ConfigView reads names and metadata only. Payloads stay in GCP.

If you need Data Access audit logs or live metrics, ask your ConfigView contact — both are tracked roadmap items.