Databricks setup

​

Part 1: Databricks Billable Usage Ingestion

Set up a Databricks app so ConfigView can pull your Azure Databricks billable usage (DBU consumption) and workspace inventory into the dashboard. This is the per-workspace, per-cluster, per-SKU detail behind the single Azure Databricks line item on your Azure invoice.

---

​

Step 1: Find your Databricks account ID

1. Sign in to the Azure Databricks account console as an account admin.
2. Click your username in the top-right corner — the Account ID (a UUID) is shown in the menu.
3. Copy it. This becomes the DATABRICKS_ACCOUNT_ID secret.

---

​

Step 2: Create a Microsoft Entra service principal

ConfigView authenticates to the Databricks account API using a Microsoft Entra ID app registration (service principal). You can reuse the same app registration you created for the Azure integration, or create a dedicated one.

​

2a. Register the app (or reuse the Azure one)

1. In the Azure portal, go to Microsoft Entra ID → App registrations → ➕ New registration.
2. Name it ConfigView Databricks, leave the defaults, and click Register.
3. On the Overview page, copy the Application (client) ID and the Directory (tenant) ID — these become DATABRICKS_AZURE_CLIENT_ID and DATABRICKS_AZURE_TENANT_ID.

​

2b. Create a client secret

1. Open Certificates & secrets → Client secrets → ➕ New client secret.
2. Add a description and expiry, click Add, and immediately copy the secret’s Value — it becomes DATABRICKS_AZURE_CLIENT_SECRET.

---

​

Step 3: Add the service principal to the Databricks account as an admin

The billable usage download API only responds to an account admin. Add the Entra service principal to the Databricks account and grant it the admin role.

1. In the account console, go to User management → Service principals → Add service principal.
2. Choose Microsoft Entra ID managed and paste the Application (client) ID from Step 2a.
3. Open the service principal and enable the Account admin role.

---

​

Step 4: Add the secrets to ConfigView

1. Go to your ConfigView dashboard: https://{companyname}.configview.com/admin/secret/
2. Click Add Secret and create each of the following:

3. Click Save.

---

​

Step 5: Enable the Databricks App in ConfigView

1. Go to: https://{companyname}.configview.com/admin/cron/
2. You should see Databricks in the list of available apps.
3. Select the scripts you want to run (Workspaces and Billable Usage).
4. Click Save.

---

​

Step 6: Verify

1. Go to: https://{companyname}.configview.com/admin/status/
2. Run the Databricks health check.
3. All checks should pass.

If the Billable usage download check returns a 403, the service principal is not yet an account admin (Step 3). A 404 means the download API is not enabled for your account — see the note below.

---

​

Data Tables

Once the scripts run, these Databricks tables are created in your database. All tables include a run_at column for historical tracking; the latest snapshot is the rows with the most recent run_at.

• databricks_workspaces — account workspace inventory: workspace name, deployment name, Azure region (location), pricing tier, status, and the Azure subscription / resource group each workspace bills into.
• databricks_billable_usage — every documented column of the billable usage log, one row per workspace / usage day / cluster / SKU:
  • workspace_id, usage_date, usage_timestamp
  • cluster_id, cluster_name, cluster_node_type
  • cluster_owner_user_id, cluster_owner_user_name (cluster creator’s email — captured only when the service principal has PII access)
  • cluster_custom_tags, tags (JSON)
  • sku (billing SKU, e.g. PREMIUM_ALL_PURPOSE_COMPUTE)
  • dbus (Databricks Units consumed), machine_hours

Join databricks_billable_usage.workspace_id to databricks_workspaces.workspace_id to attribute DBU spend to named workspaces, Azure regions, and subscriptions.