Installer Guides

BigQuery

Connecting BigQuery to LightBeam

Overview

LightBeam Spectra users can connect various data sources to the LightBeam application and these data sources will be continuously monitored for PII, PHI data.

Examples: BigQuery, SMB, MySQL, PostgreSQL, etc.

Onboarding BigQuery Data Source

  1. Login to your LightBeam Instance.

  2. Click on DATASOURCES on the Top Navigation Bar.

  3. Click on “Add a data source”.

    Figure 1. Add Data Source

    1. Search for “BigQuery”.

Figure 2. BigQuery Data Source

  1. Fill in the details as shown below and click Next:

Basic Information

  1. Instance Name: This is the unique name given to the data source.

  2. Description: This is an optional field needed to describe the use of this data source.

  3. Assign owner: Email address of the person responsible for this data source which will get alerts by default.

  4. Entity Creation: LightBeam Spectra detects and associates attributes based on the context and identifies whose data it is; these are called entities. Example: Jane Doe is an entity for whom LightBeam Spectra might have detected Name and SSN in a monitored data source.

  5. Source of Truth: LightBeam Spectra includes monitored data sources that serve as a single point of truth. These sources are utilized for looking up entities/attributes to verify the accuracy of attributes/entities discovered in other data sources. By using a Source of Truth dataset, entities are formulated based on the attributes present in the data.

  6. Location: The location of the data source.

  7. Purpose: The purpose of the data being collected/processed.

  8. Stage: The stage of the data source. Example: Source, Processing, Archival, etc.

Figure 3. BigQuery Configuration - Basic Info

  1. Insert the credentials as shown below:

Figure 4. BigQuery Configuration - Connection details

  1. Verify that you get the message Test Connection Success on the screen. Click on Next.

Figure 5. BigQuery Configuration - Test Connection Success

  1. In this step, select specific project(s) that you have to scan from the drop-down list.

Figure 6. BigQuery Configuration - Select project

  1. Check the tickboxes next to the project you would like to add. To add or remove datasets within the project, you can use the '➕' or '➖' signs.

Figure 6.1 BigQuery Configuration - Select project

Now we are ready to connect to the test database and proceed.

Click on Register.

This will show you the following message:

Figure 7. BigQuery - Register Datasource

Click on Okay.

Now you can browse the updated datasource.

APPENDIX

Creating a new role in BigQuery

Role: A role is a named set of access privileges that can be granted to users or other roles. These privileges determine what actions a user can perform and on which database objects.

  1. In the Google Cloud console, go to IAM & Admin -> Roles.

Figure 8. Create a new role.

  1. Click on Create Role.

Figure 8.1 Create a new role

  1. Fill in the role name.

Figure 8.2 Create a new role - Assign Permissions

Assign the following permissions to the role:

bigquery.bireservations.get 

bigquery.capacityCommitments.get 
bigquery.capacityCommitments.list 

bigquery.connections.get 
bigquery.connections.getIamPolicy 
bigquery.connections.list 
bigquery.connections.use 

bigquery.datasets.get 
bigquery.datasets.getIamPolicy 

bigquery.jobs.create 
bigquery.jobs.get 
bigquery.jobs.list 
bigquery.jobs.listAll 
bigquery.jobs.listExecutionMetadata 

bigquery.models.export 
bigquery.models.getData 
bigquery.models.getMetadata 
bigquery.models.list 

bigquery.readsessions.create 
bigquery.readsessions.getData 
bigquery.readsessions.update 

bigquery.reservationAssignments.list 
bigquery.reservationAssignments.search 
bigquery.reservations.get 
bigquery.reservations.list 

bigquery.routines.get 
bigquery.routines.list 

bigquery.rowAccessPolicies.getFilteredData 

bigquery.tables.createSnapshot 
bigquery.tables.export 
bigquery.tables.get 
bigquery.tables.getData 
bigquery.tables.getIamPolicy 
bigquery.tables.list 

resourcemanager.projects.get

  1. Navigate to Service Accounts and then click on Create Service Account.

Figure 8.3 Create a new role - Create service account

  1. Enter the required details, such as the name of the service account. Click Create And Continue.

Figure 8.4 Create a new role - Service account details

  1. Grant the role you created in the previous step to this service account and click on Continue And Save.

Figure 8.4 Create a new role - Service account created

  1. Once the service account has been created, select it. Navigate and click on Keys.

Figure 8.5 Create a new role - Click on Keys

  1. Choose Create New Key.

Figure 8.6 Create a new role - Create New Key

  1. For Key Type, choose JSON and then click Create.

Figure 8.7 Create a new role - Select Key Type

10. Download the JSON file and convert it to base64 by using the following command

cat <path of downloaded file> | base64

Now use the JSON key to connect the datasource.

About LightBeam

LightBeam automates Privacy, Security, and AI Governance, so businesses can accelerate their growth in new markets. Leveraging generative AI, LightBeam has rapidly gained customers’ trust by pioneering a unique privacy-centric and automation-first approach to security. Unlike siloed solutions, LightBeam ties together sensitive data cataloging, control, and compliance across structured and unstructured data applications providing 360-visibility, redaction, self-service DSRs, and automated ROPA reporting ensuring ultimate protection against ransomware and accidental exposures while meeting data privacy obligations efficiently. LightBeam is on a mission to create a secure privacy-first world helping customers automate compliance against a patchwork of existing and emerging regulations.

For any questions or suggestions, please get in touch with us at: [email protected].

PreviousAurora (MySQL)NextAWS Redshift

Last updated