> For the complete documentation index, see [llms.txt](https://docs.lightbeam.ai/lxqobxw6ak7CTnsQjikH/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.lightbeam.ai/lxqobxw6ak7CTnsQjikH/core-features/spectra-ai/data-sources/cloud-platforms/aws-auto-discovery.md).

# AWS Auto Discovery

### 1. Introduction

AWS Auto Discovery streamlines the process of integrating your AWS resources with LightBeam. This feature automatically detects and registers supported AWS services, eliminating the need for manual data source registration. It also provides visibility into unsupported resources, helping you maintain an up-to-date inventory of your AWS environment.

### 2. Supported Services

#### 2.1 Fully Supported Services

* Amazon S3 (Simple Storage Service)
* Amazon RDS (Relational Database Service)
  * MySQL
  * PostgreSQL
  * Microsoft SQL Server
  * Oracle
* Amazon Redshift
* AWS Glue
* Amazon DynamoDB
* Amazon FSx (File Systems)

#### 2.2 Discoverable Services (Limited Support)

* Amazon EC2 (Elastic Compute Cloud)
* Amazon EFS (Elastic File System)
* Amazon S3 Glacier
* AWS Backup
* Amazon CloudWatch

### 3. Onboarding Process

#### 3.1 Create an IAM Role in AWS

To enable LightBeam to access your AWS resources, you need to create an IAM role with the necessary permissions. For detailed steps for creating the IAM role go to [#appendix](#appendix "mention").

#### 3.2 Accessing the Onboarding Screen

1. Navigate to the "**Datasources**" header in the main navigation.<br>
2. Click on the "**Cloud Platforms**" tab.<br>

   <figure><img src="/files/2LZ3FFYYqQqEMmhFEb2b" alt=""><figcaption></figcaption></figure>
3. Select **AWS** from the left sidebar menu.<br>
4. Click on the **Onboard Now** button.

\
**Alternative Method:**

* Scroll to the bottom of the **Datasources** page to find a list of supported cloud platforms.<br>
* Click on the **AWS** icon to start the onboarding process.<br>

  <figure><img src="/files/LCq7kDbxg9ZzwpVWbIoE" alt=""><figcaption></figcaption></figure>

#### 3.4 Entering AWS Account Details

1. After selecting "**AWS**" as the cloud platform,\ <br>

   <figure><img src="/files/SBgubaxwEZGtgdXogEx6" alt=""><figcaption></figcaption></figure>

enter the following AWS account details:

* `Name`
* `Description`(Optional)
* `Primary Owner`
* `Co-owner`(Optional)
* `AWS Account ID`
* `Access Key ID`
* `Secret Access Key`
* `Session Key`(Optional)\ <br>

  <figure><img src="/files/bOg3kyg6KbZLunsdO2KJ" alt=""><figcaption></figcaption></figure>

2. After entering the `Secret Key`, the system automatically runs an internal test connection API to validate the credentials.<br>
3. Once you see the message `Connection Verified`, proceed to the next step.

#### 3.5 Configuring Discovery Settings

1. Set the frequency for scanning resources:<br>

   * Options include daily, weekly, or monthly scans.<br>

   <figure><img src="/files/Ka6qEtMgKjRpcXPUwjG9" alt=""><figcaption><p><br></p></figcaption></figure>
2. #### Selecting Automatic Registration

   * Choose data sources for automatic registration:

     * A dropdown list of supported data sources will be available.
     * **Example**: S3 buckets can be set for auto-registration upon discovery.<br>

     <figure><img src="/files/JMDBLO2kvob0sApBCg4A" alt=""><figcaption></figcaption></figure>
3. Click "**Save**" to confirm your settings and initiate the discovery process.<br>

   <figure><img src="/files/yceuA2Q3VTJsXmhY7CtK" alt=""><figcaption></figcaption></figure>

#### 4. Resource Discovery Process

1. After saving, an in progress message will appear: `Resource discovery is in progress, and this process may take some time.`<br>

   <figure><img src="/files/WMgywnAA7yf12QDqn7bw" alt=""><figcaption></figcaption></figure>
2. During this process:<br>
   * A Terraform workflow is triggered.<br>
   * This workflow creates necessary IAM roles and policies.<br>
   * The created roles and policies are assigned to the appropriate service account.<br>
3. The process typically takes about *5 minutes* for a single AWS account.

#### 3.6 Reviewing Discovered Resources

1. Refresh the page to view the results.

<figure><img src="/files/Kjk6Diim7dD2oxiBsg0L" alt=""><figcaption></figcaption></figure>

2. You'll see your AWS account listed.<br>
3. Discovered resources are displayed in a table format with the following details:
   * Data source name and type&#x20;
   * Account name
   * Region
   * Owner
   * Status (Registered or Unregistered)
   * Date added

{% hint style="info" %}

### Naming Convention for Discovered Resources:

LightBeam uses a specific naming convention for automatically discovered AWS resources. This convention helps in easily identifying and categorizing different types of resources across your AWS account.

**1. Data Sources with One Instance per Account**

For resources that typically have one instance per AWS account, such as S3 and DynamoDB:

**Format:** `<Data source type>_<Account Name>`

**Examples:**

* S3\_Lightbeam
* DynamoDB\_Lightbeam<br>

**2. Data Sources with One Instance per Region**

For resources that are limited to one instance per region, such as AWS Glue:

**Format:** `<Data source type>_<region>_<Account Name>`

**Example:**

* Glue\_us-east-1\_Lightbeam<br>

**3. Data Sources with Multiple Instances per Region**

For resources that can have multiple instances within a single region:

**Format:** The name will be the same as the cluster or instance name in AWS.

**Example:**

* If you have an RDS instance named "production-db" in AWS, it will appear as "production-db" in LightBeam.<br>

**4. Handling Duplicate Names**

In cases where duplicate names are detected:

* The first instance will use the standard naming convention.
* Subsequent instances with the same name will have suffixes appended: \*1, \*2, and so on.

**Example:**

* First instance: MyDatabase
* Second instance with the same name: MyDatabase\*1
* Third instance with the same name: MyDatabase\*2

***

This naming convention helps in maintaining consistency and clarity across your discovered AWS resources in LightBeam. It allows for easy identification of resource types, associated AWS accounts, and, where applicable, the AWS region.

***

{% endhint %}

2. Initial status:
   * All discovered resources are marked as "`Unregistered`" with an orange status icon.<br>
   * Each resource has a "**Register**" button for manual registration.

3. Discovery scope:
   * Regional services: `RDS`, `Redshift`, `Glue`, `FSx`<br>
   * Organization-level services: `S3`, `DynamoDB`<br>

4. Search Bar: Enables quick resource lookup using keywords.<br>

5. Filter Options: Allows refinement of results based on specific criteria.<br>

6. Status Tabs: Toggles between "Supported" and "Unsupported" resources.

***

### 4. Registration Process

#### 4.1 Automatic Registration

* Pre-selected services are registered automatically upon discovery.

#### 4.2 Manual Registration (Example: Redshift)

1. Click on the "**Register**" button next to the Redshift datasource.<br>

   <figure><img src="/files/uo1h5h98uE9XSJtzd1RF" alt=""><figcaption></figcaption></figure>
2. Add basic details for the Redshift data source and click on **Next**.<br>

   <figure><img src="/files/h1fobINYQv3PDD5ikDHj" alt=""><figcaption><p><br></p></figcaption></figure>

   Since IAM roles are already set up, the Host, Port and Frequency of Scanning fields are prefilled.<br>
3. Enter the **Username** and **Password**.<br>

   <figure><img src="/files/VU9jQQVBIphAxazcTu1V" alt=""><figcaption></figcaption></figure>
4. Click "**Test Connection**" to verify access.<br>
5. The rest of the workflow is the same as described in [AWS Redshift](/lxqobxw6ak7CTnsQjikH/core-features/spectra-ai/data-sources/databases-and-datalakes/aws-redshift.md) document.

#### 4.3 Post-Registration

* The registered data source will appear in your list of data sources.<br>
* In the Cloud Platforms view, the status for AWS S3 will update to "Sync On".<br>
* You can click on the data sourc name to access its dashboard.

### 5. Operational Procedures

#### 5.1 Manual Sync

Users can trigger a manual sync to update the status of discovered resources:

1. Navigate to the Cloud Platforms dashboard<br>
2. Click the "Manual Sync" button<br>
3. Wait for the sync process to complete

#### 5.2 Error Handling

* Resources with errors during discovery or registration are flagged with appropriate status indicators<br>
* Users can attempt to resolve issues and re-run the registration process

#### 5.3 Additional Views

An "All Datasources" section is available with two tabs:

* Scanning: Shows data sources currently being scanned.<br>
* Unregistered: Displays discovered but not yet registered data sources.

### 6. Security Considerations

* High-privileged credentials used during onboarding are discarded after initial setup<br>
* IAM roles with least-privilege permissions are created for ongoing operations<br>
* All communications with AWS APIs use secure, encrypted connections

### 7. Limitations and Future Enhancements

* Some discoverable services (e.g., EC2, CloudWatch) have limited metadata available<br>
* Future versions may include enhanced support for currently limited services<br>
* Additional AWS services may be added to the list of fully supported services in upcoming releases

***

### [Appendix](#appendix)

### Required IAM Permissions

For LightBeam AWS Auto Discovery to function properly, the AWS IAM user needs the following permissions. Create a policy with this JSON:

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sts:GetCallerIdentity",
                "eks:DescribeCluster",
                "ec2:DescribeTags",
                "iam:CreateRole",
                "iam:CreatePolicy",
                "iam:AttachRolePolicy",
                "iam:SimulatePrincipalPolicy",
                "organizations:DescribeAccount",
                "iam:GetRole",
                "iam:GetPolicy",
                "iam:GetPolicyVersion",
                "iam:ListRolePolicies", 
                "iam:ListAttachedRolePolicies",
                "iam:ListInstanceProfilesForRole",
                "iam:ListPolicyVersions"
            ],
            "Resource": "*"
        }
    ]
}
```

### 1. Create IAM Policy

1. Navigate to Console Home page<br>
2. Click on IAM&#x20;

<figure><img src="/files/mUTelOJd2ILGthpWURhl" alt=""><figcaption></figcaption></figure>

3. Click on "Policies" in left sidebar.

<figure><img src="/files/3eM0Zq9oO54VwGQsMDYr" alt=""><figcaption></figcaption></figure>

4. Click the "Create policy" button.

<figure><img src="/files/uPHxS0q70aLmMoE3TGQS" alt=""><figcaption></figcaption></figure>

5. Select "JSON" editor tab.

<figure><img src="/files/1g8ECU9WiKy2TCCoKU8M" alt=""><figcaption></figcaption></figure>

6. Configure the policy by erasing the existing code in editor.

<figure><img src="/files/CbfPtUwA7RUHpEHkzyzz" alt=""><figcaption></figcaption></figure>

7. Type the required JSON policy.

<figure><img src="/files/JTYkeyAwZUFrw4jJFUq6" alt=""><figcaption></figcaption></figure>

8. Click on "Next".<br>
9. Give a name to the policy, for example: "`lightbeam-aws-discovery-policy-test1`".<br>
10. Click on "Create policy".

<figure><img src="/files/sSiJwM3CWc6EcHL5eNLz" alt=""><figcaption></figcaption></figure>

### 2. User Creation

1. Navigate to IAM → Users

<figure><img src="/files/WJPsR7C6mz6hogHO4u1H" alt=""><figcaption></figcaption></figure>

* Click "Create user" button

<figure><img src="/files/pDSWeahhTJFekUmssLYr" alt=""><figcaption></figcaption></figure>

* Set User Details
  * Username: "lightbeam-aws-discovery-user-test1"<br>
* Click "Next"

<figure><img src="/files/xhSGDyi2x5pqLrqmyuyn" alt=""><figcaption></figcaption></figure>

* Set Permissions
  * Select "Attach policies directly"

<figure><img src="/files/OkvG0XT1xcW0ey2igGPQ" alt=""><figcaption></figcaption></figure>

* Search for "lightbeam-aws-discovery-policy-test1".<br>
* Check the box next to the policy.

<figure><img src="/files/6kyj8BkXYtpTtUgs7001" alt=""><figcaption></figcaption></figure>

* Click "Next".

<figure><img src="/files/el5VuCrBomi0jNMNzc7I" alt=""><figcaption><p><br></p></figcaption></figure>

* Click "Create user"

<figure><img src="/files/LdlYfi899Ci26er8J4Uh" alt=""><figcaption></figcaption></figure>

### 3. Access Key Generation&#x20;

1. Access User Credentials<br>
   * Search for created user "lightbeam-aws-discovery-user-test1"\
     &#x20;
   * Click on created user "lightbeam-aws-discovery-user-test1".<br>

<figure><img src="/files/GjTua8gAkV1zJzvpmOnT" alt=""><figcaption></figcaption></figure>

* Click on "Security credentials" tab.

<figure><img src="/files/NQd9FeMptgF5n6ij9W6p" alt=""><figcaption></figcaption></figure>

* Start Access Key Creation<br>

  * Locate "Access keys" section.<br>
  * Click "Create access key" button.

  <figure><img src="/files/kAIxss8KubM472qXGgJl" alt=""><figcaption></figcaption></figure>

* Configure Access Key Purpose<br>

  * Select "Command Line Interface (CLI)" use case.

  <figure><img src="/files/sUzqnd051yrv6Dc1hItr" alt=""><figcaption></figcaption></figure>

* Check the box "I understand the above recommendation and want to proceed to create an access key."

  <figure><img src="/files/nthDwcSErWphMqIK4sfZ" alt=""><figcaption><p><br></p></figcaption></figure>

* Create and Save Credentials<br>

  * Click on Create access key

  <figure><img src="/files/lmRhcAO2W7uWP6PnAPZN" alt=""><figcaption></figcaption></figure>

### 4. Securely Save AWS Access Keys

After creating the access key, you have two options to save the credentials:

1. Download .csv File Method<br>

   * Click "Download .csv file" button<br>
   * File will contain:
     * Access Key ID
     * Secret Access Key<br>
   * Store this file securely

2. Manual Copy Method<br>
   * Access Key ID:&#x20;
     * Copy directly from interface by clicking on the icon 📑 next to Access Key<br>
   * Secret Access Key:
     * Click "Show" to reveal
     * Copy directly from interface by clicking on the icon 📑 <br>
   * Record both values securely.

<figure><img src="/files/KJNjmUjTo5bPWycPqnQz" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
**Important Security Notes:**

* Never store access keys in code repositories
* Keep credentials secure and private
* Recommended to use short-term credentials where possible
* Maximum of two active access keys allowed per user
  {% endhint %}

***

### About LightBeam <a href="#heading-h.oufesc4e3cn7" id="heading-h.oufesc4e3cn7"></a>

LightBeam automates Privacy, Security, and AI Governance, so businesses can accelerate their growth in new markets. Leveraging generative AI, LightBeam has rapidly gained customers’ trust by pioneering a unique *privacy-centric* and *automation-first* approach to security. Unlike siloed solutions, LightBeam ties together sensitive data cataloging, control, and compliance across structured and unstructured data applications providing 360-visibility, redaction, self-service DSRs, and automated ROPA reporting ensuring ultimate protection against ransomware and accidental exposures while meeting data privacy obligations efficiently. \
\
LightBeam is on a mission to create a secure privacy-first world helping customers automate compliance against a patchwork of existing and emerging regulations.

For any questions or suggestions, please get in touch with us at: <support@lightbeam.ai>.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.lightbeam.ai/lxqobxw6ak7CTnsQjikH/core-features/spectra-ai/data-sources/cloud-platforms/aws-auto-discovery.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.