# AWS *** ## Setup Overview * There are 2 scenarios involved when deploying the LightBeam application: 1. EKS cluster is already present on the customer account. 2. EKS cluster is to be deployed by Lightbeam. {% hint style="info" %} **Note:** This pre-setup is required only for when EKS cluster is to be deployed by Lightbeam. {% endhint %} * In this case, the user needs to first create 2 IAM/SSO users with a required set of permissions. [If your organization’s security posture doesn’t allow for the creation of users, the EKS cluster deployment can be done using roles with the required set of permissions.](#user-content-fn-1)[^1] * The first user/role will have a larger set of permissions for deploying the EKS cluster, while the other user/role will be having EKS related permissions only to access the cluster post-deployment. {% hint style="info" %} **LightBeam Installation:** Before proceeding with the steps in this guide, ensure you have gone through the [LightBeam Installation process](https://docs.lightbeam.ai/lxqobxw6ak7CTnsQjikH/installer-guides/lightbeam-installer). Familiarize yourself with the installation prerequisites and steps to ensure a smooth setup. {% endhint %} *** ### 1.0 Deployment Architecture

Figure 1. EKS Cluster Network Diagram

Figure 2. LightBeam Microservices and Connection to Data Sources

*** ### 2.0 Create and Set up a Jump Box VM \ Create a **`t2.micro`** instance **(Ubuntu OS)** to use that as a jump box to configure and access the EKS cluster. {% hint style="info" %} **Note**: Verify that the deployed Jump Box VM has internet access (confirm that firewall rules allow outbound traffic). {% endhint %} #### Set up the Jump Box VM * **Install `unzip` package:** ``` sudo apt-get update sudo apt-get install unzip ``` * **Install `kubectl cli:`** {% code overflow="wrap" %} ``` curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" chmod +x kubectl mv kubectl version ``` {% endcode %} * **Install `helm cli:`** * Download the release from here: [`https://get.helm.sh/helm-v3.3.4-linux-amd64.tar.gz`](https://get.helm.sh/helm-v3.3.4-linux-amd64.tar.gz) * Run the following commands: * `tar -xvf` [`helm-v3.3.4-linux-amd64.tar.gz`](https://get.helm.sh/helm-v3.3.4-linux-amd64.tar.gz) * `mv linux-amd64/helm /usr/local/bin/helm` * `helm version`
* **Install `terraform cli:`** {% code overflow="wrap" %} ``` sudo apt-get update && sudo apt-get install -y gnupg software-properties-common curl curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add - sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main" sudo apt-get update && sudo apt-get install terraform=1.3.7 ``` {% endcode %} * **Install `aws cli:`** ``` curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip sudo ./aws/install aws --version ``` * **Setup `python`:** ``` sudo cp /usr/bin/python3 /usr/bin/python sudo apt install python3-pip ``` * **Install `docker`:** {% code overflow="wrap" %} ``` sudo apt-get -y remove docker docker-engine docker.io containerd runc sudo apt-get update -y sudo apt-get install -y apt-transport-https ca-certificates curl gnupg-agent software-properties-common curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" ``` {% endcode %} *** ### 3.0 Creating LightBeam EKS Deployment Users {% hint style="info" %} **`AWS_REGION:`** This refers to the AWS Region where the EKS cluster will be established. **`AWS_ACCOUNT_ID:`** This is the AWS Account ID where the EKS cluster will be created. **`EKS_CLUSTER_NAME:`** This is the chosen name for the EKS cluster. {% endhint %} #### Creating EKS admin user (Used only for EKS Cluster Deployment) 1. Log in to the AWS Console, and using the **Service** search, navigate to[ `IAM Management Console (amazon.com)`](https://console.aws.amazon.com/iamv2/home#/policies). On the left tab, click on **`Policies->Create policy.`** 2. Select the `JSON` tab, and paste the following `JSON`: ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "iam:ListRoleTags", "iam:RemoveRoleFromInstanceProfile", "iam:CreateRole", "iam:CreatePolicy", "iam:GetPolicy", "iam:CreateOpenIDConnectProvider", "iam:GetPolicyVersion", "iam:GetOpenIDConnectProvider", "iam:ListPolicyVersions", "iam:DeleteOpenIDConnectProvider", "iam:DeletePolicy", "iam:AttachRolePolicy", "iam:AddRoleToInstanceProfile", "iam:DetachRolePolicy", "iam:ListAttachedRolePolicies", "iam:ListRolePolicies", "iam:ListPolicies", "iam:GetRole", "iam:DeleteRole", "iam:CreateInstanceProfile", "iam:ListInstanceProfilesForRole", "iam:PassRole", "iam:DeleteInstanceProfile", "iam:GetInstanceProfile", "iam:ListRoles", "iam:ListInstanceProfiles", "iam:CreateServiceLinkedRole", "eks:UpdateClusterConfig", "eks:DeleteFargateProfile", "eks:DescribeFargateProfile", "eks:ListTagsForResource", "eks:DescribeNodegroup", "eks:ListNodegroups", "eks:DeleteCluster", "eks:CreateFargateProfile", "eks:DeleteNodegroup", "eks:UpdateNodegroupConfig", "eks:DescribeCluster", "eks:ListClusters", "eks:UpdateClusterVersion", "eks:UpdateNodegroupVersion", "eks:ListUpdates", "eks:CreateCluster", "eks:UntagResource", "eks:CreateNodegroup", "eks:ListFargateProfiles", "eks:DescribeUpdate", "eks:TagResource", "logs:ListLogDeliveries", "logs:CreateLogStream", "logs:TagLogGroup", "logs:CreateLogGroup", "logs:CreateLogDelivery", "logs:ListTagsLogGroup", "logs:DescribeLogStreams", "logs:DeleteLogStream", "logs:DeleteLogDelivery", "logs:DescribeLogGroups", "logs:UntagLogGroup", "logs:DeleteLogGroup", "logs:PutLogEvents", "logs:UpdateLogDelivery", "logs:PutRetentionPolicy", "logs:GetLogGroupFields", "sts:AssumeRole", "sts:GetCallerIdentity", "autoscaling:DescribeAutoScalingGroups", "ec2:RunInstances", "ec2:RevokeSecurityGroupIngress", "ec2:RevokeSecurityGroupEgress", "ec2:DescribeVpcs", "ec2:DescribeTags", "ec2:DescribeSubnets", "ec2:DescribeSecurityGroups", "ec2:DescribeRouteTables", "ec2:DescribeLaunchTemplateVersions", "ec2:DescribeLaunchTemplates", "ec2:DescribeKeyPairs", "ec2:DescribeInternetGateways", "ec2:DescribeImages", "ec2:DescribeAvailabilityZones", "ec2:DescribeAccountAttributes", "ec2:DeleteTags", "ec2:DeleteSecurityGroup", "ec2:DeleteKeyPair", "ec2:CreateTags", "ec2:CreateSecurityGroup", "ec2:CreateLaunchTemplateVersion", "ec2:CreateLaunchTemplate", "ec2:CreateKeyPair", "ec2:AuthorizeSecurityGroupIngress", "ec2:AuthorizeSecurityGroupEgress", "ec2:ReplaceRoute", "ec2:ModifyVpcAttribute", "ec2:ModifySubnetAttribute", "ec2:DisassociateRouteTable", "ec2:DetachInternetGateway", "ec2:DeleteVpc", "ec2:DeleteSubnet", "ec2:DeleteRouteTable", "ec2:DeleteRoute", "ec2:DeleteInternetGateway", "ec2:CreateVpc", "ec2:CreateSubnet", "ec2:CreateRouteTable", "ec2:CreateRoute", "ec2:CreateInternetGateway", "ec2:AttachInternetGateway", "ec2:AssociateRouteTable", "ec2:DescribeVpcAttribute", "ec2:AllocateAddress", "ec2:DescribeVpcClassicLink", "ec2:DescribeVpcClassicLinkDnsSupport", "ec2:DescribeAddresses", "ec2:ReleaseAddress", "ec2:CreateNatGateway", "ec2:DescribeNatGateways", "ec2:DeleteNatGateway", "eksCreateAddon", "eks:DeleteAddon", "eks:DescribeAddon", "eks:DescribeAddonConfiguration", "eks:DescribeAddonVersions", "eks:UpdateAddon" ], "Resource": "*" } ] } ``` 3. Give an appropriate name to the policy and click on **Save**.
4. Now, on the left panel of the[ `IAM Management Console (amazon.com)`](https://console.aws.amazon.com/iamv2/home#/policies), click **`Users`**`->`**`Add Users.`**
5. Give it an appropriate username, select only **`Programmatic Access`,** and click on **Next**.
6. Click on the **Existing Policies** tab and attach the policy created in **Step 3**. Also, attach the `VPCFullAccess managed policy` to the user and save it.
7. On the **Create** page, ensure that you copy the **Access Key Id** and the **Secret Access Key.**
Alternatively, instead of creating a user with programmatic access, you have the option to create a role: 1. Go to the [IAM Management Console (amazon.com)](https://console.aws.amazon.com/iamv2/home#/policies).
2. On the left tab, click on **`Roles -> Create role.`**
3. Select "**AWS account**" as the trusted entity and ensure the "`This account`" radio button is selected.
4. Optionally enable **`Multi-Factor Authentication (MFA)`** if required by your organization's security policy.
5. While adding permissions to this role, select the policy you created in the previous step (**Step 3**) and the "**VPCFullAccess**" managed policy.
6. To assume this role, use the AWS CLI command `aws sts assume-role`: {% code overflow="wrap" %} ``` aws sts assume-role --role-arn arn:aws:iam:::role/ --role-session-name ``` {% endcode %} This command will allow you to obtain temporary credentials for the role, including **Access Key ID, Secret Access Key**, and **Token.** Ensure you export these AWS credentials before proceeding with the installation step. [Another option instead of assuming the role is to attach the role to the jump box:](#user-content-fn-1)[^1] This means that the jump box, a separate EC2 instance, can directly use the IAM role to access the EKS cluster without assuming the role separately. To do this, instead of selecting "**AWS account**" in **Step 3**, select "**`AWS service`**", and select "**`EC2`**" under "**Common use cases**". \ \ Next, find the jump box instance in the EC2 console, select it, and click **`"Actions" -> "Security" -> "Modify IAM role".`**\ \ Find the IAM role in the drop-down menu, and click "**Update IAM role**".
#### Creating LightBeam EKS access user 1. Log in to the AWS Console, and using the **Service search**, navigate to[ `IAM Management Console (amazon.com)`](https://console.aws.amazon.com/iamv2/home#/policies). On the left tab, click **`Policies->Create policy`**.
2. Select the **`JSON`** tab, and paste the following JSON: ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "eks:ListTagsForResource", "eks:DescribeNodegroup", "eks:ListNodegroups", "eks:DescribeCluster", "eks:ListClusters", "eks:UpdateClusterVersion", "eks:UpdateNodegroupVersion", "eks:ListUpdates", "eks:UntagResource", "eks:ListFargateProfiles", "eks:DescribeUpdate", "eks:TagResource" ], "Resource": "arn:aws:eks:::cluster/" } ] } ``` 3. Give an appropriate name to the policy and click **Save**.
4. On the left panel of the **`IAM Management Console (amazon.com)`**, click on **`Users -> Add Users.`**
5. Give it an appropriate username, select only **`Programmatic Access`,** and click on **Next**.
6. Click on the **Existing Policies** tab and attach the policy created in **Step 3**.
7. On the **Create** page, ensure that you copy the **Access Key Id** and the **Secret Access Key.** [If your organization's security policies disallow creating IAM users, you can create an IAM role instead:](#user-content-fn-1)[^1] 1. Go to the [IAM Management Console (amazon.com)](https://console.aws.amazon.com/iamv2/home#/policies).
2. On the left tab, click on **`Roles -> Create role`**.
3. Select "`AWS account`" as the trusted entity and ensure the "`This account`" radio button is selected.
4. Optionally enable **Multi-Factor Authentication (MFA)** if required by your organization's security policy.
5. While adding permissions to this role, select the policy you created in the previous step (**Step 3**) which includes the necessary permissions for accessing the EKS cluster.
6. To assume this role, use the AWS CLI command `aws sts assume-role:` {% code overflow="wrap" %} ``` aws sts assume-role --role-arn arn:aws:iam:::role/ --role-session-name ``` {% endcode %} This command will allow you to obtain temporary credentials for the role, including A**ccess Key ID, Secret Access Key**, and **Token.** Ensure you export these AWS credentials before proceeding with the installation step. Another option instead of assuming the role is to attach the role to the jump box: This means that the jump box, which is a separate EC2 instance, can directly use the IAM role to access the EKS cluster without assuming the role separately. To do this, instead of selecting "**`AWS account`**" in **Step 3**, select "**`AWS service`**", and select "**`EC2`**" under "**Common use cases**". \ \ Next, find the jump box instance in the **EC2 console**, select it, and click "**`Actions" -> "Security" -> "Modify IAM role".`**\ \ Find the IAM role in the drop down menu, and click "**Update IAM role**". *** ### 4.0 EKS cluster creation #### A. AWS Cloud Credentials Setup **Option 1: Using EKS Admin User** 1. Connect to the jump box VM created above.
2. This can be done using \``aws configure`\` with EKS admin user credentials or setting E**nv Variables** as below with EKS admin user credentials: ``` export AWS_ACCESS_KEY_ID="" export AWS_SECRET_ACCESS_KEY="" export AWS_SESSION_TOKEN="" ``` \ **Option 2: Using EKS Admin Role** 1. Connect to the jump box VM created above.
2. Assume the EKS admin role by running the following AWS CLI command: {% code overflow="wrap" %} ``` aws sts assume-role --role-arn arn:aws:iam:::role/ --role-session-name ``` {% endcode %} 3. Export the temporary AWS credentials as environment variables in your terminal session by running the following commands: ``` export AWS_ACCESS_KEY_ID="" export AWS_SECRET_ACCESS_KEY="" export AWS_SESSION_TOKEN="" ``` #### B. Start Provisioning the EKS Cluster using Terraform script **Configuration:** * We need to provision **3 node EKS clusters** and each node instance type is **`m6i.2xlarge`** which has **8 CPU** and **32 GB** memory on each node.
* Copy **`eks-cluster.zip`** shared by LightBeam to the `Jump Host` and unzip the file.
* Run following commands: * `terraform init` * `terraform apply` {% hint style="info" %} **Note**: Verify that the deployed EKS cluster has internet access. {% endhint %} Once the EKS cluster configuration is complete we can set up **EKS Access Users** and continue with LightBeam application deployment. We need to update the EKS `auth-config,` so that the **EKS Access User** can then seamlessly access the cluster. * To update the **auth config**: ``` kubectl edit -n kube-system configmap/aws-auth ``` * The **auth-config** file opens up, where you need to add the following YAML content. (Ensure that you use the Lightbeam EKS access user ARN) ```yaml mapUsers: | - userarn: :user/LightbeamEKSUser> username: groups: - system:masters ``` * If you created a role instead of the user with the Lightbeam EKS Access policy, add the following to the mapRole section: {% code overflow="wrap" %} ```yaml - rolearn: arn:aws:iam:::role/ username: lb-eks-access-user groups: - system:masters ``` {% endcode %} *** ### 5.0 Access the LightBeam cluster using EKS Access user As we plan to deactivate the Deployment user credentials, it is important we verify accessing the cluster using the Lightbeam EKS user, so, export the second user credentials using the following commands executed on the Jump Box VM: ``` export AWS_ACCESS_KEY_ID="" export AWS_SECRET_ACCESS_KEY="" export AWS_SESSION_TOKEN="" ``` To access the cluster (control plane) : ``` aws eks --region update-kubeconfig --name ``` Verify EKS access: ``` kubectl get nodes ``` If you see the cluster node details, it ensures that the cluster is accessible. *** ### 6.0 Admin User Deletion/ Credential deactivation (optional) 1. Once the application is accessible, the `auth-config` is updated and the cluster access is verified, the EKS Deployment user can either be deleted or the credentials for the same can be deactivated.
2. To deactivate the credentials, navigate to the **EKS Deployment User**, and under **Security Credentials** Tab, under **Access Keys**, use the **`Make Inactive`** option to inactivate the key. *** ### 7.0 LightBeam Installation Follow the installation instructions in the [LightBeam Installer guide](/lxqobxw6ak7CTnsQjikH/getting-started/installer-guides/core/lightbeam-installation-and-upgrade.md#_w3t1uru6c4f9). Use the `--eks` flag in the installer command to specify the platform used is AWS EKS. *** ### 8.0 LightBeam Upgrade 1. The LightBeam team shares the latest release LightBeam chart which needs to be copied on the Jump Box.
2. Upgrade the LightBeam chart: {% hint style="info" %} **NOTE:** Dockerhub registry access is needed from all worker nodes of the Kubernetes cluster. {% endhint %} The `--spectra` flag specifies the spectra deployment. Use the `--privacy_ops` flag to specify the privacy ops deployment. {% code overflow="wrap" %} ``` export DOCKER_USERNAME="lbcustomers" DOCKER_REGISTRY_PASSWORD="" KBLD_REGISTRY_HOSTNAME="docker.io" KBLD_REGISTRY_USERNAME="lbcustomers" KBLD_REGISTRY_PASSWORD="" ./installer/lb-install.sh --upgrade --spectra --values charts/lightbeam/values.yaml --final_values charts/lightbeam/values.yaml ``` {% endcode %} One can verify the upgrade by running **`kubectl to get pods -n lightbeam`** and checking the status of all pods. As part of the upgrade, the following actions are taken: * Pausing datasource scanning and processing. * Updating the latest database schema. * Updating all LightBeam services to the respective release versions. As part of this step, all the latest release version Docker images from the Dockerhub registry are downloaded on respective worker nodes. * A post-upgrade job is executed which resumes scans for all data sources and starts processing, also verifies all services health. *** ## About LightBeam LightBeam automates Privacy, Security, and AI Governance, so businesses can accelerate their growth in new markets. Leveraging generative AI, LightBeam has rapidly gained customers’ trust by pioneering a unique *privacy-centric* and *automation-first* approach to security. Unlike siloed solutions, LightBeam ties together sensitive data cataloging, control, and compliance across structured and unstructured data applications providing 360-visibility, redaction, self-service DSRs, and automated ROPA reporting ensuring ultimate protection against ransomware and accidental exposures while meeting data privacy obligations efficiently. \ \ LightBeam is on a mission to create a secure privacy-first world helping customers automate compliance against a patchwork of existing and emerging regulations. For any questions or suggestions, please get in touch with us at: . [^1]: --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.lightbeam.ai/lxqobxw6ak7CTnsQjikH/getting-started/installer-guides/platform-specific/aws.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.