Kubeadapt

Aws

Learn how to integrate Kubeadapt with your AWS infrastructure for accurate cost tracking, spot instance pricing, and reserved instance visibility.

Overview

Kubeadapt provides comprehensive AWS integration capabilities:

  • Cloud Billing Integration - Connect to AWS Cost and Usage Reports via Athena for accurate cloud costs
  • Spot Instance Pricing - Real-time spot pricing from AWS Spot Instance Data Feed
  • Reserved Instances & Savings Plans - Automatic tracking of RI and SP utilization and coverage

Prerequisites

  • AWS account with appropriate permissions
  • Kubeadapt installed via Helm chart
  • kubectl access to your cluster
  • AWS CLI configured (for setup steps)

Part 1: Cloud Billing Integration

Step 1: Enable Cost Explorer

Cost Explorer API access is required for fetching cloud costs.

  1. Navigate to AWS Cost Management Console
  2. Enable Cost Explorer if not already enabled
  3. Wait 24 hours for initial data population

Step 2: Create Cost and Usage Report (CUR)

  1. Go to AWS Billing Console → Cost & Usage Reports

  2. Click "Create report"

  3. Configure report settings:

    • Report name: kubeadapt-cur
    • Time granularity: Hourly
    • Report versioning: Overwrite existing report
    • Enable report data integration: Amazon Athena
    • Compression: GZIP
    • Format: Parquet

  4. Select S3 bucket for report delivery:

    • Create new bucket or use existing: my-cur-bucket
    • Prefix: cur/
    • Region: Choose appropriate region

  5. Verify the report path format:

    text
    1s3://my-cur-bucket/cur/kubeadapt-cur/kubeadapt-cur/year=YYYY/month=MM/

Step 3: Set Up Athena Database

AWS automatically creates an Athena database when you enable CUR with Athena integration.

  1. Navigate to AWS Athena Console
  2. Verify database exists (usually named after your CUR report)
  3. Note the database name: kubeadapt_cur
  4. Note the table name: kubeadapt_cur

Alternatively, manually create the Athena integration:

bash
1aws s3 cp s3://my-cur-bucket/cur/kubeadapt-cur/crawler-cfn.yml . 2 3aws cloudformation create-stack --stack-name kubeadapt-cur-athena \ 4 --template-body file://crawler-cfn.yml \ 5 --parameters ParameterKey=ReportBucket,ParameterValue=my-cur-bucket

Step 4: Create IAM User for Athena Access

Create an IAM user with permissions to query Athena and access Cost Explorer:

json
1{ 2 "Version": "2012-10-17", 3 "Statement": [ 4 { 5 "Sid": "AthenaAccess", 6 "Effect": "Allow", 7 "Action": ["athena:*"], 8 "Resource": ["*"] 9 }, 10 { 11 "Sid": "ReadAccessToAthenaCurDataViaGlue", 12 "Effect": "Allow", 13 "Action": [ 14 "glue:GetDatabase*", 15 "glue:GetTable*", 16 "glue:GetPartition*", 17 "glue:GetUserDefinedFunction", 18 "glue:BatchGetPartition" 19 ], 20 "Resource": [ 21 "arn:aws:glue:*:*:catalog", 22 "arn:aws:glue:*:*:database/kubeadapt_cur", 23 "arn:aws:glue:*:*:table/kubeadapt_cur/*" 24 ] 25 }, 26 { 27 "Sid": "AthenaQueryResultsOutput", 28 "Effect": "Allow", 29 "Action": [ 30 "s3:GetBucketLocation", 31 "s3:GetObject", 32 "s3:ListBucket", 33 "s3:ListBucketMultipartUploads", 34 "s3:ListMultipartUploadParts", 35 "s3:AbortMultipartUpload", 36 "s3:CreateBucket", 37 "s3:PutObject" 38 ], 39 "Resource": ["arn:aws:s3:::my-cur-bucket*"] 40 }, 41 { 42 "Sid": "S3ReadAccessToAwsBillingData", 43 "Effect": "Allow", 44 "Action": ["s3:Get*", "s3:List*"], 45 "Resource": ["arn:aws:s3:::my-cur-bucket*"] 46 }, 47 { 48 "Sid": "CostExplorerAccess", 49 "Effect": "Allow", 50 "Action": [ 51 "ce:GetReservationUtilization", 52 "ce:GetSavingsPlansUtilization", 53 "ce:GetReservationCoverage", 54 "ce:GetSavingsPlansCoverage" 55 ], 56 "Resource": "*" 57 } 58 ] 59}

Create the user and save credentials:

bash
1aws iam create-user --user-name kubeadapt-athena 2 3aws iam put-user-policy --user-name kubeadapt-athena \ 4 --policy-name KubeadaptAthenaAccess \ 5 --policy-document file://athena-policy.json 6aws iam create-access-key --user-name kubeadapt-athena

Save the AccessKeyId and SecretAccessKey from the output.

Step 5: Configure Kubeadapt with Athena Integration

Create a cloud-integration.json file:

json
1{ 2 "aws": { 3 "athena": [ 4 { 5 "bucket": "s3://my-cur-bucket", 6 "database": "kubeadapt_cur", 7 "table": "kubeadapt_cur", 8 "region": "us-east-1", 9 "catalog": "AwsDataCatalog", 10 "workgroup": "Primary", 11 "account": "123456789012", 12 "authorizer": { 13 "authorizerType": "AWSAccessKey", 14 "id": "<ACCESS_KEY_ID>", 15 "secret": "<ACCESS_KEY_SECRET>" 16 } 17 } 18 ], 19 "spotDataBucket": "my-spot-data-bucket", 20 "spotDataRegion": "us-east-1", 21 "spotDataPrefix": "spot-data", 22 "projectID": "123456789012" 23 } 24}

Create Kubernetes secret:

bash
1kubectl create secret generic cloud-integration \ 2 --from-file=cloud-integration.json \ 3 --namespace kubeadapt

Update your Helm values to enable cloud cost integration:

yaml
1# values.yaml 2opencost: 3 opencost: 4 exporter: 5 cloudIntegrationSecret: "cloud-integration" 6 cloudCost: 7 enabled: true

Apply the configuration:

bash
1helm upgrade kubeadapt kubeadapt/kubeadapt \ 2 --namespace kubeadapt \ 3 -f values.yaml

Alternative: Using IAM Roles for Service Accounts (IRSA)

For EKS clusters, use IRSA instead of access keys for better security:

  1. Create IAM OIDC provider for your EKS cluster:
bash
1eksctl utils associate-iam-oidc-provider \ 2 --cluster=my-cluster \ 3 --approve
  1. Create IAM role with trust relationship:
json
1{ 2 "Version": "2012-10-17", 3 "Statement": [ 4 { 5 "Effect": "Allow", 6 "Principal": { 7 "Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/oidc.eks.REGION.amazonaws.com/id/OIDC_ID" 8 }, 9 "Action": "sts:AssumeRoleWithWebIdentity", 10 "Condition": { 11 "StringEquals": { 12 "oidc.eks.REGION.amazonaws.com/id/OIDC_ID:sub": "system:serviceaccount:kubeadapt:kubeadapt-cost-analyzer" 13 } 14 } 15 } 16 ] 17}

  1. Attach the Athena access policy to this role

  2. Update cloud-integration.json:

json
1{ 2 "aws": { 3 "athena": [ 4 { 5 "bucket": "s3://my-cur-bucket", 6 "database": "kubeadapt_cur", 7 "table": "kubeadapt_cur", 8 "region": "us-east-1", 9 "catalog": "AwsDataCatalog", 10 "workgroup": "Primary", 11 "account": "123456789012", 12 "authorizer": { 13 "authorizerType": "AWSServiceAccountKey" 14 } 15 } 16 ] 17 } 18}
  1. Annotate service account in your Helm values file:
yaml
1# values.yaml 2opencost: 3 serviceAccount: 4 annotations: 5 eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/KubeadaptAthenaRole

Part 2: Spot Instance Pricing

Spot instance prices change frequently. Kubeadapt retrieves real-time spot pricing from your AWS Spot Instance Data Feed stored in S3.

Important: Spot Data Feed is configured via customPricing in values.yaml. This is separate from cloud-integration.json which is only for Athena/CUR integration.

Step 1: Enable Spot Instance Data Feed

bash
1# Create S3 bucket for spot data 2aws s3 mb s3://my-spot-data-bucket --region us-east-1 3 4# Subscribe to Spot Instance Data Feed 5aws ec2 create-spot-datafeed-subscription \ 6 --bucket my-spot-data-bucket \ 7 --prefix spot-data

Or via AWS Console:

  1. Navigate to EC2 Console → Spot Requests → Data Feed
  2. Click "Subscribe to data feed"
  3. Configure:
    • S3 bucket: my-spot-data-bucket
    • Prefix: spot-data/ (optional)

AWS will start delivering hourly spot pricing data to this bucket within ~1 hour.

Step 2: Create IAM Policy for S3 Access

Create policy for reading spot data:

json
1{ 2 "Version": "2012-10-17", 3 "Statement": [ 4 { 5 "Sid": "SpotDataAccess", 6 "Effect": "Allow", 7 "Action": ["s3:ListBucket", "s3:GetObject"], 8 "Resource": ["arn:aws:s3:::my-spot-data-bucket", "arn:aws:s3:::my-spot-data-bucket/*"] 9 } 10 ] 11}

Step 3: Configure Spot Data Feed in Kubeadapt

Configure via customPricing in your values.yaml:

yaml
1# values.yaml 2opencost: 3 opencost: 4 customPricing: 5 enabled: true 6 provider: aws # CRITICAL: Must be "aws" to create aws.json 7 costModel: 8 awsSpotDataBucket: "my-spot-data-bucket" 9 awsSpotDataRegion: "us-east-1" 10 awsSpotDataPrefix: "spot-data" 11 projectID: "123456789012" # Your AWS Account ID 12 exporter: 13 aws: 14 access_key_id: "AKIAIOSFODNN7EXAMPLE" 15 secret_access_key: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

Apply with Helm:

bash
1helm upgrade kubeadapt kubeadapt/kubeadapt \ 2 --namespace kubeadapt \ 3 -f values.yaml

Important: The provider: aws setting is critical. It determines the ConfigMap filename:

  • provider: custom → creates default.json (OpenCost won't find spot config)
  • provider: aws → creates aws.json (OpenCost reads spot config correctly)

Note: Spot Data Feed configuration is separate from cloud-integration.json. Even if you have Athena/CUR configured via cloud-integration.json, you still need to configure Spot Data Feed via customPricing.

Verification

Check logs to verify spot pricing integration:

bash
1kubectl logs -n kubeadapt deployment/kubeadapt-cost-analyzer | grep -i spot

You should see logs indicating successful spot data retrieval.

Part 3: Reserved Instances & Savings Plans

Reserved Instance and Savings Plans data is automatically retrieved when you configure the Athena/CUR integration with Cost Explorer API access.

Prerequisites

Ensure your IAM user/role has these Cost Explorer permissions:

json
1{ 2 "Sid": "CostExplorerAccess", 3 "Effect": "Allow", 4 "Action": [ 5 "ce:GetReservationUtilization", 6 "ce:GetSavingsPlansUtilization", 7 "ce:GetReservationCoverage", 8 "ce:GetSavingsPlansCoverage" 9 ], 10 "Resource": "*" 11}

These permissions were included in the IAM policy from Part 1.

What Gets Tracked

With Cost Explorer API access enabled, Kubeadapt automatically tracks:

  • RI Utilization - How much of your purchased reserved capacity is being used
  • RI Coverage - Percentage of instance usage covered by RIs
  • Savings Plans Utilization - SP commitment utilization percentage
  • Savings Plans Coverage - Instance usage covered by Savings Plans
  • Net Savings - Actual savings compared to On-Demand pricing

Viewing RI/SP Data

RI and Savings Plans data appears automatically in:

  1. Dashboard - Cost breakdown shows On-Demand vs Reserved vs Spot
  2. Node View - Each node displays pricing type (On-Demand, Reserved, Spot)
  3. Cost Reports - Historical trends show RI/SP coverage over time
  4. Savings Recommendations - Suggestions for additional RI/SP purchases

No additional configuration required beyond the Athena/CUR setup.

Troubleshooting

Common Issues

Issue: "Failed to query Athena"

  • Verify IAM permissions include all required Athena and Glue actions
  • Check S3 bucket policy allows the IAM user to read CUR data
  • Ensure Athena database and table names match exactly

Issue: "Spot pricing data not found"

  • Verify Spot Instance Data Feed is enabled
  • Check S3 bucket name and prefix are correct
  • Data feed can take up to 1 hour to start delivering data
  • Ensure IAM permissions include S3 read access

Issue: "Reserved Instance data missing"

  • Verify Cost Explorer is enabled (requires 24 hours for initial data)
  • Check IAM permissions include ce:GetReservation* actions
  • RI data appears only for instances actually using reservations

Issue: "Cross-account billing not working"

  • For multi-account setups, CUR must be enabled in the payer account
  • IAM user needs cross-account access to the payer account's CUR bucket
  • Update cloud-integration.json with payer account credentials

Multi-Account Setup (AWS Organizations)

For organizations with multiple AWS accounts under AWS Organizations with consolidated billing, you need to configure Kubeadapt to access the CUR data stored in the management (payer) account.

Architecture Overview

text
1AWS Organization (Consolidated Billing) 2├── Management Account: 111111111111 (CUR stored here) 3│ └── KubeadaptRole (IAM role with Athena/S3 permissions) 4│ 5├── Member Account 1: 222222222222 (EKS Cluster 1) 6│ └── Kubeadapt → Assumes KubeadaptRole via masterPayerARN 7│ 8├── Member Account 2: 333333333333 (EKS Cluster 2) 9│ └── Kubeadapt → Assumes KubeadaptRole via masterPayerARN 10│ 11├── Member Account 3: 444444444444 (EKS Cluster 3) 12│ └── Kubeadapt → Assumes KubeadaptRole via masterPayerARN 13│ 14├── Member Account 4: 555555555555 (EKS Cluster 4) 15│ └── Kubeadapt → Assumes KubeadaptRole via masterPayerARN 16│ 17└── Member Account 5: 666666666666 (EKS Cluster 5) 18 └── Kubeadapt → Assumes KubeadaptRole via masterPayerARN

Step 1: Create IAM Role in Management Account

In your management (payer) account (111111111111), create an IAM role that member accounts can assume:

Trust Policy (KubeadaptRole-trust-policy.json):

json
1{ 2 "Version": "2012-10-17", 3 "Statement": [ 4 { 5 "Effect": "Allow", 6 "Principal": { 7 "AWS": [ 8 "arn:aws:iam::222222222222:root", 9 "arn:aws:iam::333333333333:root", 10 "arn:aws:iam::444444444444:root", 11 "arn:aws:iam::555555555555:root", 12 "arn:aws:iam::666666666666:root" 13 ] 14 }, 15 "Action": "sts:AssumeRole", 16 "Condition": { 17 "StringEquals": { 18 "sts:ExternalId": "kubeadapt-cross-account" 19 } 20 } 21 } 22 ] 23}

Create the role:

bash
1aws iam create-role \ 2 --role-name KubeadaptRole \ 3 --assume-role-policy-document file://KubeadaptRole-trust-policy.json

Permissions Policy (same as Part 1, Step 4):

bash
1aws iam put-role-policy \ 2 --role-name KubeadaptRole \ 3 --policy-name KubeadaptAthenaAccess \ 4 --policy-document file://athena-policy.json

Step 2: Configure Member Account IAM

In each member account, create an IAM user or role that can assume the management account's KubeadaptRole:

IAM User Permissions (in member account):

json
1{ 2 "Version": "2012-10-17", 3 "Statement": [ 4 { 5 "Effect": "Allow", 6 "Action": "sts:AssumeRole", 7 "Resource": "arn:aws:iam::111111111111:role/KubeadaptRole" 8 } 9 ] 10}

Step 3: Configure cloud-integration.json

In each member account's Kubeadapt installation, use this configuration:

json
1{ 2 "aws": { 3 "athena": [ 4 { 5 "bucket": "s3://organization-cur-bucket", 6 "database": "athenacurcfn_organization_cur", 7 "table": "organization_cur", 8 "region": "us-east-1", 9 "catalog": "AwsDataCatalog", 10 "workgroup": "Primary", 11 "account": "111111111111", 12 "masterPayerARN": "arn:aws:iam::111111111111:role/KubeadaptRole", 13 "authorizer": { 14 "authorizerType": "AWSAccessKey", 15 "id": "<MEMBER_ACCOUNT_ACCESS_KEY_ID>", 16 "secret": "<MEMBER_ACCOUNT_SECRET_ACCESS_KEY>" 17 } 18 } 19 ], 20 "spotDataBucket": "organization-spot-data-bucket", 21 "spotDataRegion": "us-east-1", 22 "spotDataPrefix": "spot-data", 23 "projectID": "222222222222" 24 } 25}

Key Fields Explained:

  • account: Management/Payer Account ID where CUR is stored (111111111111)
  • masterPayerARN: ARN of the role in management account that has Athena/S3 permissions
  • projectID: Member Account ID where this specific cluster is running (222222222222, 333333333333, etc.)
  • authorizer: Credentials from the member account that can assume the masterPayerARN role

Important: Each of the 5 clusters uses:

  • Same account (management account)
  • Same masterPayerARN (role in management account)
  • Different projectID (their own member account ID)
  • Different authorizer credentials (their own member account credentials)

Step 4: Apply Configuration to Each Cluster

For each member account cluster, create the secret and deploy:

bash
1# Create secret 2kubectl create secret generic cloud-integration \ 3 --from-file=cloud-integration.json \ 4 --namespace kubeadapt 5 6# Update Helm values 7helm upgrade kubeadapt kubeadapt/kubeadapt \ 8 --namespace kubeadapt \ 9 -f values.yaml

How It Works

  1. Kubeadapt in Member Account 222222222222:

    • Uses member account credentials (from authorizer)
    • Assumes KubeadaptRole in management account 111111111111
    • Queries CUR via Athena
    • Sees only costs for account 222222222222 (filtered by line_item_usage_account_id)

  2. Kubeadapt in Member Account 333333333333:

    • Uses member account credentials (from authorizer)
    • Assumes same KubeadaptRole in management account
    • Queries same CUR via Athena
    • Sees only costs for account 333333333333

  3. No Double-Counting:

    • Each Kubeadapt-Opencost instances filters CUR by its own projectID
    • AWS has already allocated RI/SP discounts in the CUR
    • Each account sees only its portion of shared RIs/SPs

Alternative: Using IRSA for Member Accounts

For better security, use IRSA in each member account:

Member Account Configuration:

json
1{ 2 "aws": { 3 "athena": [ 4 { 5 "bucket": "s3://organization-cur-bucket", 6 "database": "athenacurcfn_organization_cur", 7 "table": "organization_cur", 8 "region": "us-east-1", 9 "workgroup": "Primary", 10 "account": "111111111111", 11 "masterPayerARN": "arn:aws:iam::111111111111:role/KubeadaptRole", 12 "authorizer": { 13 "authorizerType": "AWSServiceAccountKey" 14 } 15 } 16 ], 17 "projectID": "222222222222" 18 } 19}

IRSA Role in Member Account needs permission to assume KubeadaptRole:

json
1{ 2 "Version": "2012-10-17", 3 "Statement": [ 4 { 5 "Effect": "Allow", 6 "Action": "sts:AssumeRole", 7 "Resource": "arn:aws:iam::111111111111:role/KubeadaptRole" 8 } 9 ] 10}

Annotate service account:

yaml
1opencost: 2 serviceAccount: 3 annotations: 4 eks.amazonaws.com/role-arn: arn:aws:iam::222222222222:role/MemberAccountKubeadaptRole

Validation

Test your configuration:

bash
1# Check if cloud costs are being retrieved 2kubectl logs -n kubeadapt deployment/kubeadapt-cost-analyzer | grep -i "cloud cost" 3 4# Verify secret is mounted correctly 5kubectl describe pod -n kubeadapt -l app=cost-analyzer | grep cloud-integration 6 7# Check Athena connectivity 8kubectl exec -n kubeadapt deployment/kubeadapt-cost-analyzer -- \ 9 curl -s http://localhost:9003/healthz

Support

For additional help:

Next Steps

Back to Docs
Edit this page