AWS EKS_Log Integration Guide

AWS EKS log integration using CloudWatch & SQS queues, implementation guide

AWS EKS Control Plane → CloudWatch → Firehose → S3 → SQS

1. Introduction

This document describes the end-to-end procedure for integrating Amazon Elastic Kubernetes Service (EKS) control plane logs with Amazon CloudWatch Logs, Amazon Data Firehose, Amazon S3, and Amazon SQS, enabling SIEM onboarding and long-term log archival for enterprise cloud operations teams.

1.1 Component Overview

Component	Purpose
EKS Control Plane Logging	Exports Kubernetes API server, audit, authenticator, controller manager, and scheduler logs.
Amazon CloudWatch Logs	Receives and stores EKS control plane log streams in a dedicated log group per cluster.
CloudWatch Subscription Filter	Forwards log events from CloudWatch to an Amazon Data Firehose delivery stream in near-real time.
Amazon Data Firehose	Buffers and delivers log data to an S3 bucket with configurable compression, prefix, and error handling.
Amazon S3 Bucket	Provides durable, cost-effective archival storage for EKS log files organized by date and hour.
Amazon SQS Queue	Receives S3 object-creation event notifications, enabling downstream SIEM or collector workflows to process new log files.

2. Architecture Overview

The integration follows two distinct flows that together enable log delivery and downstream ingestion:

2.1 Primary Log Delivery Flow

EKS control plane logs are streamed from the Kubernetes control plane to CloudWatch Logs. A CloudWatch Logs Subscription Filter forwards log events in near-real time to Amazon Data Firehose, which buffers and delivers the data to an S3 bucket in compressed, partitioned files.

EKS Control Plane Logs

↓

Amazon CloudWatch Logs (/aws/eks//cluster)

↓

CloudWatch Subscription Filter

↓

Amazon Data Firehose (Direct PUT)

↓

Amazon S3 Bucket (aws-eks-logs-dev)

awsekslogs/year=YYYY/month=MM/day=DD/hour=HH/

2.2 Event Notification Flow

When Firehose writes a new object to S3, the S3 event notification mechanism publishes an object-creation event to an SQS queue. A downstream SIEM platform or log collector polls this queue to discover and retrieve newly available log files.

S3 Object Created Event

↓

S3 Event Notification

↓

Amazon SQS Queue (aws-eks-logs-queue)

↓

SIEM / Log Collector (polls SQS for new object keys)

3. Prerequisites

Ensure the following conditions are met before beginning the implementation:

• An active AWS account with permissions to manage EKS, CloudWatch, S3, SQS, IAM, and Amazon Data Firehose.

• An existing Amazon EKS cluster deployed in the target region.

• AWS Management Console access or sufficient AWS CLI credentials (v1.16.139 or later).

• The target S3 bucket region must match the EKS cluster region.

• IAM administrator access or the ability to create IAM roles and attach policies.

4. Environment Resources

The following AWS resources will be provisioned during this implementation. Resource names must remain consistent throughout all configuration steps.

Resource Type	Resource Name	Notes
Amazon S3 Bucket		Log archival storage; region must match EKS cluster.
Amazon SQS Queue	aws-eks-logs-queue	Standard queue; receives S3 object-creation events.
Amazon Data Firehose Stream		Name must match the ARN used in the IAM policy.
IAM Role (Firehose)		Assumed by Firehose and CloudWatch Logs.
IAM Inline Policy		Grants S3 write, CloudWatch Logs, and Firehose permissions.

**NOTE:**Substitute placeholder values (e.g., , , ) with your actual environment values before applying any policy or configuration shown in this guide.

5. Create S3 Bucket

The S3 bucket serves as the primary log archive destination for Amazon Data Firehose. The bucket must be created in the same region as the EKS cluster.

Procedure

1. Sign in to the AWS Management Console and navigate to the S3 service.

2. Verify that the selected region matches the region of your EKS cluster.

3. Click Create bucket.

4. Enter the bucket name:

5. Leave all other settings at their defaults unless your organization requires specific encryption, versioning, or access control policies.

6. Click Create bucket to confirm.

6. Create SQS Queue

The SQS queue receives S3 object-creation event notifications whenever Firehose writes a new log file. A downstream SIEM platform or log collector polls this queue to discover new log files.

Procedure

1. Navigate to Amazon SQS in the same region as the S3 bucket.

2. Click Create queue.

3. Select Standard queue type.

4. Enter the queue name: aws-eks-logs-queue

5. Configure the following queue attributes:

   1. Visibility timeout: 15 minutes

   2. Receive message wait time: 20 seconds

6. Under Access policy, click Advanced and replace the default policy with the following:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "S3SendMessage",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "sqs:SendMessage",
"Resource": "arn:aws:sqs:::aws-eks-logs-queue",
"Condition": {
"ArnLike": {
"aws:SourceArn": "arn:aws:s3:::"
}
}
}
]
}

**NOTE:**Replace and with your actual AWS region and 12-digit account ID before applying this policy.

1. Click Create queue.

7. Configure S3 Event Notifications

Configure the S3 bucket to publish object-creation events to the SQS queue. This enables the SIEM or log collector to be notified when Firehose delivers a new log file.

Procedure

1. Navigate to the S3 service and select the bucket created

2. Click the Properties tab.

3. Scroll to the Event notifications section and click Create event notification.

4. Configure the following settings:

• Event notification name: enter a descriptive name (e.g., eks-logs-put-event)

• Prefix: awsekslogs

• Event types: select PUT

• Destination: select SQS queue and choose aws-eks-logs-queue

1. Click Save changes.

Figure: S3 event notification targeting the aws-eks-logs-queue SQS queue

8. Create IAM Policy for Firehose

Create the permissions policy that will be attached to the Firehose IAM role. This policy grants Firehose write access to S3, allows CloudWatch Logs to put log events, and allows CloudWatch Logs to write records to the Firehose delivery stream.

Procedure

1. Navigate to the IAM service and select Policies.

2. Click Create policy, then select the JSON tab.

3. Paste the following policy. Replace placeholder values with your actual resource identifiers:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "S3WriteAccess",
"Effect": "Allow",
"Action": [
"s3:AbortMultipartUpload",
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:PutObject"
],
"Resource":
"arn:aws:s3:::",
"arn:aws:s3:::/*"
]
},
{
"Sid": "CloudWatchLogsAccess",
"Effect": "Allow",
"Action": [
"logs:PutLogEvents"
],
"Resource": "*"
},
{
"Sid": "AllowCloudWatchToWriteToFirehose",
"Effect": "Allow",
"Action": [
"firehose:PutRecord",
"firehose:PutRecordBatch"
],
"Resource": "arn:aws:firehose:::deliverystream/"
}
]
}

**NOTE:**Note the Firehose stream name used in the Resource ARN of the AllowCloudWatchToWriteToFirehose statement. This exact name must be used when creating the Firehose delivery stream in Section 10.

1. Enter a descriptive policy name (e.g., EKSLogsFirehoseInlinePolicy).

2. Click Create policy.

9. Create IAM Role for Firehose

Create an IAM role that can be assumed by both Amazon Data Firehose and Amazon CloudWatch Logs. This role will be attached to the Firehose delivery stream and referenced in the CloudWatch Logs subscription filter.

9.1 Create the Trust Policy (IAM Role)

1. Navigate to the IAM service and select Roles.

2. Click Create role.

3. Select Custom trust policy as the trusted entity type.

4. Paste the following trust policy:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "FirehoseAssumeRole",
"Effect": "Allow",
"Principal": {
"Service": "firehose.amazonaws.com"
},
"Action": "sts:AssumeRole"
},
{
"Sid": "CloudWatchLogsAssumeRole",
"Effect": "Allow",
"Principal": {
"Service": "logs.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}

1. Click Next.

2. On the Permissions screen, click Next (the inline policy will be attached in the following step).

3. Enter a descriptive role name (e.g., EKSLogsFirehoseRole).

4. Click Create role.

Figure: IAM role with trust relationships for firehose.amazonaws.com and logs.amazonaws.com

9.2 Attach IAM Policy to the Role

1. Open the newly created IAM role.

2. Click Add permissions → Attach policies.

3. Search for and select the policy created in Section 8 (e.g., EKSLogsFirehoseInlinePolicy).

4. Click Add permissions to confirm.

**NOTE:**Alternatively, you may attach the permissions as an inline policy. In the role, click Add permissions → Create inline policy, select JSON, and paste the same permissions policy from Section 8.

10. Create Amazon Data Firehose Stream

Create the Firehose delivery stream that receives log events from the CloudWatch Logs subscription filter and delivers them to the S3 bucket. The Firehose stream name must match the name referenced in the IAM policy ARN from Section 8.

Procedure

1. Navigate to Amazon Data Firehose in the same region as the EKS cluster.

2. Click Create Firehose stream.

3. Configure the source and destination:

• Source: Direct PUT

• Destination: Amazon S3

1. Enter the Firehose stream name. This name must match the name used in the IAM policy ARN from Section 8.

2. Under Destination settings, configure the S3 target:

• S3 bucket: aws-eks-logs-dev

• S3 bucket prefix: awsekslogs/year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/hour=!{timestamp:HH}/

• S3 bucket error output prefix: awsekslogs/!{firehose:error-output-type}/year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/

1. Configure buffer settings:

• Buffer size: 10 MB

• Buffer interval: 120 seconds

1. Configure output format:

• Compression for data records: Disabled

• File extension: .json.gz

1. Expand Advanced settings and configure:

• Amazon CloudWatch error logging: Enabled

• Service access — IAM role: select the IAM role created in Section 9 (e.g., EKSLogsFirehoseRole)

1. Click Create Firehose stream.

Figure: Firehose stream with Direct PUT source, S3 destination, and IAM role configured

11. Verify Firehose Configuration

After creating the Firehose stream, confirm that it is active and the configuration is correct before enabling EKS control plane logging.

1. Navigate to Amazon Data Firehose and locate the stream.

2. Confirm that the stream status shows Active

3. Verify that the destination S3 bucket, prefix, and IAM role are correctly reflected in the stream details.

4. Verify that CloudWatch error logging is enabled under the Monitoring section of the stream.

Figure: Firehose stream in Active state with correct S3 destination and IAM role

12. Enable EKS Control Plane Logging

By default, control plane logging for Amazon EKS clusters is not enabled and logs are not sent to Amazon CloudWatch Logs. Each log type must be explicitly enabled at the cluster level.

Supported Log Types

Log Type	Log Category Key	Description
API Server	api	Kubernetes API server requests and responses. Component: kube-apiserver.
Audit	audit	User, administrator, and system actions affecting the cluster. Required for compliance.
Authenticator	authenticator	IAM-to-Kubernetes RBAC authentication events. Unique to Amazon EKS.
Controller Manager	controllerManager	Kubernetes control loop activity. Component: kube-controller-manager.
Scheduler	scheduler	Pod scheduling decisions and placement events. Component: kube-scheduler.

Procedure

1. Open the Amazon EKS Console:

https://console.aws.amazon.com/eks/home#/clusters

1. Select the target EKS cluster from the cluster list.

2. Navigate to the Observability section on the cluster details page.

3. Click Update under the Control Plane Logging section.

4. For each required log type, set the status to Enabled. All log types are set to Disabled by default.

5. Click Update to apply the changes.

6. Verify that the update status changes to Successful.

Figure: EKS cluster Observability tab with control plane log types enabled

13. Verify CloudWatch Log Group Creation

After enabling EKS control plane logging, AWS automatically creates a CloudWatch Log Group for the cluster. Verify that this log group has been created before configuring the subscription filter.

1. Navigate to the Amazon CloudWatch Console.

2. Select Log groups from the left navigation panel.

3. Confirm that a log group exists in the following format:

/aws/eks//cluster

1. Expand the log group and confirm that individual log streams are present for each enabled log type.

14. Configure CloudWatch Logs Subscription Filter

Configure a CloudWatch Logs subscription filter on the EKS log group to forward all log events to the Firehose delivery stream in near-real time.

Procedure

1. Navigate to the Amazon CloudWatch Console and select Log groups.

2. Select the EKS log group: /aws/eks//cluster

3. Click Actions → Subscription filters → Create Amazon Data Firehose subscription filter.

4. Configure the subscription filter:

• Firehose stream: select the Firehose stream created in Section 10.

• IAM role: select the IAM role created in Section 9 (e.g., EKSLogsFirehoseRole)

• Log format: select JSON

• Subscription filter name: enter a descriptive name (e.g., eks-logs-to-firehose)

1. Click Start streaming to activate the subscription filter.

15. Validation Steps

Perform the following checks after completing the full configuration to confirm that the integration is operational.

15.1 Verify CloudWatch Log Ingestion

• Navigate to CloudWatch → Log groups → /aws/eks//cluster

• Confirm that log streams contain recent log events.

• Verify that events are timestamped within the last 10–15 minutes (allow time for logs to propagate after enabling logging).

15.2 Verify Firehose Delivery

• Navigate to Amazon Data Firehose and open the delivery stream.

• Select the Monitoring tab.

• Confirm that the Incoming records and Delivery to S3 metrics show recent activity.

• Verify that the Delivery to S3.Success metric is non-zero.

15.3 Verify S3 Object Delivery

• Navigate to the S3 bucket: aws-eks-logs-dev

• Browse to the prefix: awsekslogs/year=YYYY/month=MM/day=DD/hour=HH/

• Confirm that compressed log files with extension .json.gz are present.

15.4 Verify SQS Notifications

• Navigate to Amazon SQS and select the queue: aws-eks-logs-queue

• Click Send and receive messages.

• Click Poll for messages and confirm that messages are present, indicating that S3 events are being delivered to the queue.

• Each message body should contain JSON with the s3.object.key of the newly created log file.

16. Troubleshooting

The following table describes common issues, likely causes, and recommended remediation steps.

Symptom	Likely Cause	Remediation
No logs appear in CloudWatch Log Group	EKS control plane logging is not enabled, or the incorrect log types are selected.	Return to Section 12 and confirm all required log types are set to Enabled. Allow 5–10 minutes for the change to propagate.
Firehose delivery failures	IAM role lacks required S3 permissions, or the role is not attached to the Firehose stream.	Review the inline policy in Section 8. Confirm the IAM role ARN is correctly selected in Firehose Advanced settings. Check CloudWatch error logs for the Firehose stream.
S3 objects are not being created	Subscription filter is not configured or is referencing an incorrect Firehose stream name or IAM role.	Return to Section 14 and verify the subscription filter configuration. Confirm the Firehose stream name matches the ARN in the IAM policy.
SQS queue not receiving messages	S3 event notification is missing, or the SQS access policy does not permit s3.amazonaws.com to publish messages.	Return to Section 7 and verify the event notification configuration. Confirm the SQS access policy contains the correct S3 bucket ARN as the aws:SourceArn condition.
IAM permission errors in CloudWatch or Firehose	The IAM role trust policy does not include logs.amazonaws.com, or the permissions policy is missing required actions.	Verify the trust policy in Section 9.1 includes both firehose.amazonaws.com and logs.amazonaws.com as trusted principals. Review the permissions policy in Section 8.
Firehose stream in error state	CloudWatch error logging will surface the root cause.	Navigate to CloudWatch → Log groups → /aws/kinesisfirehose/ to review Firehose error logs. Common causes include S3 access denied and invalid bucket prefix format.

End of Document — AWS EKS Log Integration Using CloudWatch & SQS Queues