MongoDB Atlas
MongoDB Atlas integration using S3
You can configure your M10+ Atlas clusters to export system logs every minute to an AWS (Amazon Web Services) S3 (Simple Storage Service) bucket.
Required Access
To export logs to an AWS (Amazon Web Services) S3 (Simple Storage Service) bucket, you must have Project Owner or Organization Owner access to Atlas.
Considerations
Each Atlas host typically produces 1 GB of logs per day. Exporting logs incurs a data transfer cost. The exact data transfer cost varies depending on the destination, region, and cloud provider.
Network issues or retries can cause duplicate log entries in your AWS (Amazon Web Services) S3 (Simple Storage Service) bucket.
The AWS (Amazon Web Services) IAM (Identity and Access Management) role and the S3 (Simple Storage Service) bucket must belong to the same AWS (Amazon Web Services) account.
Prerequisites
You'll need:
An AWS (Amazon Web Services) IAM (Identity and Access Management) role with sts:AssumeRole that grants Atlas access to your AWS (Amazon Web Services) resource with a maximum session duration set to 12 hours.
An existing AWS (Amazon Web Services) S3 (Simple Storage Service) bucket.
An M10+ Atlas cluster running MongoDB 7.0 or later.
Procedure
To export logs to an AWS (Amazon Web Services) S3 (Simple Storage Service) bucket, complete the following steps.
In Atlas, go to the Project Settings page.
If it's not already displayed, select the organization that contains your desired project from the Organizations menu in the navigation bar.
If it's not already displayed, select your desired project from the Projects menu in the navigation bar.
In the sidebar, click the icon next to Project Overview.
The Project Settings page displays.
In Atlas, go to the Project Integrations page.
Click the Integrations tab.
The Project Integrations page displays.
Click Configure on the AWS S3 Log Monitoring card.
Grant Atlas access to your AWS (Amazon Web Services) account.
From the Authorize an AWS IAM Role dropdown, select your ARN (Amazon Resource Name). To add an ARN (Amazon Resource Name), see Set Up Unified AWS Access.
Click Next.
Connect Atlas to your AWS (Amazon Web Services) S3 (Simple Storage Service) bucket.
In the Bucket Name field, enter the name of your S3 (Simple Storage Service) bucket as it appears in your AWS (Amazon Web Services) account.
In the Prefix field, enter a directory name to organize the contents of your S3 (Simple Storage Service) bucket. For example, entering logs/ creates a logs directory in your S3 (Simple Storage Service) bucket to store the exported logs.
Under Log Type, select the below types of logs.
MongoDB Audit Logs
MongoDB Router Audit Logs
Click Next.
Assign an access policy to your AWS (Amazon Web Services) IAM (Identity and Access Management) role.
Click to copy the access policy generated by Atlas and save it locally with the file name: AtlasS3LogExportPolicy.
Click to copy the CLI command generated by Atlas, then run the command in your terminal to attach the access policy to your AWS (Amazon Web Services) IAM (Identity and Access Management) role.Click Validate to confirm your configuration and credentials are correct before enabling the export.
Click Finish.
Create an sqs queue for the S3 bucket and share access key , secret key , queue url for the same with necessary permissions.
You can refer to mentioned cloudtrail guide for sqs creation link
Last updated