Email delta cost usage report in a multi-account organization using AWS Lambda
AWS Architecture BlogEmail delta cost usage report in a multi-account organization using AWS Lambdaby Ashutosh Dubey, Tom Kuipers, and Venkatesh Muthusami | on05 DEC 2022| inAmazon CloudWatch,Amazon Simple Email Service (SES),AWS Cloud Financial Management,AWS Cost Explorer,AWS Lambda,Customer Solutions,Messaging|Permalink|ShareAWS Organizationsgives customers the ability to consolidate their billing across accounts. This reduces billing complexity and centralizes cost reporting to a single account. These reports and cost information are available only to users with billing access to the primary AWS account.In many cases, there are members of senior leadership or finance decision makers who don’t have access to AWS accounts, and therefore depend on individuals or additional custom processes to share billing information. This task becomes specifically complicated when there is a complex account organization structure in place.In such cases, you can email cost reports periodically and automatically to these groups or individuals usingAWS Lambda. In this blog post, you’ll learn how to send automated emails for AWS billing usage and consumption drifts from previous days.Solution architectureFigure 1. Account structure and architecture diagramAWS provides theCost Explorer APIto enable you to programmatically query data for cost and usage of AWS services. This solution uses a Lambda function to query aggregated data from the API, format that data and send it to a defined list of recipients.Amazon EventBridge(Amazon CloudWatch Events) is configured to cue the Lambda function at a specific time.The function uses theAWS Cost ExplorerAPI to fetch the cost details for each account.The Lambda function calculates the change in cost over time and formats the information to be sent in an email.The formatted information is passed toAmazon Simple Email Service(Amazon SES).The report is emailed to the recipients configured in the environment variables of the function.PrerequisitesFor this walkthrough, you should have the following prerequisites:AnAWS accountwithCost Explorer enabled.AWSuser or rolewith permissions to deploy theAWS CloudFormationtemplate.Valid email IDs to receive email notifications.WalkthroughDownload the AWS CloudFormation template from this link:AWS CloudFormation templateOnce downloaded, open the template in your favorite text editorUpdate account-specific variables in the template. You need to update thetuple, dictionary, display list, and display list monthly sections of the script for all the accounts which you want to appear in the daily report email. Refer to Figure 2 for an example of some dummy account IDs and email IDs.Figure 2. Account IDs in AWS LambdaOptionally, locate “def send_report_email” in the template. The subject variable controls the subject line of the email. This can be modified to something meaningful to the recipients.After these changes are made according to your requirements, you can deploy the CloudFormation template:Log in to theCloud Formation console.ChooseCreate Stack. From the dropdown, chooseWith new resources (standard).On the next screen underSpecify Template, chooseUpload a template file.ClickChoose file. Choose the local template you modified earlier, then chooseNext.Fill out the parameter fields with valid email address. ForSchduleExpression, use a validCron expressionfor when you would like the report sent. ChooseNext.Here is an example for a cron schedule:18 11 * * ? *(This example cron expression sets the schedule to send every day at 11:18 UTC time.)This creates the Lambda function and neededAWS Identity and Access Management(AWS IAM) roles.You will now need to make a few modifications to the created resources.Log in to theIAM console.ChooseRoles.Locate the role created by the CloudFormation template called “daily-services-usage-lambdarole”Under the Permissions tab, choose Add Permissions. From the dropdown., chooseAttach Policy.In the search bar, search for “Billing”.Select the check box next to the AWS Managed Billing Policy and then chooseAttach Policy.Log in to theAWS Lambda console.Choose theDailyServicesUsagefunction.Choose theConfigurationtab.In the options that appear, chooseGeneral Configuration.Choose theEditbutton.Change the timeout option to 10 seconds, because the default of three seconds may not be enough time to run the function to retrieve the cost details from multiple accounts.ChooseSave.Still under theGeneral Configurationtab, choose thePermissionsoption and validate the execution role.The edited IAM execution role should display theResourcedetails for which the access has been gained. Figure 3 shows that the allow actions toaws-portalforBilling,Usage,PaymentMethods, and ViewBilling are enabled. If the Resource summary does not show these permissions, the IAM role is likely not correct. Go back to the IAM console and confirm that you updated the correct role with billing access.Figure 3. Lambda role permissionsOptionally, in the left navigation pane, chooseEnvironment variables. Here you will see the email recipients you configured in the Cloud Formation template. If changes are needed to the list in the future, you can add or remove recipients by editing the environment variables. You can skip this step if you’re satisfied with the parameters you specified earlier.Next, you will create a few Amazon SES identities for the email addresses that were provided as environment variables for the sender and recipients:Log in to theSES console.UnderConfiguration, chooseVerified Identities.ChooseCreate Identity.Choose the identity typeEmail Address, fill out theEmail addressfield with the sender email, and chooseCreate Identify.Repeat this step for all receiver emails.The email IDs included will receive an email for the confirmation. Once confirmed, the status shows as verified in theVerified Identitiestab of the SES console. The verified email IDs will start receiving the email with the cost reports.Amazon EventBridge (CloudWatch) event configurationTo configure events:Go to theAmazon EventBridge console.ChooseCreate rule.Fill out the rule details with meaningful descriptions.UnderRule Type, chooseSchedule.Schedule the cron pattern from when you would like the report to run.Figure 4 shows that the highlighted rule is configured to run the Lambda function every 24 hours.Figure 4. EventBridge ruleAn example AWS Daily Cost Report emailFrom: xxx@example.com (the email ID mentioned as “sender”)Sent: Tuesday, April 12, 2022 1:43 PMTo: yyy@example.com (the email ID mentioned as “receiver”)Subject: AWS Daily Cost Report for Selected Accounts (the subject of email as set in the Lambda function)Figure 5 shows the first part of the cost report. It provides the cost summary and delta of the cost variance percentage compare to the previous day. You can also see the trend based on the last seven days from the same table. This helps in understanding a pattern around cost and usage.This summary is broken down per account, and then totaled, in order to help you understand the accounts contributing to the cost changes. The daily change percentages are also color coded to highlight significant variations.Figure 5. AWS Daily Cost Report email body part 1The second part of the report in the email provides the service-related cost breakup for each account configured in the Account dictionary section of the function. This is a further drilldown report; you will get these for all configured accounts.Figure 6. AWS Daily Cost Report email body part 2CleanupDelete the Amazon CloudFormation stack.Delete the identities on Amazon SES.Delete the Amazon EventBridge (CloudWatch) event rule.ConclusionThe blog demonstrates how you can automatically and seamlessly share your AWS accounts’ billing and change information with your leadership and finance teams daily (or on any schedule you choose). While the solution was designed for accounts that are part of an organization in the service AWS organizations, it could also be deployed in a standalone account without making any changes. This allows information sharing without the need to provide account access to the recipients, and avoids any dependency on other manual processes. As a next step, you can also store these reports inAmazon Simple Storage Service(Amazon S3), generate a historical trend summary for consumption, and continue making informed decisions.Additional readingAWS Cloud Financial Management 2022 Q1 RecapGet AWS Cost Anomaly Detection alert notifications in Slack through AWS ChatbotAshutosh DubeyAshutosh is a Global Technical leader and Solutions Architect at Amazon Web Services based out of New Jersey, USA. He has extensive experience specializing in the Data, Analytics, and Machine Learning field, and has helped Fortune 500 companies in their cloud journey to AWS.Tom KuipersTom Kuipers is a Partner Solutions Architect at AWS. Based out of Phoenix, Arizona, He has spent 14+ years working in everything from large scale enterprise environments to fast paced startups. He has expertise in Networking/Security and Cloud Operations.Venkatesh MuthusamiVenkatesh is a Principal Consultant with Infosys, assisting customers with AWS platform builds, application migration, and modernization to the cloud. He is a member of the Infosys cloud practice team and leads various AWS initiatives. Venkatesh has a keen interest in industry and cloud trends.
AWS Organizations gives customers the ability to consolidate their billing across accounts. This reduces billing complexity and centralizes cost reporting to a single account. These reports and cost information are available only to users with billing access to the primary AWS account. In many cases, there are members of senior leadership or finance decision makers […]
Ashutosh Dubey
Amazon CloudWatch Amazon Simple Email Service (SES) AWS Cloud Financial Management AWS Cost Explorer AWS Lambda Customer Solutions Messaging