Improve workload sustainability with services and features from re:Invent 2021
AWS Architecture BlogImprove workload sustainability with services and features from re:Invent 2021by Ernesto Bethencourt Plaz | on01 APR 2022| inAWS re:Invent,Sustainability|Permalink|ShareAt our recent annualAWS re:Invent2021 conference, we had important announcements regarding sustainability, including the newSustainability PillarforAWS Well-Architected Frameworkand theAWS Customer Carbon FootprintTool.In this blog post, I highlight services and features from these announcements to help you design and optimize your AWS workloads from a sustainability perspective.Architecting for sustainability basicsEnvironmental sustainabilityis a shared responsibilitybetween customers and AWS. We maintain sustainabilityofthe cloud by delivering efficient, shared infrastructure. As a customer, you maintain sustainabilityinthe cloud. This means you optimize your workload to efficiently use resources.Figure 1. Shared responsibility model for sustainabilityThe Sustainability Pillar of the Well-Architected Framework provides the followingdesign principlesthat you can use to optimize your workload for sustainability in the cloud:Understand your impact.Establish sustainability goals.Maximize utilization.Anticipate and adopt new, more efficient hardware and software offerings.Use managed services.Reduce the downstream impact of your cloud workloads.In the next sections, I share new services and service features that were announced at re:Invent 2021 that are related to these design principles and provide recommendations on how they can help you design and operate your workloads more sustainably.Understand your impactMeasure the environmental impact of your workloads and propose optimizations to meet your sustainability goals:Understand your workload’s environmental impact with newAWS Compute Optimizerresource efficiency metricsandenhanced metrics.Optimize scaling decisions and reduce idle resources associated with domain controllers withAWS Directory Service for Microsoft Active DirectoryandinformationinAmazon CloudWatch.Use theSQL-based metrics query engineto simplify gathering metrics to find optimization opportunities with CloudWatch.Maximize utilizationReduce the total energy required to power your workloads by right-sizing your workloads to ensure high utilization and eliminating or minimizing idle resources.Optimize your storageUse new storage tiers inAmazon Elastic Block Store (Amazon EBS) snapshots,Amazon Timestream,AWS IoT SiteWise,Amazon DynamoDB, andAmazon ElastiCache for Redistomove data to “colder” tiers.Support new storage access patterns with the newAmazon Simple Storage Service (Amazon S3) Glacier Instant RetrievalandAmazon S3 Intelligent-Tiering Archive Instant AccessDelete data that you don’t need byusing lifecycle policieswith fine-grained controls that usenew actions and filters.Usestorage optimizationforAWS Lake Formationto reduce and optimize your storage with its garbage collection and data compaction features.Optimize data movementMoving data across networks can contribute to energy consumption and your overall costs. Optimizing networks and data movement can save energy and costs. Optimize the way you access andmove data across networksby:UseAWS Direct Connect SiteLinkto connect an on-premises network directly through the AWS global network. This helps your data travel more efficiently (across the shortest path) rather than using public network.Migrate tape backups to the cloud withAWS Snow Family offline tape data migration. This helps you eliminate physical tapes and store your virtual tapes in the cloud with cold storage.Automakers that build connected cars generate loads of data. UseAWS IoT FleetWiseto reduce the amount of unnecessary data transferred to the cloud.Optimize your processingMinimize the resources you use and increase the utilization of resources you use to run your workloads:Optimize the resources you use for your machine learning (ML) models withAmazon SageMakerInference Recommender.ReduceAWS Lambdafunction executions with thenew event filteringforAmazon Simple Queue Service (Amazon SQS), DynamoDB, andAmazon Kinesis.Minimize the amount of data that needs to be accessed by consumer clusters and provision flexible compute resources inAmazon Redshiftwithnew performance enhancements.Use the open-source Kubernetes cluster auto scaling project,AWS Karpenter, to automatically adjust cluster capacity and meet actual demand.Anticipate and adopt new, more efficient hardware and software offeringsRapid adoption of new, more efficient technologies helps you reduce the impact of your workloads.Adopt more efficient instancesDeploy your workloads on newer, price-performant hardware with the latestAmazon Elastic Compute Cloud (Amazon EC2)instance families from partner-based chips like theIntel-based R6i,AMD EPYC-based M6a, andMac-based M1Use new AWS-built chip instances based inAWS Graviton2likeG5g,Im4gn and Is4genfor more energy-efficient instances.Deploy the latest generation of custom-designed AWS Graviton3-based instances:C7g. These instances offer the same performance and use 60% less energy than comparable EC2 instances.Gain efficiency with theAWS TrainiumML chipTrn1.Use managed services with AWS Graviton2-based chips likeAWS Fargate for Amazon Elastic Container Service (Amazon ECS),Amazon Neptune,ElastiCache, andAWS Elastic Beanstalk, which are more energy efficient than comparable EC2 instances.Right-size the on-premises AWS workloads that you run withsmaller AWS Outposts form factors.For workloads that need bare metal instances, use more efficient hardware with thenew Amazon EC2 bare metal instances.Use more efficient software offeringsUse thenext generation of Amazon FSx for Lustreto reduce storage space consumption and make use of more energy-efficient underlying hardware.Minimize data movement withpull-through cache repositoriesinAmazon Elastic Container Registry (Amazon ECR).Accelerate deep learning model training by using GPU instances more efficiently withAmazon SageMaker Training Compiler.Improve query performance in Amazon Redshift withAutomated Materialized Views.Reduce self-managed resources with intra-Region peering forAWS Transit Gateway.Build event-driven applications based on changes to the data you store in Amazon S3 to be optimized, flexible, and scalable with AWS serverless offerings and use advanced filtering and routing capabilities to send events to 18 targets by integratingAmazon S3 Event Notifications and Amazon EventBridge.Usenew Amazon S3 Event Notificationsto build event-driven applications and optimize resource efficiency of your workloads.Simplify operations and retire infrastructure you use to provide remote access to Windows instances withone-click login with AWS Single Sign-OnorAWS System Manager Fleet Manager console-based access.Decrease SMS-related usage when you test your campaigns inAmazon Pinpointwith theSMS simulator.Use managed servicesWe launched several managed services thatshift the responsibility of sustainabilityoptimization to AWS:Analytics:Amazon RedShift Serverless,Amazon Managed Streaming for Apache Kafka (Amazon MSK) Serverless,Amazon EMR Serverless, andKinesis Data Streams On-Demand. AWS managed services are scalable, have high average utilization of resources, and are optimized for sustainability.ML:Amazon SageMaker Serverless Inferenceis ideal for applications with intermittent or unpredictable traffic.Storage:Amazon FSx for OpenZFSis a fully managed file system that replaces ZFS and other on-premises Linux-based file servers.End-user compute:Amazon Workspaces WebandAmazon Appstream2 Elastic (serverless) fleetshelp reduce idle resources.Managed backup options:AWS BackupforAmazon S3andDynamoDBautomate backup retention and deletion.Build and manage your wide area network (WAN)withAWS Cloud WAN.Migrate and modernize your workloadwithAWS Mainframe Modernization, which helps move on-premises mainframe workloads to a managed and highly available runtime environment on AWS.Create digital twins of real-world systemswithAWS IoT TwinMaker.Publish and subscribe to third-party APIsusing a managed service withAWS Data Exchange for APIs.Reduce the downstream impact of your cloud workloadsUse theAWS Rust SDKfor the nativeenergy efficiency gains of the Rustprogramming language.UseAmazon CloudWatch RUMto understand the performance of your application and use that information to make it more efficient.Review your carbon emissions with the new AWS Customer Carbon Footprint Tool. This helps you define your sustainability key performance indicators (KPIs), optimize your workloads for sustainability, and improve your KPIs.ConclusionHaving a well-architected, sustainable workload is a continuous process. This blog post brings you AWS announcements from the sustainability perspective. I encourage you to review your workload and find out which of these announcements can be adopted in your workload.Ready to get started?I encourage you to check out on ourWhat’s New blogfor announcements and check them under a sustainability point of view, to identify if they help you improve and meet your sustainability goals.Looking for more architecture content?AWS Architecture Centerprovides reference architecture diagrams, vetted architecture solutions,Well-Architectedbest practices, patterns, icons, and more!Related informationWatch theAWS re:Invent 2021 Breakout Sessions– SustainabilityOther blog posts onarchitecting for sustainabilityAWS Architecture Monthly magazine, Sustainability issue, August 2021TAGS:re:Invent 2021,SustainabilityErnesto Bethencourt PlazErnesto is a Solutions Architect in Iberia, working with Public Sector customers, helping them take advantage of the cloud to deliver modern services to citizens and internal users. Previous to joining AWS, he worked as a Product Owner for a security solution in the Banking Sector and before that, a diverse IT roles experience in finance and consulting firms.
At our recent annual AWS re:Invent 2021 conference, we had important announcements regarding sustainability, including the new Sustainability Pillar for AWS Well-Architected Framework and the AWS Customer Carbon Footprint Tool. In this blog post, I highlight services and features from these announcements to help you design and optimize your AWS workloads from a sustainability perspective. […]
Ernesto Bethencourt Plaz
AWS re:Invent Sustainability