Optimizing your AWS Infrastructure for Sustainability, Part IV: Databases
AWS Architecture BlogOptimizing your AWS Infrastructure for Sustainability, Part IV: Databasesby Otis Antoniou, Ibtehaj Ahmed, Darren Ko, and Ceren Tahtasiz | on17 AUG 2022| inArchitecture,Database,Sustainability|Permalink|ShareInPart I: Compute,Part II: Storage, andPart III: Networkingof this series, we introduced strategies to optimize the compute, storage, and networking layers of your AWS architecture for sustainability.This post, Part IV, focuses on the database layer and proposes recommendations to optimize your databases’ utilization, performance, and queries. These recommendations are based on design principles ofAWS Well-Architected Sustainability Pillar.Optimizing the database layer of your AWS infrastructureFigure 1. AWS database servicesAs your application serves more customers, the volume of data stored within your databases will increase. Implementing the recommendations in the following sections will help you use databases resources more efficiently and save costs.Use managed databasesUsually, customers overestimate the capacity they need to absorb peak traffic, wasting resources and money on unused infrastructure. AWS fully managed database services provide continuous monitoring, which allows you to increase and decrease your database capacity as needed. Additionally, most AWS managed databases use a pay-as-you-go model based on the instance size and storage used.Managed services shift responsibility to AWS for maintaining high average utilization and sustainability optimization of the deployed hardware.Amazon Relational Database Service (Amazon RDS)reduces your individual contribution compared to maintaining your own databases onAmazon Elastic Compute Cloud (Amazon EC2). In a managed database, AWS continuously monitors your clusters to keep your workloads running with self-healing storage and automated scaling.AWS offers 15+ purpose-built enginesto support diverse data models. For example, if an Internet of Things (IoT) application needs to process large amounts of time series data,Amazon Timestreamis designed and optimized for this exact use case.Rightsize, reduce waste, and choose the right hardwareTo see metrics, thresholds, and actions you can take to identify underutilized instances and rightsizing opportunities,Optimizing costs in Amazon RDSprovides great guidance. The following table provides additional tools and metrics for you to find unused resources:ServiceMetricSourceAmazon RDSDatabaseConnectionsAmazon CloudWatchAmazon RDSIdle DB InstancesAWS Trusted AdvisorAmazon DynamoDBAccountProvisionedReadCapacityUtilization, AccountProvisionedWriteCapacityUtilization, ConsumedReadCapacityUnits, ConsumedWriteCapacityUnitsCloudWatchAmazon RedshiftUnderutilized Amazon Redshift ClustersAWS Trusted AdvisorAmazon DocumentDBDatabaseConnections, CPUUtilization, FreeableMemoryCloudWatchAmazon NeptuneCPUUtilization, VolumeWriteIOPs, MainRequestQueuePendingRequestsCloudWatchAmazon KeyspacesProvisionedReadCapacityUnits, ProvisionedWriteCapacityUnits, ConsumedReadCapacityUnits, ConsumedWriteCapacityUnitsCloudWatchThese tools will help you identify rightsizing opportunities. However,rightsizing databasescan affect your SLAs for query times, so consider this before making changes.We also suggest:Evaluating if your existing SLAs meet your business needs or if they could be relaxed as an acceptable trade-off to optimize your environment for sustainability.If any of your RDS instances only need to run during business hours, consider shutting them down outside business hours either manually or withInstance Scheduler.Consider using a more power-efficient processor likeAWS Graviton-based instances for your databases. Graviton2 delivers 2-3.5 times better CPU performance per watt than any other processor in AWS.Make sure to choose the rightRDS instance typefor the type of workload you have. For example,burstable performance instancescan deal with spikes that exceed the baseline without the need to overprovision capacity. In terms of storage, Amazon RDS providesthree storage typesthat differ in performance characteristics and price, so you can tailor the storage layer of your database according to your needs.Use serverless databasesProduction databases that experience intermittent, unpredictable, or spiky traffic may be underutilized. To improve efficiency and eliminate excess capacity, scale your infrastructure according to its load.AWS offers relational and non-relational serverless databases that shut off when not in use, quickly restart, and automatically scale database capacity based on your application’s needs. This reduces your environmental impact because capacity management is automatically optimized. By selecting the best purpose-built database for your workload, you’ll benefit from the scalability and fully-managed experience of serverless database services, as shown in the following table.Serverless Relational DatabasesServerless Non-relational DatabasesAmazon Aurora Serverlessfor an on-demand, autoscaling configurationAmazon DynamoDB(inOn-Demand mode) for a fully managed, serverless, key-value NoSQL databaseAmazon Redshift Serverlessruns and scales data warehouse capacity; you don’t need to set up and manage data warehouse infrastructureAmazon Timestreamfor a time series database service for IoT and operational applicationsAmazon Keyspacesfor a scalable, highly available, and managed Apache Cassandra–compatible database serviceAmazon Quantum Ledger Databasefor a fully managed ledger database that provides a transparent, immutable, and cryptographically verifiable transaction log ‎owned by a central trusted authorityUse automated database backups and remove redundant dataManual Amazon RDS backups, unlike automated backups, take a manual snapshot of your database and do not have a retention period set by default. This means that unless you delete a manual snapshot, it will not be removed automatically. Removing manual snapshots you don’t need will use fewer resources, which will reduce your costs. If you want manual snapshots of RDS, you can set an “expiration” withAWS Backup. To keep long-term snapshots of MariaDB, MySQL, and PostgreSQL data, we recommend exporting snapshot data toAmazon Simple Storage Service (Amazon S3). You can also export specific tables or databases. This way, you can move data to “colder” longer-term archival storage instead of keeping it within your database.Optimize long running queriesIdentify and optimize queries that are resource intensive because they can affect the overall performance of your application. By using thePerformance Insights dashboard, specifically the Top Dimensions table, which displays theTop SQL, waits, and hosts, you’ll be able to view and download SQL queries to diagnose and investigate further.Tuning Amazon RDS for MySQL with Performance Insightsand thisknowledge center articlewill help you optimize and tune queries in Amazon RDS for MySQL. TheOptimizing and tuning queries in Amazon RDS PostgreSQL based on native and external toolsandImprove query performance with parallel queries in Amazon RDS for PostgreSQL and Amazon Aurora PostgreSQL-Compatible Editionblog posts outline how to use native and external tools to optimize and tune Amazon RDS PostgreSQL queries, as well as improve query performance using the parallel query feature.Improve database performanceYou can improve your database performance by monitoring, identifying, and remediating anomalous performance issues. Instead of relying on a database administrator (DBA), AWS offers native tools to continuously monitor and analyze database telemetry, as shown in the following table.ServiceCloudWatch MetricSourceAmazon DynamoDBCPUUtilization, FreeStorageSpaceCloudWatchAmazon RedshiftCPUUtilization, PercentageDiskSpaceUsedCloudWatchAmazon AuroraCPUUtilization, FreeLocalStorageAmazon RDSDynamoDBAccountProvisionedReadCapacityUtilization, AccountProvisionedWriteCapacityUtilizationCloudWatchAmazon ElastiCacheCPUUtilizationCloudWatchCloudWatch displaysinstance-levelandaccount-level usagemetrics for Amazon RDS.Create CloudWatch alarmsto activate and notify you based on metric value thresholds you specify or when anomalous metric behavior is detected. EnableEnhanced Monitoringreal-time metrics for the operating system the DB instance runs on.Amazon RDS Performance Insightscollects performance metrics, such asdatabase load, from each RDS DB instance. This data gives you a granular view of the databases’ activity every second. You can enable Performance Insights without causing downtime, reboot, or failover.Amazon DevOps Guru for RDSuses the data from Performance Insights, Enhanced Monitoring, and CloudWatch to identify operational issues. It uses machine learning to detect and notify of database-related issues, including resource overutilization or misbehavior of certain SQL queries.ConclusionIn this blog post, we discussed technology choices, design principles, and recommended actions to optimize and increase efficiency of your databases. As your data grows, it is important to scale your database capacity in line with your user load, remove redundant data, optimize database queries, and optimize database performance. Figure 2 shows an overview of the tools you can use to optimize your databases.Figure 2. Tools you can use on AWS for optimizationOther blog posts in this seriesOptimizing your AWS Infrastructure for Sustainability, Part I: ComputeOptimizing your AWS Infrastructure for Sustainability, Part II: StorageOptimizing your AWS Infrastructure for Sustainability, Part III: NetworkingTAGS:Optimizing your AWS Infrastructure for Sustainability Series,SustainabilityOtis AntoniouOtis Antoniou is an Associate Solutions Architect based in London. He is part of the UK and Ireland SMB team and he enjoys helping customers innovate and solve business problems using AWS Services. Outside of work, Otis enjoys playing team sports, racket sports, and he is passionate about music.Ibtehaj AhmedIbtehaj Ahmed is a Solutions Architect at AWS based in London, UK. He loves to help early cloud adopters set up for success and utilize the right technologies. He is passionate about mobile development and purpose-built databases. Outside of work, he plays football regularly and enjoys participating in other sports.Darren KoDarren Ko is an Associate Solutions Architect based in London. He advises UK and Ireland SMB customers on rearchitecting and innovating on the cloud. Darren is passionate about solving sustainability challenges with machine learning.Ceren TahtasizCeren Tahtasiz is a Startup Solutions Architect based in London. She helps startups grow by understanding their goals and challenges, and guiding them on how they can get started on AWS. She’s passionate about enabling customers to launch cloud-native products that are resilient and scalable. Her core focus areas are serverless technologies and sustainability in cloud.
In Part I: Compute, Part II: Storage, and Part III: Networking of this series, we introduced strategies to optimize the compute, storage, and networking layers of your AWS architecture for sustainability. This post, Part IV, focuses on the database layer and proposes recommendations to optimize your databases’ utilization, performance, and queries. These recommendations are based […]
Otis Antoniou
Architecture Database Sustainability