Optimize AI/ML workloads for sustainability: Part 1, identify business goals, validate ML use, and process data
AWS Architecture BlogOptimize AI/ML workloads for sustainability: Part 1, identify business goals, validate ML use, and process databy Benoit de Chateauvieux, Eddie Pick, Dan Ferguson, and Brendan Sisson | on10 FEB 2022| inArtificial Intelligence,AWS Well-Architected,Sustainability|Permalink|ShareTraining artificial intelligence (AI) services and machine learning (ML) workloads uses a lot of energy—and they are becomingbigger and more complex. As an example, theCarbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelsstudy estimates that a single training session for a language model like GPT-3 can have a carbon footprint similar to traveling 703,808 kilometers by car.Although ML uses a lot of energy, it is also one of the best tools we have to fight the effects of climate change. For example, we’ve used ML tohelp deliver food and pharmaceuticals safely and with much less waste,reduce the cost and risk involved in maintaining wind farms,restore at-risk ecosystems, andpredict and understand extreme weather.In this series of three blog posts, we’ll provide guidance from theSustainability Pillarof theAWS Well-Architected Frameworkto reduce the carbon footprint of your AI/ML workloads.This first post follows the first three phases provided in theWell-Architected machine learning lifecycle(Figure 1):Business goal identificationML problem framingData processing (data collection, data preprocessing, feature engineering)You’ll learn best practices for each phase to help you review and refine your workloads to maximize utilization and minimize waste and the total resources deployed and powered to support your workload.Figure 1. ML lifecycleBusiness goal identificationDefine the overall environmental impact or benefitMeasure your workload’s impact and its contribution to the overall sustainability goals of the organization. Questions you should ask:How does this workload support our overall sustainability mission?How much data will we have to store and process? What is the impact of training the model? How often will we have to re-train?What are the impacts resulting from customer use of this workload?What will be the productive output compared with this total impact?Asking these questions will help you establish specific sustainability objectives and success criteria to measure against in the future.ML problem framingIdentify if ML is the right solutionAlways ask if AI/ML is right for your workload. There is no need to use computationally intensive AI when a simpler, more sustainable approach might succeed just as well.For example, using ML to route Internet of Things (IoT) messages may be unwarranted; you can express the logic with aRules Engine.Consider AI services and pre-trained modelsOnce you decide if AI/ML is the right tool, consider whether the workload needs to be developed as a custom model.Many workloads can use the managedAWS AI servicesshown in Figure 2. Using these services means that you won’t need the associated resources to collect/store/process data and to prepare/train/tune/deploy an ML model.Figure 2. Managed AWS AI servicesIf adopting a fully managed AI service is not appropriate, evaluate if you can use pre-existing datasets, algorithms, or models.AWS Marketplaceoffers over 1,400 ML-related assets that customers can subscribe to. You can alsofine-tune an existing modelstarting from a pre-trained model, like those available onHugging Face. Using pre-trained models from third parties can reduce the resources you need for data preparation and model training.Select sustainable RegionsSelect an AWS Region with sustainable energy sources. When regulations and legal aspects allow, choose Regionsnear Amazon renewable energy projectsand Regions where the grid has low published carbon intensity to host your data and workloads.Data processing (data collection, data preprocessing, feature engineering)Avoid datasets and processing duplicationEvaluate if you can avoid data processing by using existing publicly available datasets likeAWS Data ExchangeandOpen Data on AWS(which includes theAmazon Sustainability Data Initiative). They offer weather and climate datasets, satellite imagery, air quality or energy data, among others. When you use these curated datasets, it avoids duplicating the compute and storage resources needed to download the data from the providers, store it in the cloud, organize, and clean it.For internal data, you can also reduce duplication and rerun of feature engineering code across teams and projects by using a feature storage, such asAmazon SageMaker Feature Store.Once your data is ready for training, usepipe input modeto stream it fromAmazon Simple Storage Service (Amazon S3)instead of copying it toAmazon Elastic Block Store (Amazon EBS). This way, you can reduce the size of your EBS volumes.Minimize idle resources with serverless data pipelinesAdopt a serverless architecture for your data pipeline so it only provisions resources when work needs to be done. For example, when you useAWS GlueandAWS Step Functionsfor data ingestion and preprocessing, you are not maintaining compute infrastructure 24/7. As shown in Figure 3, Step Functions can orchestrate AWS Glue jobs to create event-based serverless ETL/ELT pipelines.Figure 3. Orchestrating data preparation with AWS Glue and Step FunctionsImplement data lifecycle policies aligned with your sustainability goalsClassify data to understand its significance to your workload and your business outcomes. Use this information to determine when you can move data to more energy-efficient storage or safely delete it.Manage the lifecycle of all your data and automatically enforce deletion timelines to minimize the total storage requirements of your workload usingAmazon S3 Lifecycle policies. TheAmazon S3 Intelligent-Tiering storage classwill automatically move your data to the most sustainable access tier when access patterns change.Define data retention periods that support your sustainability goals while meeting your business requirements, not exceeding them.Adopt sustainable storage optionsUse the appropriate storage tier to reduce the carbon impact of your workload. On Amazon S3, for example, you can use energy-efficient, archival-class storage for infrequently accessed data, as shown in Figure 4. And if you can easily recreate an infrequently accessed dataset, use the Amazon S3 One Zone-IA class to minimize the total data stored.Figure 4. Data access patterns for Amazon S3Don’t over-provision block storage for notebooks and use object storage services like Amazon S3 for common datasets.Tip: You can check the free disk space on your SageMaker Notebooks using!df -h.Select efficient file formats and compression algorithmsUse efficient file formats such asParquetorORCto train your models. Compared to CSV, they can help you reduce your storage byup to 87%.Migrating to a more efficient compression algorithm can also greatly contribute to your storage reduction efforts. For example, Zstandard produces10–15% smaller filesthan Gzip at the same compression speed. SomeSageMaker built-in algorithmsacceptx-recordio-protobufinput, which can be streamed directly from Amazon S3 instead of being copied to a notebook instance.Minimize data movement across networksCompress your data before moving it over the network.Minimize data movement across networks when selecting a Region; store your data close to your producers and train your models close to your data.Measure results and improveTo monitor and quantify improvements, track the following metrics:Total size of your S3 buckets and storage class distribution, usingAmazon S3 Storage LensDiskUtilizationmetricof your SageMaker processing jobsStorageBytesmetricof your SageMaker Studio shared storage volumeConclusionIn this blog post, we discussed the importance of defining the overall environmental impact or benefit of your ML workload and why managed AI services or pre-trained ML models are sustainable alternatives to custom models. You also learned best practices to reduce the carbon footprint of your ML workload in the data processing phase.In the next post, we will continue our sustainability journey through the ML lifecycle and discuss the best practices you can follow in the model development phase.Want to learn more?Check out theSustainability Pillar of the AWS Well-Architected Framework, theArchitecting for sustainability session at re:Invent 2021, and other blog posts onarchitecting for sustainability.Looking for more architecture content?AWS Architecture Centerprovides reference architecture diagrams, vetted architecture solutions,Well-Architectedbest practices, patterns, icons, and more!Other posts in this seriesOptimize AI/ML workloads for sustainability: Part 2, model developmentOptimize AI/ML workloads for sustainability: Part 3, deployment and monitoringTAGS:AWS Well-Architected Framework,Optimize AI/ML workloads for sustainability,SustainabilityBenoit de ChateauvieuxBenoit de Chateauvieux is a Startup Solutions Architect at AWS, based in Montreal, Canada. As a former CTO, he enjoys helping startups build great and sustainable products using the cloud. Outside of work, you’ll find Benoit in canoe-camping expeditions, paddling across Canadian rivers.Eddie PickEddie Pick is a Senior Startup Solutions Architect at AWS based in Montréal, Canada. As an ex co-founder and former CTO, his goal is to help startups build great products faster on the Cloud, particularly using machine learning.Dan FergusonDan Ferguson is a Solutions Architect at AWS, based in New York, USA. As a machine learning services expert, Dan works to support customers on their journey to integrating ML workflows efficiently, effectively, and sustainably.Brendan SissonBrendan Sisson is a Principal Sustainability Solutions Architect at AWS based in London, UK. As a contributor to the Sustainability Pillar of the AWS Well-Architected Framework, he supports customers on how they can optimize their workloads running in the AWS Cloud and how they can use the AWS Cloud to help solve their wider sustainability challenges.
Training artificial intelligence (AI) services and machine learning (ML) workloads uses a lot of energy—and they are becoming bigger and more complex. As an example, the Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models study estimates that a single training session for a language model like GPT-3 can have a carbon footprint […]
Benoit de Chateauvieux
Artificial Intelligence AWS Well-Architected Sustainability