Introducing the new AWS Well-Architected Machine Learning Lens
AWS Architecture BlogIntroducing the new AWS Well-Architected Machine Learning Lensby Haleh Najafzadeh | on13 OCT 2021| inAmazon Machine Learning,Architecture,AWS Well-Architected Framework|Permalink|ShareTheAWS Well-Architected Frameworkprovides you with a formal approach to compare your workloads against best practices. It also includes guidance on how to make improvements.Machine learning (ML) algorithms discover and learn patterns in data, and construct mathematical models to predict future data. These solutions can revolutionize lives through better diagnoses of diseases, environmental protections, products and services transformation, and more.Your ML models depend on the quality of input data to generate accurate results. As data changes over time, monitoring is required to continuously detect, correct, and mitigate issues. This improves accuracy and performance. It also may require you to retrain your model with the latest refined data.Application workloads rely on step-by-step instructions to solve a problem. ML workloads enable algorithms to learn from data through an iterative and continuous cycle. We are announcing a brand-new version of theAWS Well-Architected Machine Learning Lenswhitepaper. It complements and builds upon the Well-Architected Framework to address this difference between these two types of workloads.The whitepaper provides you with a set of established cloud and technology agnostic best practices. You can apply this guidance and architectural principles when designing your ML workloads, or after your workloads have entered production as part of continuous improvement. The paper includes guidance and resources to help you implement these best practices on AWS.The Well-Architected Machine Learning Lens componentsThe Lens includes four focus areas:1.The Well-Architected Machine Learning Design Principles —A set of considerations that are used as the basis for a Well-Architected ML workload. These design principles are the guiding light for the collection of the best practices in the ML Lens.2.The Well-Architected Machine Learning Lifecycle —This integrates theWell-Architected Frameworkinto theMachine Learning Lifecycleas can be seen in figure 1.The Well-Architected Frameworkpillars includes:Operational ExcellenceSecurityReliabilityPerformance EfficiencyCost OptimizationThe Machine Learning Lifecyclephases referenced in the ML Lens include:Business goal identificationML problem framingData processing (data collection, data pre-processing, feature engineering)Model development (training, tuning, evaluation)Model deployment (prediction, inference)Model monitoringFigure 1. Well-Architected Machine Learning LifecycleIn theWell-Architected ML Lens whitepaper, theWell-Architected Machine Learning Lifecycleapplies theWell-Architected Framework pillarsto each of the lifecycle phases.3.Cloud and technology agnostic best practices —These are best practices for each ML lifecycle phase across the Well-Architected Framework pillars. Best practices are accompanied by:Implementation guidancethat provides AWS implementation plans for each best practice with references to AWS technologies and resources.Resourcesas a set of links to AWS documents, blogs, videos, and code examples as supporting resources to the best practices and their implementation plans.4.ML Lifecycle architecture diagrams —These illustrate processes, technologies, and components that support many of the best practices, shown in Figure 2. They include: Feature stores, Model Registry, lineage tracker, alarm manager, scheduler, and more. Different pipeline technologies are illustrated using these architecture diagrams.Figure 2. Machine Learning Lifecycle phases with expanded componentsWhere should you apply the Well-Architected Machine Learning Lens?Use the Well-Architected ML Lens to:Make informed decisions —Plan early and make informed decisions by reviewing best practices before a new workload design begins.Build and deploy faster —Use the best practices to guide you through building new Well-Architected workloads across the ML lifecycle.Lower or mitigate risks —Evaluate existing workloads regularly to identify, mitigate, and address potential issues early.Learn AWS best practices —Use the provided implementation plans as guidance on implementing the best practices on AWS.ConclusionThe newWell-Architected Machine Learning Lenswhitepaper is available now. Use the Lens to help ensure that your ML workloads are architected with operational excellence, security, reliability, performance efficiency, and cost optimization in mind.Special thanks to everyone across the AWS Solution Architecture and Machine Learning communities who contributed. These contributions encompassed diverse perspectives, expertise, and experiences in developing the newAWS Well-Architected Machine Learning Lens.Haleh NajafzadehHaleh is a Senior Solutions Architecture Manager, AWS Well-Architected. She has over 20 years of experience in applying scientific techniques to challenging industrial problems while sharing technology best practices enabling customers with architecting and implementing solutions at scale.
The AWS Well-Architected Framework provides you with a formal approach to compare your workloads against best practices. It also includes guidance on how to make improvements. Machine learning (ML) algorithms discover and learn patterns in data, and construct mathematical models to predict future data. These solutions can revolutionize lives through better diagnoses of diseases, environmental […]
Haleh Najafzadeh
Amazon Machine Learning Architecture AWS Well-Architected Framework