Journey to Adopt Cloud-Native Architecture Series: #4 – Governing Security at Scale and IAM Baselining
AWS Architecture BlogJourney to Adopt Cloud-Native Architecture Series: #4 – Governing Security at Scale and IAM Baseliningby Anuj Gupta and Neeraj Kumar | on01 OCT 2021| inArchitecture,AWS Control Tower,AWS IAM Identity Center,AWS Identity and Access Management (IAM),AWS Secrets Manager,AWS Trusted Advisor|Permalink|ShareIn Part 3 of this series,Improved Resiliency and Standardized Observability, we talked about design patterns that you can adopt to improve resiliency, achieve minimum business continuity, and scale applications with lengthy transactions (more than 3 minutes).As a refresher from previous blogs in this series, our example ecommerce company’s “Shoppers” application runs in the cloud. The company experiencedhypergrowth, which posed a number of platform and technology challenges, namely, they needed to scale on the backend without impacting users.Because of this hypergrowth, distributed denial of service (DDoS) attacks on the ecommerce company’s services increased 10 times in 6 months. Some of these attacks led to downtime and loss of revenue. This blog post shows you how we addressed these threats by implementing a multi-account strategy and applyingAWS Identity and Access Management (IAM)best practices.A multi-account strategy ensures security at scaleOriginally, the company’s production and non-production services were running in a single account. This meant non-production vulnerabilities like frequently changing code or privileged access could impact the production environment. Additionally, the application experienced issues due to unexpectedly reachingservice quotas. These include (but are not limited to) number of read replicas per master inAmazon Relational Database Service (Amazon RDS)and total storage for all DB instances inAuto Scaling Service QuotasforAmazon Elastic Compute Cloud (Amazon EC2).To address these issues, we followedmulti-account strategy best practices. We established the multi-account hierarchy shown in Figure 1 that includes the following eight organizational units (OUs) to meet business requirements:Security PROD OUSecurity SDLC OUInfrastructure PROD OUInfrastructure SDLC OUWorkload PROD OUWorkload SDLC OUSandbox OUTransitional OUToidentify the right fit for our needs, we evaluatedAWS Landing ZoneandAWS Control Tower. To reduce operation overhead of maintaining a solution, we used AWS Control Tower to deployguardrailsas service control policies (SCPs). These guardrails were then separated into production and non-production environments, creating the hierarchy shown in Figure 1.We created a new Payer (or Management) Account with Sandbox OU and Transitional OU under Root OU. We thenmoved existing AWS accountsunder the Transitional OU and Sandbox OU. We provisioned new accounts withAccount Factoryand gradually migrated services from existing AWS accounts into the newly formed Log Archive Account, Security Account, Network Account, and Shared Services Account andapplied appropriate guardrails. We thenregistered Sandbox OU with Control Tower. Additionally, we migrated the centralized logging solution fromPart 3 of this blog seriesto the Security Account. We moved non-production applications into the Dev and Test Accounts, respectively, to isolate workloads. We then moved existing accounts that had production services from the Transitional OU to Workload PROD OU.Figure 1. Multi-account hierarchyImplementing a multi-account strategy alleviated service quota challenges. It isolated variable demand non-production environments from more consistent production environments, which reduced the downtime caused by unplanned scaling events. The multi-account strategy enforces governance at scale, but also promotes innovation by allocating separate accounts with distinct security requirements for proof of concepts and experimentation. This reduces impact risks to production accounts and allows the required guardrails to be automatically applied.Improving access management and least privilege accessWhen the company experienced hypergrowth, they not only had to scale their application’s infrastructure, but they also had to increase how often they release their code. They also hired and onboarded new internal teams.To strengthen new/existing employees’ credentials, we usedAWS Trusted AdvisorforIAM Access Key Rotation. This identifies IAM users whose access keys have not been rotated for more than 90 days andcreated an automated way to rotate them. We then generated anIAM credential reportto identify IAM users that don’t need console access or that don’t need access keys. We gradually assigned these users role-based access versus IAM access keys.During aWell-Architected Security Pillarreview, we identified some applications that used hardcoded passwords that hadn’t been updated for more than 90 days. We re-factored these applications to get passwords fromAWS Secrets Managerand followedbest practicesfor performance.Additionally, we set up a system toautomatically change passwords for RDS databasesand wrote anAWS Lambdafunction to update passwords for third-party integration. Some applications onAmazon EC2were using IAM access keys to access AWS services. We re-factored them to get permissions from the EC2 instance role attached to the EC2 instances, which reduced operational burden of rotating access keys.UsingIAM Access Analyzer, we analyzedAWS CloudTraillogs and generated policies for IAM roles. This helped us determine the least privilege permissions required for the roles as mentioned in theIAM Access Analyzer makes it easier to implement least privilege permissions by generating IAM policies based on access activityblog.To streamline access for internal users, we migrated users toAWS Single Sign-On (AWS SSO)federated access. We enabled all features inAWS Organizationsto use AWS SSO and created permission sets to define access boundaries for different functions. We assigned permission sets to different user groups and assigned users to user groups based on their job function. This allowed us to reduce the number of IAM policies and use tag-based control when defining AWS SSO permissions policies.We followed the guidance in theAttribute-based Access Control with AWS SSOblog post to map user attributes and use tags to define permissions boundaries for user groups. This allowed us to provide access to users based on specific teams, projects, and departments. We enforced multi-factor authentication (MFA) for all AWS SSO users by configuringMFA settingsto allow sign in only when an MFA device has been registered.These improvements ensure that only the right people have access to the required resources for the right time. They reduce the risk of compromised security credentials by usingAWS Security Token Service (AWS STS)to generate temporary credentials when needed. System passwords are better protected from unwanted access and automatically rotated for improved security. AWS SSO also allows us to enforce permissions at scale when people’s job functions change within or across teams.ConclusionIn this blog post, we described design patterns we used to implement security governance at scale using multi-account strategy and AWS SSO integrations. We also talked about patterns you can adopt for IAM baselining that allow least privilege access, checking for IAM best practices, and proactively detecting unwanted access.This blog post also covers why you need to refresh your threat model during hyperscale growth and how different services can make it easier to enforce security controls. In the next blog, we will talk about more security design patterns to improve infrastructure security and incident response during hyperscale.Find out moreOther blogs in this seriesJourney to Adopt Cloud-Native Architecture Series: #1 – Preparing your Applications for HypergrowthJourney to Adopt Cloud-Native Architecture Series: #2 – Maximizing System ThroughputJourney to Adopt Cloud-Native Architecture Series: #3 – Improved Resilience and Standardized ObservabilityJourney to Adopt Cloud-Native Architecture Series #5 – Enhancing Threat Detection, Data Protection, and Incident ResponseRelated informationBlog:How to use AWS Organizations to simplify security at enormous scaleBlog:IAM Access Analyzer makes it easier to implement least privilege permissions by generating IAM policies based on access activityUser guide:Getting started with AWS Control TowerUser guide:AWS Secrets Manager, Cache secrets to improve performanceWhitepaper:Organizing Your AWS Environment Using Multiple AccountsTAGS:Hyperscale SeriesAnuj GuptaAnuj Gupta is a Principal Solutions Architect working with digital-native business customers on their cloud-native journey. He is passionate about using technology to solve challenging problems and has worked with customers to build highly distributed and low-latency applications. He also contributes to open-source solutions. Outside of work, he loves traveling with his family and meeting new people.Neeraj KumarNeeraj Kumar is a Principal Solutions Architect at AWS. He has 20+ year experience in IT, including Cloud Computing, Enterprise Architecture, IT Strategy & Transformation. Prior to joining AWS in Jan 2016, he worked across multiple industry verticals - Oil & Gas, Consumer Tech & Consulting. He is currently based out of San Francisco Bay Area. Neeraj’s passion is solving business problems on customer’s behalf using distributed, cloud-native and well-architected design patterns.
In Part 3 of this series, Improved Resiliency and Standardized Observability, we talked about design patterns that you can adopt to improve resiliency, achieve minimum business continuity, and scale applications with lengthy transactions (more than 3 minutes). As a refresher from previous blogs in this series, our example ecommerce company’s “Shoppers” application runs in the cloud. […]
Anuj Gupta
Architecture AWS Control Tower AWS IAM Identity Center AWS Identity and Access Management (IAM) AWS Secrets Manager AWS Trusted Advisor