Selecting the right database and database migration plan for your workloads
AWS Architecture BlogSelecting the right database and database migration plan for your workloadsby Nikhil Anand | on28 MAR 2022| inAmazon Aurora,Amazon RDS Custom,Architecture,AWS Database Migration Service,AWS Schema Conversion Tool|Permalink|ShareThere has been a tectonic shift in the approach to hosting enterprise workloads. Companies are rapidly moving from on-premises data centers to cloud-based services. The driving factor has been the ability to innovate faster on the cloud.Your transition to cloud can be straightforward, but it does go beyond the usual ‘lift-and-shift’ approach. To start with, you’ll need a cloud provider that provides a data store and computing facility. But you’ll be able togrowyour business with a provider that has purpose-built resources and a platform that supports innovation. Companies are using higher-level services such as fully managed databases and serverless compute offered byAmazon Web Services (AWS)to get the most out of their cloud adoption journey.In this blog, we will focus on thedatabasetier of enterprise applications. We will evaluate the reasons for moving to managed or purpose-built databases. Then we’ll discuss in more detail the options you have and how to implement the right database migration plan for your workloads.Why move to a managed database?Managed databases, orpurpose-built databases, are managed services by AWS that free you to focus on other business and technical priorities. Different types of databases fill certain needs, and so the database tier in an application should never be a one-size-fits-all approach. Based on the kind of application you are developing, you should research managed options that suit your business and enterprise use cases.Database TypeUse CasesAWS ServiceRelationalTraditional applications, enterprise resource planning (ERP), customer relationship management (CRM), ecommerceAmazon AuroraAmazon RDSAmazon RedshiftKey-ValueHigh-traffic web applications, ecommerce systems, gaming applicationsAmazon DynamoDBIn-MemoryCaching, session management, gaming leaderboards, geospatial applicationsAmazon ElastiCacheAmazon MemoryDB for RedisDocumentContent management, catalogs, user profilesAmazon DocumentDB (with MongoDB compatibility)Wide ColumnHigh-scale industrial apps for equipment maintenance, fleet management, and route optimizationAmazon KeyspacesGraphFraud detection, social networking, recommendation enginesAmazon NeptuneTime SeriesInternet of Things (IoT) applications, DevOps, industrial telemetryAmazon TimeStreamLedgerSystems of record, supply chain, registrations, banking transactionsAmazon QLDBTable 1. Managed databases by AWSManaged database featuresManageability.The top priority and most valuable asset that you own as a company is your data. While data remains your key asset, spending time on database management is not the optimum use of your time. Managed services have built-in reliable tooling for multiple aspects of database management, which can help your database operate at the highest level.Availability and disaster recovery.Most managed databases at AWS are highly available by default. For example,Amazon Aurorais designed to offer 99.99% availability, replicating six copies of your data across three Availability Zones (Figure 1). It backs up your data continuously toAmazon S3. It recovers transparently from physical storage failures; instance failover typically takes less than 30 seconds.Figure 1. Replication across three Availability Zones with Amazon Aurora DB clusterWith managed databases, you get multiple options to create a highly available and fault tolerant database. Alternatively, if you choose to self-host a database elsewhere you will have to formulate your own disaster recovery (DR) strategy. This takes meticulous DR planning and relies heavily on a constant monitoring solution.Elasticity and agility:Cloud offers elasticity for scaling your database. You can scale in minutes and spin up and down additional servers and storage size as needed. It offers you flexibility with capacity planning. You can always reassess your database tier to see if it is over or under provisioned.Self-managed databases on AWSIf I do not need a managed database, should I still use Amazon EC2 to host my database?Here are some cases when you may find it beneficial to host your database onAmazon EC2instances:You need full control over the database and access to its underlying operating system, database installation, and configuration.You are ready to plan for the high availability of your database, and prepare your disaster recovery plan.You want to administer your database, including backups and recovery. You must perform tasks such as patching the operating system and the database, tuning the operating system and database parameters, managing security, and configuring high availability or replication.You want to use features that aren’t currently supported by managed services.You need a specific database version that isn’t supported by AWS managed database service.Your database size and performance needs exceed the limits of the managed service.You want to avoid automatic software patches that might not be compliant with your applications.You want to achieve higher IOPS and storage capacity than the current limits of the managed services.Can I customize my underlying OS and database environment?Amazon RDS Customis a managed database service for applications that require customization of the underlying operating system and database environment. With Amazon RDS Custom, you have access to the underlying EC2 instance that hosts the DB engine. You can access the EC2 instance via secure shell (AWS Systems Manager and SSH) and perform the customizations to suit your application needs.Choosing which migration plan to implementThe first step in your cloud database migration journey is identifying which database that you want to migrate to. For relational databases, determine the migration strategy. In the majority of the database migrations, you can choose to rehost, replatform, or refactor.Refer theAWS Prescriptive Guidance for choosing a migration strategyfor relational databases.The next is determining which database migration plan best serves your needs. AWS provides a number of options to help correctly handle your data migration with minimal downtime. Here are the database migration plans that you can consider using for your cloud database adoption journey:1. AWS-offered AWS Database Migration Service (AWS DMS):AWS Database Migration Service (AWS DMS)is a self-service option for migrating databases. You can use AWS DMS to migrate between homogeneous database types, such as going from one MySQL instance to a new one. You can also use AWS DMS between heterogeneous database types, such as moving from a commercial database like Oracle to a cloud-native relational database like Aurora. Read tutorials about migrating sample data:AWS Database Migration Service Step-by-Step Walkthroughs.AWS DMS offers minimal downtime, supports widely used databases, supports on-going replication, is offered as a low-cost service, and is highly reliable. If you are looking for an end-to-end service for database migration, consider AWS DMS.2. AWS Database Migration Service (DMS) with AWS Schema Conversion Tool (SCT):If you are migrating between heterogenous databases, use theAWS Schema Conversion Tool (SCT). It converts the source database schema and the database code objects (like views, stored procedures, and functions) to a format compatible with the target database (Figure 2).Figure 2. Supported conversions with AWS Schema Conversion ToolFor heterogeneous migrations, consider using AWS DMS with AWS SCT.3. Database Freedom Program:If you are new to cloud computing or if your database migration plan must be evaluated and validated by industry experts, then try the Database Freedom Program. Database Freedom is a unique program designed to assist customers in migrating to AWS databases and analytics services. They will provide technical advice, migration support, and financial assistance.You can use theContact Uspage for the Database Freedom program to get in touch with experts that can assist you in your cloud adoption journey.4. AWS Professional Services and Partners:You may have in-house database expertise, but need end-to-end implementation assistance across different application tiers. Get help from theProfessional Services of AWSor connect with the certified network of AWS Partners.AWS Database Migration Service Delivery Partnershelp customers use AWS DMS to migrate databases to AWS securely, while minimizing application downtime and following best practices.ConclusionMigrating to the cloud is a journey that is ever-evolving. To remain focused on your innovations, leverage the managed services of AWS for your migration journey.I hope this post helps you consider using a managed database service when available, and effectively evaluate and choose the right database migration plan for your enterprise. For any queries, feel free to get in touch with us. We will be happy to help you in your cloud journey.Happy migrating!Additional ReadingOptimizing costs in Amazon RDSSecurity best practices for Amazon RDSBest practices for working with Amazon Aurora ServerlessBest Practices for Designing and Architecting with DynamoDBBest Practices for Amazon RDSNikhil AnandNikhil is a Solutions Architect at AWS with an extensive background in DevOps and Compute offerings at AWS. He is an avid reader keen to explore different technologies. Outside of work, he likes to keep himself engaged with sports and never misses a workout opportunity.
There has been a tectonic shift in the approach to hosting enterprise workloads. Companies are rapidly moving from on-premises data centers to cloud-based services. The driving factor has been the ability to innovate faster on the cloud. Your transition to cloud can be straightforward, but it does go beyond the usual ‘lift-and-shift’ approach. To start […]
Nikhil Anand
Amazon Aurora Amazon RDS Custom Architecture AWS Database Migration Service AWS Schema Conversion Tool