Deploying Oracle RAC in AWS Outposts via FlashGrid Cluster
AWS Architecture BlogDeploying Oracle RAC in AWS Outposts via FlashGrid Clusterby Andreas Bogner, Art Danielov, and Martin Seman | on29 DEC 2022| inAmazon EC2,Amazon Simple Storage Service (S3),Architecture,AWS Outposts|Permalink|ShareAmazon Web Services (AWS) customers are deployingAWS Outpostsas a fully managed solution that delivers AWS infrastructure and services to on-premises or edge locations for a truly consistent hybrid experience. Those hybrid cloud workloads can require highly available Oracle databases running on- or close-to premises. One way to meet this requirement isOracle Real Application Clusters (RAC)on top of two or more AWS Outposts racks using the FlashGrid Cluster solution.In this post, we follow up on aMarketplace blogthat describes how to deploy Oracle RAC via FlashGrid Cluster in the AWS Regions. Deploying the solution onto AWS Outposts requires additional steps as the Outpost racks communicate between VPCs and across the on-premises network. In the following we explain how to configure the network for a multi-Outpost setup, how to deploy FlashGrid Cluster with Oracle RAC, and how to connect to the database cluster from the on-premises network.Solution overviewThis architecture uses two logical Outposts (42U racks) that are connected to different Availability Zones (AZs) in the same region for high availability. The networking is configured such that the communication betweenAmazon Elastic Compute Cloud (Amazon EC2)instances on two distinct Outpost racks uses Outpost’s local gateways and the corporate network. Therefore, data will not leave the premises unless explicitly copied to the Cloud region.The FlashGrid Cluster solution deploys one node to each Outpost rack and an additional quorum node to the Cloud region. The cluster nodes provide the Oracle ASM disk groups and the Oracle RAC nodes. FlashGrid Cluster for Oracle RAC ensures that storage is replicated between nodes (Figure 1).Figure 1. Architecture for a deployment across two logical AWS OutpostsWe provide a complete, step-by-step guide that deploys an Oracle RAC database across two Outpost racks.It takes three steps to get your database up and running:Networking: prepare the virtual private clouds (VPCs), subnets, and route tablesFlashGrid Cluster: use the FlashGrid Launcher to create an Oracle RAC clusterDatabase: configure Oracle RAC and connect to the databaseThe deployment uses a CloudFormation template that is generated based on the workload’s specific parameters.PrerequisitesFor this solution, you require:AnAWS accountTwo or more OutpostsAnevaluation licenseor aAWS Marketplacesubscription for a FlashGrid Cluster productBasic Oracle database knowledge for the database configurationNetworking setupA single VPC cannot span multiple Outposts. Therefore, each Outpost should have a separately configured VPC.A security group allowing traffic between the nodes of the cluster must be created in each VPC. Private IP addresses of the other nodes in the cluster should be configured as allowed sources of the traffic. (A security group rule cannot reference a security group in a different VPC.)The steps suggested for configuring the network, as in Figure 2:Create a VPC for each Outpost and a VPC peering connection between those VPCs. Ensure that the VPCs are associated with the respective local gateways on each Outpost.Within each VPC, create a subnet for the corresponding Outpost and a subnet in the Region. Add route tables to each Outpost subnet that allow cross-VPC routing (as in Table 1).Table 1. Main route table for VPC ‘Outpost 1’DestinationTarget10.0.1.0/24local10.0.2.0/25lgw-1111111111111111110.0.2.128/25pcx-11223344556677889pl-12345678vpce-12345678901234567For each of the cluster nodes, allocate a private IP address within the corresponding subnet.Within each VPC create a separate security group for each cluster. The security group must allow all inbound and outbound traffic from all nodes of the same cluster using their private IP addresses.Optional: Create a public subnet in the region in either one of the VPCs. Deploying a bastion host EC2 instance into this subnet will allow you to connect by SSH to the Oracle RAC nodes later.Open SSH access (TCP 22) and Oracle client access (default ports: TCP 1521, 1522) either by adding the corresponding rules to the same security groups or assigning additional security groups to all cluster node instances after the cluster deployment is complete.Create anAmazon Simple Storage Service (Amazon S3)gateway endpoint in each VPC and add corresponding routes. It is critical to useAmazon S3 gateway endpoints, otherwise the Oracle RAC nodes cannot download the installation binaries from the S3 bucket.Figure 2. Setting up subnets on each Outpost via the AWS consoleFlashGrid Cluster setupThe process of deploying FlashGrid Cluster consists of five main steps:Subscribing to a FlashGrid Cluster product inAWS MarketplaceUploading Oracle installation files to an S3 bucket in the RegionModifying the cluster configuration template with parameters specific to OutpostsUsing the FlashGrid Launcher tool to finalize cluster configuration and generate anAWS CloudFormationtemplateDeploying the CloudFormation templateSubscribing to a FlashGrid Cluster product in AWS MarketplaceOpen one of theFlashGrid Cluster product pages in AWS Marketplacecorresponding with the preferred operating system.SelectContinue.Select theManual Launchtab.ChooseAccept Software Terms.Creating an S3 bucket for Oracle installation filesDuring cluster provisioning, Oracle installation files are downloaded from an S3 bucket. The list of files that must be placed in the S3 bucket is displayed on the Oracle Files tab of the FlashGrid Launcher tool (next step). The S3 bucket is not supposed to have any sensitive data in it and can be hosted in any AWS Region. You can use the same bucket for multiple deployments.To create an S3 bucket with the correctIAMrole access:Create an S3 bucket or folder for uploading the installation filesWithin the IAM console, create a policy namedGetOracleFilesFromS3, which allows thes3:GetObjectaction on all uploaded filesCreate an EC2 instance role named GetOracleFilesFromS3 and attach theGetOracleFilesFromS3policy to itUse theGetOracleFilesFromS3role when configuring cluster parameters in theFlashGrid LauncherOnce the FlashGrid Launcher has provided a list of required Oracle installation files, place those into the S3 bucketModifying the cluster configuration template with parameters specific to OutpostsDownload theconfiguration file template, and open it in a text editor. The template is for a two-node RAC cluster on Outposts in Multi-AZ configuration.For the database node instances in the configuration file, manually set the following attributes:outpost_arn:ARN of the target Outpostip:private IP address assigned to the EC2 instancesg:security group IDins_type:instance type (ensure that this matches the available types for the Outposts)Creating a cluster configuration file using FlashGrid Launcher toolFlashGrid Launcher is an online tool that allows creating your desired configuration for the cluster and then generating a CloudFormation template for it.OpenFlashGrid Launcherand upload the customized configuration file created at the previous step.Follow the step-by-step instructions in the Launcher tool (see Figure 3) until you get to the last Launch step.a. In theOracle Filesstep, there is a list of installer files. Upload these files into the S3 bucket from the previous section (Creating an S3 bucket for Oracle installation files).b. In theStoragestep,ignore theIOPSandMBPSparameters, as these parameters will not be used because Outposts uses the GP2 type of volumes.c. In theNetworkstep, provide theSecurity Group IDthat will be used for thequorumnode located in the Region. Make sure to use a security group from the quorum node’s VPC.At theLaunchstep, clickLaunch FlashGrid. This generates a CloudFormation template and takes you to theAWS CloudFormation console.Figure 3. The FlashGrid Launcher generates a CloudFormation template based on various input parametersDeploying the CloudFormation templateOnce you are in AWS CloudFormation console, selectNext.Select the SSH key; do not change network parameters if you set them correctly in the Launcher. SelectNext.On theOptionspage, if you added tags in FlashGrid Launcher, thendo notadd the same tags in CloudFormation console. You are free to add further tags in this step. SelectNext.SelectCreate, and wait until the status of the stack changes toCREATE_COMPLETE.Connect by using SSH from the bastion host to the first cluster node asec2-userwith the SSH key you were provided when the stack was deployed.The welcome message details the current initialization status of the cluster:in progress,failed, orcompleted.If initialization is still in progress, wait for it to complete (this includes Oracle software installation and configuration). A broadcast message is delivered when initialization completes or fails. Cluster initialization takes 1 to 2 hours, depending on configuration.Oracle database configurationYou can create an Oracle RAC database (or multiple databases) usingOracle DBCAtool and following standard Oracle best practices.To connect to the Oracle RAC database using the SCAN listener, configure the Domain Name System (DNS) records and the connection string on client side:On the DNS server(s) used by clients, add a record resolving to the VPC Private IP address of the node instance for each database node. In a test environment without a DNS server, the entries can be added to/etc/hosts on the clients instead of the DNS server.In our example deployment, this is:rac1.example.com 10.0.1.77
rac2.example.com 10.0.2.77It is important that hostnames and domain names in the DNS records exactly match the hostnames as reported by the hostname command on the database servers.Finally, define a connection string with the addresses of all database nodes listed:SCAN=
     (DESCRIPTION=
           (TRANSPORT_CONNECT_TIMEOUT=3) (RETRY_COUNT=6)
           (ADDRESS=(PROTOCOL=tcp) (HOST=rac1.example.com) (PORT=1521))
           (ADDRESS=(PROTOCOL=tcp) (HOST=rac2.example.com) (PORT=1521))
           (CONNECT_DATA=
              (SERVER=DEDICATED)
              (SERVICE_NAME=<service name>)
           )
      )Visit theFlashGrid Help Centerfor more information on creating and connecting to a database.CleanupThe Oracle RAC instance together with the FlashGrid Cluster solution is deployed as a single CloudFormation stack. Deleting this stack will remove all associated resources, including the EBS volumes. Follow the approach detailed in theDelete Your Stacks But Keep Your Datablog post at the deployment stage to retain snapshots of all volumes automatically. You can also take snapshots of all EBS volumes manually before deleting the stack.ConclusionIn this blog post, we have explored how to deploy Oracle RAC across two or more Outpost racks using FlashGrid Cluster. Running Oracle RAC on top of Outposts provides a highly available database solution for use cases that require you to run the workload on-premises, as in data-residency or latency-critical scenarios. Using the growing number of features for AWS Outposts rack, and you can more efficiently run hybrid workloads using the same tools and automation both in Cloud Regions and on-premises.Further readingEstablishing Amazon EC2 infrastructure for Oracle RAC database migrations with FlashGridAWS Outposts User Guide for racks: Local gatewayFlashGrid Help Center: Creating a DatabaseFlashGrid Help Center: Connecting Database Clients (cloud only)FlashGrid Help Center: Connecting Database Clients using SCAN address (alternative method)Andreas BognerAndreas Bogner is a Global Solutions Architect at AWS based in Munich, Germany. He works with Automotive customers on large-scale cloud projects in the autonomous driving, manufacturing, and product innovation domains. A mathematician by training, he is obsessed with numbers and efficiency.Art DanielovArt Danielov is the CEO and CTO at FlashGrid and a frequent speaker on running Oracle database workloads on AWS, with a special focus on highly available, clustered, and high-performance databases.Martin SemanMartin Seman is a Senior Specialist Solutions Architect at AWS. He has more than 14 years of experience in the technology industry, from telecommunications to banking and security. Martin is now focusing on helping customers with guidance when building hybrid designs with AWS Outposts and Local Zones.
Amazon Web Services (AWS) customers are deploying AWS Outposts as a fully managed solution that delivers AWS infrastructure and services to on-premises or edge locations for a truly consistent hybrid experience. Those hybrid cloud workloads can require highly available Oracle databases running on- or close-to premises. One way to meet this requirement is Oracle Real […]
Andreas Bogner
Amazon EC2 Amazon Simple Storage Service (S3) Architecture AWS Outposts