Using AWS Backup and Oracle RMAN for backup/restore of Oracle databases on Amazon EC2: Part 1
AWS Architecture BlogUsing AWS Backup and Oracle RMAN for backup/restore of Oracle databases on Amazon EC2: Part 1by Jeevan Shetty, Bhanu Ganesh Gudivada, Santhosh Kumar Adapa, and Srini Ramaswamy | on08 JUL 2022| inAmazon EC2,Amazon Elastic Block Store (Amazon EBS),Amazon Elastic File System (EFS),Amazon Simple Storage Service (S3),Architecture,AWS Backup,AWS Identity and Access Management (IAM)|Permalink|ShareCustomers running Oracle databases onAmazon Elastic Compute Cloud (Amazon EC2)often take database and schema backups using Oracle native tools, likeData PumpandRecovery Manager(RMAN), to satisfy data protection, disaster recovery (DR), and compliance requirements. A priority is to reduce backup time as the data grows exponentially and recover sooner in case of failure/disaster.In situations where RMAN backup is used as a DR solution, usingAWS Backupto backup the file system and using RMAN to backup the archive logs are an efficient method to perform Oracle database point-in-time recovery in the event of a disaster.Sample use cases:Quickly build a copy of production database to test bug fixes or for a tuning exercise.Recover from a user error that removes data or corrupts existing data.A complete database recovery after a media failure.There are two options to backup the archive logs using RMAN:UsingOracle Secure Backup (OSB)and anAmazon Simple Storage Service (Amazon S3)bucket as the storage for archive logsUsingAmazon Elastic File System (Amazon EFS)as the storage for archive logsThis is Part 1 of this two-part series, we provide a mechanism to use AWS Backup to create a full backup of the EC2 instance, including the OS image, Oracle binaries, logs, and data files. In this post, we will use Oracle RMAN to perform archived redo log backup to an Amazon S3 bucket. Then, we demonstrate the steps to restore a database to a specific point-in-time using AWS Backup and Oracle RMAN.Solution overviewFigure 1 demonstrates the workflow:Oracle database on Amazon EC2 configured with Oracle Secure Backup (OSB)AWS Backup service to backup EC2 instance at regular intervals.AWS Identity and Access Management (IAM) role for EC2 instance that grants permission to write archive log backups to Amazon S3S3 bucket for storing Oracle RMAN archive log backupsFigure 1. Oracle Database in Amazon EC2 using AWS Backup and S3 for backup and restorePrerequisitesFor this solution, the following prerequisites are required:An AWS accountOracle database and AWS CLI in an EC2 instanceAccess to configure AWS BackupAcces to S3 bucket to store the RMAN archive log backup1. Configure AWS BackupYou can choose AWS Backup to schedule daily backups of the EC2 instance. AWS Backup efficiently stores your periodic backupsusing backup plans. Only the first EBS snapshot performs a full copy fromAmazon Elastic Block Storage (Amazon EBS)to Amazon S3. All subsequent snapshots are incremental snapshots, copying just the changed blocks from Amazon EBS to Amazon S3, thus, reducing backup duration and storage costs. Oracle supportsStorage Snapshot Optimization, which takes third-party snapshots of the database without placing the database in backup mode. By default, AWS Backup now creates crash-consistent backups of Amazon EBS volumes that are attached to an EC2 instance. Customers no longer have to stop their instance or coordinate between multiple Amazon EBS volumes attached to the same EC2 instance to ensure crash-consistency of their application state.You cancreate daily scheduled backupof EC2 instances. Figures 2, 3, and 4 are sample screenshots of the backup plan, associating an EC2 instance with the backup plan.Figure 2. Configure backup rule using AWS BackupFigure 3. Select EC2 instance containing Oracle Database for backupFigure 4. Summary screen showing the backup rule and resources managed by AWS BackupOracle RMAN archive log backupWhile AWS Backup is now creating a daily backup of the EC2 instance, we also want to make sure we backup the archived log files to a protected location. This will let us do point-in-time restores and restore to other recent times than just the last daily EC2 backup. Here, we provide the steps to backup archive log using RMAN to S3 bucket.Backup/restore archive logs to/from Amazon S3 using OSBBacking-up theOracle archivelogs is an important part of the process. In this section, we will describe how you can backup their Oracle Archive logs to Amazon S3 using OSB. Note: OSB is a separately licensed product from Oracle Corporation, so you will need to be properly licensed for OSB if you use this approach.2. Setup S3 bucket and IAM roleOracle Archive log backups can be scheduled usingcronscript to run at regular interval (for example, every 15 minutes). These backups are stored in an S3 bucket.a.Create an S3 bucketwithlifecycle policyto transition the objects to S3 Standard-Infrequent Access.b. Attach the following policy to the IAM Role of EC2 containing Oracle database orcreate an IAM role(ec2access) with the following policy and attach it to the EC2 instance. Updatebucket-namewith the bucket created in previous step.{
            "Sid": "S3BucketAccess",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObjectAcl",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-name",
                "arn:aws:s3:::bucket-name/*"
            ]
        }3. Setup OSBAfter we have configured the backup of EC2 instance using AWS Backup, we setup OSB in the EC2 instance. In these steps, we show the mechanism to configure OSB.a.Verify hardware and software prerequisitesfor OSB Cloud Module.b. Login to the EC2 instance with User ID owning the Oracle Binaries.c. Download Amazon S3 backup installer file (osbws_install.zip)d. Create Oracle wallet directory.mkdir $ORACLE_HOME/dbs/osbws_wallete. Create a file (osbws.sh) in the EC2 instance with the following commands. Update IAM role with the one created/updated in Step 2b.java -jar osbws_install.jar —IAMRole ec2access walletDir $ORACLE_HOME/dbs/osbws_wallet -libDir $ORACLE_HOME/lib/f. Change permission and run the file.chmod 700 osbws.sh
./osbws.shSample output:AWS credentials are valid.
Oracle Secure Backup Web Service wallet created in directory /u01/app/oracle/product/19.3.0.0/db_1/dbs/osbws_wallet.
Oracle Secure Backup Web Service initialization file /u01/app/oracle/product/19.3.0.0/db_1/dbs/osbwsoratst.ora created.
Downloading Oracle Secure Backup Web Service Software Library from file osbws_linux64.zip.
Download complete.g. Set ORACLE_SID by executing below command:. oraenvh. Running the script – osbws.sh installs OSB libraries and creates a file calledosbws<ORACLE_SID>.ora.i. Add/modify below with S3 bucket(bucket-name) and region(ex:us-west-2) created in Step 2a.OSB_WS_HOST=http://s3.us-west-2.amazonaws.com
OSB_WS_BUCKET=bucket-name
OSB_WS_LOCATION=us-west-24. Configure RMAN backup to S3 bucketWith OSB installed in the EC2 instance, you can backup Oracle archive logs to S3 bucket. These backups can be used to perform database point-in-time recovery in case of database crash/corruption. oratstis used as an example in below commands.a. Configure RMAN repository. Example below uses Oracle 19c and Oracle Sid –oratst.RMAN> configure channel device type sbt parms='SBT_LIBRARY=/u01/app/oracle/product/19.3.0.0/db_1/lib/libosbws.so,SBT_PARMS=(OSB_WS_PFILE=/u01/app/oracle/product/19.3.0.0/db_1/dbs/osbwsoratst.ora)';b. Create a script (for example,rman_archive.sh) with below commands, and schedule using crontab (example entry:*/5 * * * * rman_archive.sh) to run every 5 minutes. This will makes sure Oracle Archive logs are backed up to Amazon S3 frequently, thus ensuring an recovery point objective (RPO) of 5 minutes.dt=`date +%Y%m%d_%H%M%S`

rman target / log=rman_arch_bkup_oratst_${dt}.log <<EOF

RUN
{
	allocate channel c1_s3 device type sbt
	parms='SBT_LIBRARY=/u01/app/oracle/product/19.3.0.0/db_1/lib/libosbws.so,SBT_PARMS=(OSB_WS_PFILE=/u01/app/oracle/product/19.3.0.0/db_1/dbs/osbwsoratst.ora)' MAXPIECESIZE 10G;

	BACKUP ARCHIVELOG ALL delete all input;
	Backup CURRENT CONTROLFILE;

release channel c1_s3;
	
}

EOFc. Copy RMAN logs to S3 bucket. These logs contain the database identifier (DBID) that is required when we have to restore the database using Oracle RMAN.aws s3 cp rman_arch_bkup_oratst_${dt}.log s3://bucket-name5. Perform database point-in-time recoveryIn the event of a database crash/corruption, we can use AWS Backup service and Oracle RMAN Archive log backup to recover database to a specific point-in-time.a. Typically, you would pick the most recent recovery point completed before the time you wish to recover. Using AWS Backup, identify the recovery point ID to restore by following the steps onrestoring an Amazon EC2 instance. Note: when following the steps, be sure to set the “User data” settings as described in the next bullet item.After the EBS volumes are created from the snapshot, there is no need to wait for all of the data to transfer from Amazon S3 to your EBS volume before your attached instance can start accessing the volume. Amazon EBS snapshots implement lazy loading, so that you can begin using them right away.b. Be sure the database does not start automatically after restoring the EC2 instance, by renaming/etc/oratab. Use the following command in “User data” section while restoring EC2 instance. After database recovery, we can rename it back to/etc/oratab.#!/usr/bin/sh
sudo su - 
mv /etc/oratab /etc/oratab_bkc. Login to the EC2 instance once it is up, and execute the RMAN recovery commands mentioned. Identify theDBIDfrom RMAN logs saved in the S3 bucket. These commands use databaseoratstas an example:rman target /

RMAN> startup nomount

RMAN> set dbid DBID

# Below command is to restore the controlfile from autobackup

RMAN> RUN
{
    allocate channel c1_s3 device type sbt
	parms='SBT_LIBRARY=/u01/app/oracle/product/19.3.0.0/db_1/lib/libosbws.so,SBT_PARMS=(OSB_WS_PFILE=/u01/app/oracle/product/19.3.0.0/db_1/dbs/osbwsoratst.ora)';

    RESTORE CONTROLFILE FROM AUTOBACKUP;
    alter database mount;

    release channel c1_s3;
}


#Identify the recovery point (sequence_number) by listing the backups available in catalog.

RMAN> list backup;In Figure 5, the most recent archive log backed up is 380, so you can use this sequence number in the next set of RMAN commands.Figure 5. Sample output of Oracle RMAN “list backup” commandRMAN> RUN
{
    allocate channel c1_s3 device type sbt
	parms='SBT_LIBRARY=/u01/app/oracle/product/19.3.0.0/db_1/lib/libosbws.so,SBT_PARMS=(OSB_WS_PFILE=/u01/app/oracle/product/19.3.0.0/db_1/dbs/osbwsoratst.ora)';

    recover database until sequence sequence_number;
    ALTER DATABASE OPEN RESETLOGS;
    release channel c1_s3;
}d. To avoid performance issues due to lazy loading, after the database is open, run the following command to force a faster restoration of the blocks from S3 bucket to EBS volumes (this example allocates two channels andvalidatesthe entire database).RMAN> RUN
{
  ALLOCATE CHANNEL c1 DEVICE TYPE DISK;
  ALLOCATE CHANNEL c2 DEVICE TYPE DISK;
  VALIDATE database section size 1200M;
}e. This completes the recovery of database, and we can let the database automatically start by renaming file back to/etc/oratab.mv /etc/oratab_bk /etc/oratab6. Backup retentionEnsure that theAWS Backup lifecycle policymatches the Oracle Archive log backup retention. Also, follow documentation to configure Oracle backupretentionand deleteexpiredbackups. This is a sample command for Oracle backup retention:CONFIGURE BACKUP OPTIMIZATION ON;
CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 31 DAYS; 

RMAN> RUN
{
    allocate channel c1_s3 device type sbt
	parms='SBT_LIBRARY=/u01/app/oracle/product/19.3.0.0/db_1/lib/libosbws.so,SBT_PARMS=(OSB_WS_PFILE=/u01/app/oracle/product/19.3.0.0/db_1/dbs/osbwsoratst.ora)';

            crosscheck backup;
            delete noprompt obsolete;
            delete noprompt expired backup;

    release channel c1_s3;
}CleanupFollow below instructions to remove or cleanup the setup:Delete the backup plancreated in Step 1.Uninstall Oracle Secure Backupfrom the EC2 instance.Delete/Update IAM role(ec2access) to remove access from the S3 bucket used to store archive logs.Remove the cron entry from the EC2 instance configured in Step 4b.Delete the S3 bucketthat was created in Step 2a to store Oracle RMAN archive log backups.ConclusionIn this post, we demonstrate how to use AWS Backup and Oracle RMAN Archive log backup of Oracle databases running on Amazon EC2 can restore and recover efficiently to a point-in-time, without requiring an extra-step of restoring data files. Data files are restored as part of the AWS Backup EC2 instance restoration. You can leverage this solution to facilitate restoring copies of your production database for development or testing purposes, plus recover from a user error that removes data or corrupts existing data.To learn more about AWS Backup, refer to the AWS BackupAWS Backup Documentation.Jeevan ShettyJeevan Shetty is a Consultant with the AWS Proserve Team. He has been supporting and enabling customers to migrate their database from OnPrem data center to AWS cloud and also in migration from commercial database engines to open source database in Amazon.Bhanu Ganesh GudivadaBhanu Ganesh Gudivada is a Database Consultant with the Professional Services team at AWS and specializes in database migrations. He has been supporting and enabling customers to build high-available, cost-effective database solutions and migrate their commercial engines to AWS cloud. Bhanu is curious to learn and implement new technologies around the databases and orchestrate migrations through automation.Santhosh Kumar AdapaSanthosh Kumar Adapa is a Database Consultant with the Professional Services team at AWS. He works as a database migration specialist to help Amazon customers in design and implementing scalable, secure, performant, and robust database solutions on the cloud.Srini RamaswamySrini Ramaswamy is a Consultant with the AWS Proserve Team. He has been supporting and enabling customers to migrate their database from OnPrem data center to AWS cloud and also in migration from commercial database engines to open source database in Amazon.
Customers running Oracle databases on Amazon Elastic Compute Cloud (Amazon EC2) often take database and schema backups using Oracle native tools, like Data Pump and Recovery Manager (RMAN), to satisfy data protection, disaster recovery (DR), and compliance requirements. A priority is to reduce backup time as the data grows exponentially and recover sooner in case […]
Jeevan Shetty
Amazon EC2 Amazon Elastic Block Store (Amazon EBS) Amazon Elastic File System (EFS) Amazon Simple Storage Service (S3) Architecture AWS Backup AWS Identity and Access Management (IAM)