[박상준] RAC환경에서 Voting Disk OCR Backup

[박상준]

▒ About Oracle Clusterware

Oracle Real Application Clusters (Oracle RAC) uses Oracle Clusterware as the infrastructure that binds together multiple nodes that then operate as a single server. Oracle Clusterware is a portable cluster management solution that is integrated with Oracle Database. In an Oracle RAC environment, Oracle Clusterware monitors all Oracle components (such as instances and Listeners). If a failure occurs, Oracle Clusterware automatically attempts to restart the failed component and also redirects
operations to a surviving component.

Oracle Clusterware includes a high availability framework for managing any application that runs on your cluster. Oracle Clusterware manages applications to ensure they start when the system starts. Oracle Clusterware also monitors the applications to make sure that they are always available. For example, if an application process fails, then Oracle Clusterware attempts to restart the process based on scripts that you customize. If a node in the cluster fails, then you can program application processes that typically run on the failed node to restart on another node in the cluster.
Oracle Clusterware includes two important components: the voting disk and the OCR. The voting disk is a file that manages information about node membership, and the OCR is a file that manages cluster and Oracle RAC database configuration information.

The Oracle Clusterware installation process creates the voting disk and the OCR on shared storage. If you select the option for normal redundant copies during the installation process, then Oracle Clusterware automatically maintains redundant copies of these files to prevent the files from becoming single points of failure. The normal redundancy feature also eliminates the need for third-party storage redundancy solutions. When you use normal redundancy, Oracle Clusterware
automatically maintains two copies of the OCR file and three copies of the voting disk file.

▒ Backing Up and Recovering Voting Disks

High availability configurations have redundant hardware and software that maintain operations by avoiding single points of failure. When a component is down, Oracle Clusterware redirects its managed resources to a backup component.
The voting disk records node membership information. A node must be able to access more than half of the voting disks at any time. To avoid simultaneous loss of multiple voting disks, each voting disk should be on a storage device that does not share any components (controller, interconnect, and so on) with the storage devices used for the other voting disks.

For example, if you have five voting disks configured, then a node must be able to access at least three of the voting disks at any time. If a node cannot access the minimum required number of voting disks it is evicted, or removed, from the cluster.
After the cause of the failure has been corrected and access to the voting disks has been restored, you can instruct Oracle Clusterware to recover the failed node and restore it to the cluster.

▒ Backing Up Voting Disks

Because the node membership information does not usually change, you do not need to back up the voting disk every day. However, back up the voting disks at the following times:

    ■ After installation
    ■ After adding nodes to or deleting nodes from the cluster
    ■ After performing voting disk add or delete operations

To make a backup copy of the voting disk, use the Linux dd command. Perform this operation on every voting disk as needed where voting_disk_name is the name of the active voting disk and backup_file_name is the name of the file to which you
want to back up the voting disk contents:

dd if=voting_disk_name of=backup_file_name

If your voting disk is stored on a raw device, use the device name in place of voting_disk_name. For example:

dd if=/dev/sdd1 of=/tmp/voting.dmp
When you use the dd command for making backups of the voting disk, the backup can be performed while the Cluster Ready Services (CRS) process is active; you do not need to stop the crsd.bin process before taking a backup of the voting disk.

▒ Recovering Voting Disks

If a voting disk is damaged, and no longer usable by Oracle Clusterware, you can recover the voting disk if you have a backup file. Run the following command to recover a voting disk where backup_file_name is the name of the voting disk
backup file and voting_disk_name is the name of the active voting disk:

dd if=backup_file_name of=voting_disk_name

▒ Adding and Removing Voting Disks

You can dynamically add and remove voting disks after installing Oracle RAC. Do this using the following commands where path is the fully qualified path for the additional voting disk. Run the following command as the root user to add a voting disk:

crsctl add css votedisk path

Run the following command as the root user to remove a voting disk:

crsctl delete css votedisk path

[참고]

Note: If your cluster is down, then you can use the -force option to modify the voting disk configuration when using either of these commands without interacting with active Oracle Clusterware daemons. However, you may corrupt your cluster configuration if you use the -force option while a cluster node is active

▒ Backing Up and Recovering the Oracle Cluster Registry

Oracle Clusterware automatically creates OCR backups every 4 hours. At any one time, Oracle Clusterware always retains the latest 3 backup copies of the OCR that are 4 hours old, 1 day old, and 1 week old.

You cannot customize the backup frequencies or the number of files that Oracle Clusterware retains. You can use any backup software to copy the automatically generated backup files at least once daily to a different device from where the primary OCR file resides. The default location for generating backups on Red Hat Linux systems is CRS_home/cdata/cluster_name where cluster_name is the name of your cluster and CRS_home is the home directory of your Oracle Clusterware
installation. This section contains the following topics:

    ■ Viewing Available OCR Backups
    ■ Backing Up the OCR
    ■ Recovering the OCR

▒ Viewing Available OCR Backups

To find the most recent backup of the OCR, on any node in the cluster, use the following command:

ocrconfig -showbackup

▒ Backing Up the OCR

Because of the importance of OCR information, Oracle recommends that you use theocrconfig tool to make copies of the automatically created backup files at least once a day.

In addition to using the automatically created OCR backup files, you should also export the OCR contents to a file before and after making significant configuration changes, such as adding or deleting nodes from your environment, modifying Oracle Clusterware recource, or createing a database. Exporting the OCR contents to a file lets you restore the OCR if your configuration changes cause errors. For example, if you have unresolvable configuration problems, or if you are unable to restart your cluster database after such changes, then you can restore your configuration by importing the saved OCR content from the valid configuration.

To export the contents of the OCR to a file, use the following command, where backup_file_name is the name of the OCR backup file you want to create:

ocrconfig -export backup_file_name

[참고]

 Note: You must be logged in as the root user to run the ocrconfig command

▒ Recovering the OCR
This section describes two methods for recovering the OCR. The first method uses automatically generated OCR file copies and the second method uses manually created OCR export files.

In event of a failure, before you attempt to restore the OCR, ensure that the OCR is unavailable. Run the following command to check the status of the OCR:

ocrcheck

If this command does not display the message 'Device/File integrity check succeeded' for at least one copy of the OCR, then both the primary OCR and the OCR mirror have failed. You must restore the OCR from a backup.

Restoring the Oracle Cluster Registry from Automatically Generated OCR Backups

When restoring the OCR from automatically generated backups, you first have to determine which backup file you will use for the recovery.

To restore the OCR from an automatically generated backup on a Red Hat Linux system:
1. Identify the available OCR backups using the ocrconfig command:

# ocrconfig -showbackup

2. Review the contents of the backup using the following ocrdump command, where file_name is the name of the OCR backup file:

$ ocrdump -backupfile file_name

3. As the root user, stop Oracle Clusterware on all the nodes in your Oracle RAC cluster by executing the following command:

# crsctl stop crs

Repeat this command on each node in your Oracle RAC cluster.

4. As the root user, restore the OCR by applying an OCR backup file that you identified in step 1 using the following command, where file_name is the name of the OCR that you want to restore. Make sure that the OCR devices that you specify in the OCR configuration exist, and that these OCR devices are valid before running this command.

# ocrconfig -restore file_name

5. As the root user, restart Oracle Clusterware on all the nodes in your cluster by restarting each node, or by running the following command:

# crsctl start crs

Repeat this command on each node in your Oracle RAC cluster.

6. Use the Cluster Verify Utility (CVU) to verify the OCR integrity. Run the following command, where the -n all argument retrieves a list of all the cluster nodes that are configured as part of your cluster:

$ cluvfy comp ocr -n all [-verbose]

Recovering the OCR from an OCR Export File

Using the ocrconfig -export command enables you to restore the OCR using the-import option if your configuration changes cause errors.

To restore the previous configuration stored in the OCR from an OCR export file:
1. Place the OCR export file that you created previously with the ocrconfig-export command in an accessible directory on disk.

2. As the root user, stop Oracle Clusterware on all the nodes in your Oracle RAC cluster by executing the following command:

crsctl stop crs

Repeat this command on each node in your Oracle RAC cluster.

3. As the root user, restore the OCR data by importing the contents of the OCR export file using the following command, where file_name is the name of the OCR export file:

ocrconfig -import file_name

4. As the root user, restart Oracle Clusterware on all the nodes in your cluster by restarting each node, or by running the following command:

crsctl start crs

Repeat this command on each node in your Oracle RAC cluster.

5. Use the CVU to verify the OCR integrity. Run the following command, where the -n all argument retrieves a list of all the cluster nodes that are configured as part of your cluster:

cluvfy comp ocr -n all [-verbose]