ORA-00245: control file backup failed; in Oracle RAC, target might not be on shared storage

ORA-00245: control file backup failed;
in Oracle RAC, target might not be on shared storage

RMAN Full database backup of 4 node RAC got failed, when trying to run autobackup of controlfile.

Problem :

Starting Control File and SPFILE Autobackup at 2015-09-14:23:13:44

RMAN-00571: ===========================================================

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================

RMAN-03009: failure of Control File and SPFILE Autobackup command on Disk channel at 2015-09-14:23:13:45

ORA-00245: control file backup failed; in Oracle RAC, target might not be on shared storage

Cause :

From 11gR2 onwards, the controlfile backup happens without holding the controlfile enqueue. For non-RAC database, this doesn't change anything. But for RAC database, due to the changes made to the controlfile backup mechanism in 11gR2, any instance in the cluster may write to the snapshot controlfile. Due to this snapshot controlfile need to be visible to all instances. 

The snapshot controlfile MUST be accessible by all nodes of a RAC database, if the snapshot controlfile does not reside on a shared device error will be raised at the time of RMAN backup while taking snapshot of controlfile. 

This applies to backing up controlfile using sqlplus / having autobackup of controlfile configured on non shared location.

ORA-245 error message description
----------------------------------------

00245, 00000, "control file backup operation failed"
*Cause: Failed to create a control file backup because some process signalled an error during backup creation.

*Action: Check alert files for further information. This usually happens because some process could not access the backup file during backup creation. Any process of any instance that starts a read/write control file transaction must have an access to the
backup control file during backup creation.

1.   In RAC environment controlfile autobackup fails with ora-0245

Autobackup of controlfile in RMAN is failing with error:
RMAN-571: ===========================================================
RMAN-569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-571: ===========================================================
RMAN-3009: failure of Control File and SPFILE Autobackup command on 
ORA_DISK_1 channel at 08/14/2015 23:10:00
ORA-245: control file backup operation failed

2.   In RAC environment, backup controlfile to non shared location fails

SQL> ALTER DATABASE BACKUP CONTROLFILE TO '/backup/RMAN/KRISHNA/control_14Aug2015.bkp' REUSE
*
ERROR at line 1:
ORA-245: control file backup operation failed

3.   In RAC environment backing up standby controlfile to non shared location fails

SQL> alter database create standby controlfile as '/backup/RMAN/KRISHNA/STBY_Control.ctl';

alter database create standby controlfile as 
'/backup/RMAN/KRISHNA/STBY_Control.ctl'
*
ERROR at line 1:
ORA-245: control file backup operation failed

Solution

This is a RAC specific 'configuration' issue and the correct configuration is as described below

It is changed behaviour which requires that the snapshot controlfile in a RAC environment, is on a shared location. 

1.    Check the snapshot controlfile location: 

RMAN> show snapshot controlfile name;

2.    Configure the snapshot controlfile to a shared disk like (ACFS/NFS/DNFS):

RMAN> CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/SHARED_ACFS1/snapcf_KRISHNA.f';

3.    In case of ASM use

RMAN> CONFIGURE SNAPSHOT CONTROLFILE NAME TO '+DG_DATA/snapcf_KRISHNA.f';