Saturday, March 26, 2016

Failed to record pid for GPNPD / CRS-2302:Cannot get GPnP profile.

Failed to record pid for GPNPD
CRS-2302:Cannot get GPnP profile.
Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).

After restart Oracle Grid Infrastructure version 11.2.0.1/later or after rebooting a cluster node, the following error is encountered:

Error

[client(20896)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2016-03-27 00:03:26.114
[client(20896)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/app/11.2.0.3/grid/log/rac122/client/crsctl_oracle.log.



From gpnp.log

CLSGPNP_NO_PROFILE. GPnP profile 'profile.xml' does not exist in '/grid/app/11.2.0.3/grid/gpnp/rac122/profiles/peer/'

Grid Home : alertrac122.log

[gpnpd(20759)]CRS-2329:GPNPD on node rac122 shutdown.
2016-03-27 00:03:26.112
[client(20896)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2016-03-27 00:03:26.114
[client(20896)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/app/11.2.0.3/grid/log/rac122/client/crsctl_oracle.log.
2016-03-27 00:03:27.714
[client(20958)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2016-03-27 00:03:27.715
[client(20958)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/app/11.2.0.3/grid/log/rac122/client/crsctl_oracle.log.
2016-03-27 00:03:28.203
[client(20970)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2016-03-27 00:03:28.204
[client(20970)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/app/11.2.0.3/grid/log/rac122/client/crsctl_oracle.log.
2016-03-27 00:04:10.023
[/grid/app/11.2.0.3/grid/bin/oraagent.bin(19737)]CRS-5818:Aborted command 'start' for resource 'ora.gpnpd'. Details at (:CRSAGF00113:) {0:0:2} in /grid/app/11.2.0.3/grid/log/rac122/agent/ohasd/oraagent_oracle/oraagent_oracle.log.
2016-03-27 00:04:14.025
[ohasd(19619)]CRS-2757:Command 'Start' timed out waiting for response from the resource 'ora.gpnpd'. Details at (:CRSPE00111:) {0:0:2} in /grid/app/11.2.0.3/grid/log/rac122/ohasd/ohasd.log.
2016-03-27 00:04:14.416

gpnpd.log

[oracle@rac122 gpnpd]$ more /grid/app/11.2.0.3/grid/log/rac122/gpnpd/gpnpd.log

================================================================================
2016-03-27 00:06:19.082: [ default][3804149536]gpnpd START pid=21149 Oracle Grid Plug-and-Play Daemon
2016-03-27 00:06:19.082: [    GPNP][3804149536]clsgpnp_Init: [at clsgpnp0.c:585] '/grid/app/11.2.0.3/grid' in effect as GPnP home base.
2016-03-27 00:06:19.082: [    GPNP][3804149536]clsgpnp_Init: [at clsgpnp0.c:619] GPnP pid=21149, GPNP comp tracelevel=1, depcomp tracelevel=0, tlsrc:ORA_DAEMON_LOGGING_LEVELS, apitl:0, complog:1, tstenv:0, devenv:0, envopt:0, flags=3
2016-03-27 00:06:19.086: [    GPNP][3804149536]clsgpnpkwf_initwfloc: [at clsgpnpkwf.c:399] Using FS Wallet Location : /grid/app/11.2.0.3/grid/gpnp/rac122/wallets/peer/

[   CLWAL][3804149536]clsw_Initialize: OLR initlevel [70000]
[  clsdmt][3792512768]Listening to (ADDRESS=(PROTOCOL=ipc)(KEY=rac122DBG_GPNPD))
2016-03-27 00:06:19.098: [  clsdmt][3792512768]PID for the Process [21149], connkey 10
2016-03-27 00:06:19.098: [  clsdmt][3792512768]Creating PID [21149] file for home /grid/app/11.2.0.3/grid host rac122 bin gpnp to /grid/app/11.2.0.3/grid/gpnp/init/
2016-03-27 00:06:19.098: [  clsdmt][3792512768]Writing PID [21149] to the file [/grid/app/11.2.0.3/grid/gpnp/init/rac122.pid]
2016-03-27 00:06:19.098: [  clsdmt][3792512768]Failed to record pid for GPNPD
2016-03-27 00:06:19.098: [  clsdmt][3792512768]Terminating process
2016-03-27 00:06:19.098: [    GPNP][3792512768]CLSDM requested exit
2016-03-27 00:06:19.098: [ default][3792512768]GPNPD on node rac122 shutdown.



CAUSE :

[oracle@rac122 gpnpd]$ cd /grid/app/11.2.0.3/grid/gpnp/init/
[oracle@rac122 init]$ ls -ltrah
total 12K
drwxr-x--- 6 oracle oinstall 4.0K Mar 26 16:01 ..
-rw-r--r-- 1 oracle oinstall    0 Mar 26 16:01 rac122
drwxr-x--- 2 oracle oinstall 4.0K Mar 26 16:01 .
-rw-r--r-- 1 root   root        6 Mar 26 22:52 rac122.pid

File permission has been changed to root:root. Where this should have oracle:oinstall.


SOLUTION

To fix the issue, we have to move the rac122.pid to some different name  or correct the permission of files or create a new blank file with same name.


[oracle@rac122 init]$ cd /grid/app/11.2.0.3/grid/gpnp/init
[oracle@rac122 init]$ ls -l

-rw-r--r-- 1 oracle oinstall 0 Mar 26 16:01 rac122
-rw-r--r-- 1 oracle oinstall 6 Mar 26 23:10 rac122.pid

[oracle@rac122 init]$ mv rac122.pid rac122.pid_bkp

[oracle@rac122 init]$ touch rac122.pid

[oracle@rac122 init]$ cd /grid/app/11.2.0.3/grid/gpnp/init
[oracle@rac122 init]$ ls -l

-rw-r--r-- 1 oracle oinstall 0 Mar 26 16:01 rac122
-rw-r--r-- 1 oracle oinstall 6 Mar 27 00:32 rac122.pid
-rw-r--r-- 1 root   root     6 Mar 27 00:32 rac122.pid_bkp

Now stopping the cluster services on node rac122.

[oracle@rac122 init]$ sudo $ORACLE_HOME/bin/crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac122'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac122'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac122'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac122' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac122' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac122' has completed
CRS-4133: Oracle High Availability Services has been stopped.

Start the cluster services on node rac122

[oracle@rac122 init]$ sudo $ORACLE_HOME/bin/crsctl start crs
CRS-4123: Oracle High Availability Services has been started.

Validate the cluster services on node rac122

[oracle@rac122 init]$ crsctl check cluster
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online


NOTE:

This can happen to other daemons like crsd,ctssd,...

$GRID_HOME/<daemon>/init/<node_name>
$GRID_HOME/<daemon>/init/<node_name>.pid



References :

Unable To Start CRS 11gR2 Due to Missing Files in $GRID_HOME/<daemon>/init Directory (Doc ID 1146055.1)



Hope this help :)

No comments:

Post a Comment