Agent and OMS Rollback from 13c OEM to 12c OEM
SCENARIO
Scenario
which we are using is we have upgraded the OEM13c to new environment using
direct upgrade method. After some time
we feel the current OMS13c version is not stable and we are experience some
issue which we never faced on 12cR5. So
we decided to complete rollback to existing 12cR5 environment.
Here
we are using old OMR backup for repository restoration and for OMS we use the
backup configuration file for oms12c to roll back the changes.
APPROACH
·
Here we are
using RMAN to restore the approach.
· We need to
restore the OMR backup which we have taken before 13c upgrade. Configure listener, TNS alias and services as
per our existing setup.
· Once we have
restored the existing OMR backup, it is having all configuration related to OMS
like emkey…. etc. So we only need to
install the OMS 12cR5 binary and recover the oms configuration via OMSCA to
achieve the goal.
· After that we
just need to re-point the centralized agent and deployed target agent on all
server.
In this scenario we are using OMR and OMS on same
server. We can change the repository
server if we want.
Restoring
Repository Database from Backup, we and choose any below approach as per our
convenience.
1. RMAN Backup restoration and recovery
2. Export
and Import
3. Data
Guard with broker
4. Cold
Backup
OMS
Movement and Configuration Agent to New Host
Repository Configuration:
In new server we have
restore the OMR repository database and configure the listener and tns
alias/services.
After OMR repository configuration we
need to follow below approach to get the plugins.
Connected to:
Oracle Database 12c Enterprise Edition Release
12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced
Analytics and Real Application Testing options
SQL> conn sysman/********
col url for a40
col display_name for a35
col plugin_id for a20
col version for a20
set lin200 pages 200
SELECT
epv.display_name, epv.plugin_id, epv.version,
epv.rev_version,decode(su.aru_file, null, 'Media/External',
'https://updates.oracle.com/Orion/Services/download/'||aru_file||'?aru='||aru_id||chr(38)||'patch_file='||aru_file)
URL
FROM
EM_PLUGIN_VERSION epv, EM_CURRENT_DEPLOYED_PLUGIN ecp, EM_SU_ENTITIES su
WHERE
epv.plugin_type NOT IN ('BUILT_IN_TARGET_TYPE', 'INSTALL_HOME')
AND
ecp.dest_type='2'
AND
epv.plugin_version_id = ecp.plugin_version_id
AND
su.entity_id = epv.su_entity_id ;
DISPLAY_NAME
PLUGIN_ID
VERSION REV_VERSION URL
-------------------------------
-------------------- ----------- ----------- --------------
Oracle MOS (My Oracle Support)
oracle.sysman.mos 12.1.0.6.0
0 Media/External
Oracle Cloud
Framework
oracle.sysman.cfw 12.1.0.3.0
0 Media/External
Oracle Fusion
Middleware
oracle.sysman.emas 12.1.0.8.0
0 Media/External
Oracle
Database
oracle.sysman.db 12.1.0.8.0
0 Media/External
Oracle
Exadata
oracle.sysman.xa 12.1.0.6.0
0 Media/External
SQL> Disconnected from Oracle
Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced
Analytics and Real Application Testing options
Installed the OEM12cR5
in Software only approach.
Before
installation I have create the same directory structure on new server.
Unzip the OEM-12c-R5 and then run ./runInstaller
Set the Inventory Location
Verify the prerequisites
Here we are choosing Install Software Only as we have configure using old oms export configuration file.
Setting the Middleware home and Agent Base directory
Review and install ....
Run Root.sh to continue the OMS Software installation.
[oracle@oem251 OEM_12c_R5]$ sudo
/u01/app/oraInventory/orainstRoot.sh
Changing permissions of
/u01/app/oraInventory
Adding read,write permissions for
group,Removing read,write,execute permissions for world.
Changing groupname of
/u01/app/oraInventory to oinstall.
The execution of the script is complete
[oracle@oem251 OEM_12c_R5]$ sudo
/u01/app/oracle/Middleware12105/oms/allroot.sh
Starting to execute allroot.sh
.........
Starting to execute
/u01/app/oracle/Middleware12105/oms/root.sh ......
Running Oracle 11g root.sh script...
The following environment variables are
set as:
ORACLE_OWNER=
oracle
ORACLE_HOME= /u01/app/oracle/Middleware12105/oms
Enter the full pathname of the local
bin directory: [/usr/local/bin]:
Copying dbhome to
/usr/local/bin ...
Copying oraenv to
/usr/local/bin ...
Copying coraenv to
/usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the
/etc/oratab file as needed by
Database Configuration Assistant when a
database is created
Finished running generic part of
root.sh script.
Now product-specific root actions will
be performed.
/etc exist
Creating /etc/oragchomelist file...
/u01/app/oracle/Middleware12105/oms
Finished execution
of /u01/app/oracle/Middleware12105/oms/root.sh ......
Starting to execute
/u01/app/oracle/gc_inst1/core/12.1.0.5.0/root.sh ......
Finished product-specific root actions.
/etc exist
/u01/app/oracle/gc_inst1/core/12.1.0.5.0
Finished execution of /u01/app/oracle/gc_inst1/core/12.1.0.5.0/root.sh
......
[oracle@oem251 OEM_12c_R5]$
Click
OK.
Now run the PlugginInstall.sh under sysman/install directory.
Choose as per existing configuration and copy the existing software
library to OMS server.
col display_name for a35
col plugin_id for a20
col version for a20
set lin200 pages 200
SELECT epv.display_name, epv.plugin_id,
epv.version, epv.rev_version,decode(su.aru_file, null, 'Media/External',
'https://updates.oracle.com/Orion/Services/download/'||aru_file||'?aru='||aru_id||chr(38)||'patch_file='||aru_file)
URL
FROM EM_PLUGIN_VERSION epv,
EM_CURRENT_DEPLOYED_PLUGIN ecp, EM_SU_ENTITIES su
WHERE epv.plugin_type NOT IN
('BUILT_IN_TARGET_TYPE', 'INSTALL_HOME')
AND ecp.dest_type='2'
AND epv.plugin_version_id =
ecp.plugin_version_id
AND su.entity_id =
epv.su_entity_id ;
DISPLAY_NAME
PLUGIN_ID
VERSION REV_VERSION URL
-------------------------------
-------------------- ----------- ----------- --------------
Oracle MOS (My Oracle Support)
oracle.sysman.mos 12.1.0.6.0
0 Media/External
Oracle Cloud
Framework
oracle.sysman.cfw 12.1.0.3.0
0 Media/External
Oracle Fusion
Middleware
oracle.sysman.emas 12.1.0.8.0 0
Media/External
Oracle
Database
oracle.sysman.db 12.1.0.8.0
0 Media/External
Oracle
Exadata
oracle.sysman.xa 12.1.0.6.0
0 Media/External
SQL> Disconnected from Oracle
Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced
Analytics and Real Application Testing options
From above query we have list of
installed plugins. Now it's high time to
installed plugins into new host.
Now run the PlugginInstall.sh under sysman/install directory.
Choose as per existing configuration.
Continue the plugins installation
Software installation has completed. Now it's time to configure new OMS.
Copy the OMS Configuration backup file to this
server.
[oracle@oem251 ~]$ cp
/nfs/BACKUP/REPDB_BACKUP/opf_ADMIN_20160303_105032.bka /backup/opf_ADMIN_20160303_105032.bka
Recreate the OMS with OMSCA
Shutdown everything on your old 13c.
Recovering/recreating OMS using backup
configuration via omsca
oms:/u02/app/oracle/Middleware/oms:N
REPDB:/u01/app/oracle/product/12.1.0.2/DB_1:N
[oracle@oem251 ~]$ omsca recover -as -ms
-nostart -backup_file /nfs/BACKUP/REPDB_BACKUP/opf_ADMIN_20160303_105032.bka
Oracle Enterprise Manager Cloud Control
12c Release 12.1.0.5.0
Copyright (c) 1996, 2015, Oracle. All
rights reserved.
OS check passed.
OMS version check passed.
Performing Admin Server Recovery...
Retrieved Admin Server template.
Source Instance Host name where
configuration is exported : oem254.ora.com
instHost name retrieved from local host
name : oem251.ora.com
Populated install params from backup...
Enter Administration Server user
password:
Confirm Password:
Enter Node Manager Password:
Confirm Password:
Enter Repository database user
password:
Enter Agent Registration password:
Confirm Password:
Doing pre requisite checks ......
Pre requisite checks completed
successfully
Checking Plugin software bits
Proceed to recovery
Setting up domain from template...
Setup EM infrastructure succeeded!
Admin Server recovered from backup.
Now performing cleanup of OMS
EMGC_OMS1...
Now launching DeleteOMS...
OMS Deleted successfully
Delete finished successfully
Now launching AddOMS...
Infrastructure setup of EM completed
successfully.
Doing pre deployment operations ......
Pre deployment of EM completed
successfully.
Deploying EM ......
Deployment of EM completed
successfully.
Configuring webtier ......
Configuring webTier completed
successfully.
Importing OMS configuration from
recovery file...
If you have software library configured
please make sure it is functional and
accessible
from this OMS by visiting:
Setup->Provisioning and
Patching->Software Library
Securing OMS ......
EM Key is secured and is backed up at
/u02/app/oracle/Middleware/oms/sysman/config/emkey.ora
Adapter already exists: emgc_USER
Adapter created successfully:
emgc_GROUP
Post "Deploy and Repos Setup"
operations completed successfully.
Performing Post deploy operations ....
Total 0 errors, 81 warnings. 0 entities
imported.
pluginID:oracle.sysman.core
Done with csg import
pluginID:oracle.sysman.core
Done with csg import
No logging has been configured and
default agent logging support is unavailable.
Post deploy operations completed
successfully.
EM configuration completed
successfully.
EM URL is:https://oem251.ora.com:7802/em
Add OMS finished successfully
Recovery of server EMGC_OMS1 completed
successfully
OMSCA Recover completed successfully
GET THE EM_UPLOAD_PORT
Get the status of oms to know the EM_UPLOAD port.
[oracle@oem251 ~]$ . oraenv
ORACLE_SID = [oracle] ? oms
Oracle Enterprise Manager Cloud Control
12c Release 5
Copyright (c) 1996, 2015 Oracle
Corporation. All rights reserved.
Enter Enterprise Manager Root (SYSMAN)
Password :
Console Server
Host : oem251.ora.com
HTTP Console
Port : 7788
HTTPS Console
Port : 7802
HTTP Upload
Port : 4889
HTTPS Upload
Port : 4903
EM Instance Home
: /u02/app/oracle/gc_inst/em/EMGC_OMS1
OMS Log Directory Location :
/u02/app/oracle/gc_inst/em/EMGC_OMS1/sysman/log
OMS is not configured with SLB or
virtual hostname
Agent Upload is locked.
OMS Console is locked.
Active CA ID: 1
Console URL: https://oem251.ora.com:7802/em
Upload URL: https://oem251.ora.com:4903/empbs/upload
WLS Domain Information
Domain
Name :
GCDomain
Admin Server
Host : oem251.ora.com
Admin Server HTTPS Port: 7102
Admin Server is RUNNING
Oracle Management Server Information
Managed Server Instance Name: EMGC_OMS1
Oracle Management Server Instance Host:
oem251.ora.com
WebTier is Up
Oracle Management Server is Up
BI Publisher is not configured to run
on this host.
Configure
Central Agent on New Host
[oracle@oem251 agent_inst]$
/u02/app/oracle/Agent12c/core/12.1.0.5.0/sysman/install/agentDeploy.sh
AGENT_BASE_DIR=/u02/app/oracle/Agent12c
AGENT_INSTANCE_HOME=/u02/app/oracle/Agent12c/agent_inst AGENT_PORT=3872
-configOnly OMS_HOST=oem251.ora.com EM_UPLOAD_PORT=4903
AGENT_REGISTRATION_PASSWORD=********
Executing command :
/u02/app/oracle/Agent12c/core/12.1.0.5.0/jdk/bin/java -classpath
/u02/app/oracle/Agent12c/core/12.1.0.5.0/oui/jlib/OraInstaller.jar:/u02/app/oracle/Agent12c/core/12.1.0.5.0/oui/jlib/xmlparserv2.jar:/u02/app/oracle/Agent12c/core/12.1.0.5.0/oui/jlib/srvm.jar:/u02/app/oracle/Agent12c/core/12.1.0.5.0/oui/jlib/emCfg.jar:/u02/app/oracle/Agent12c/core/12.1.0.5.0/jlib/agentInstaller.jar:/u02/app/oracle/Agent12c/core/12.1.0.5.0/oui/jlib/share.jar
oracle.sysman.agent.installer.AgentInstaller
/u02/app/oracle/Agent12c/core/12.1.0.5.0
/u02/app/oracle/Agent12c/core/12.1.0.5.0/sysman/install
/u02/app/oracle/Agent12c /u02/app/oracle/Agent12c/agent_inst
AGENT_BASE_DIR=/u02/app/oracle/Agent12c
Executing command:
/u02/app/oracle/Agent12c/core/12.1.0.5.0/oui/bin/runConfig.sh
ORACLE_HOME=/u02/app/oracle/Agent12c/core/12.1.0.5.0 RESPONSE_FILE=/u02/app/oracle/Agent12c/core/12.1.0.5.0/agent.rsp
ACTION=configure MODE=perform
COMPONENT_XML={oracle.sysman.top.agent.11_1_0_1_0.xml} RERUN=true
Agent Configuration completed
successfully
The following configuration scripts
need to be executed as the "root" user.
#!/bin/sh
#Root script to run
/u02/app/oracle/Agent12c/core/12.1.0.5.0/root.sh
To execute the configuration scripts:
1. Open a terminal window
2. Log in as "root"
3. Run the scripts
Agent Deployment Successful.
Run the root.sh script to complete the central
agent creation part.
[oracle@oem251 agent_inst]$ sudo
/u02/app/oracle/Agent12c/core/12.1.0.5.0/root.sh
Finished product-specific root actions.
/etc exist
Login to oms using emcli and sync the repository
[oracle@oem251 agent_inst]$ emcli login
-username=sysman -password=********
Login successful
[oracle@oem251 agent_inst]$ emcli sync
Synchronized successfully
Relocate the repository target to the new OMS host.
[oracle@oem251 agent_inst]$ emctl config
emrep -agent oem251.ora.com:3872
Oracle Enterprise Manager Cloud Control
12c Release 5
Copyright (c) 1996, 2015 Oracle
Corporation. All rights reserved.
Please enter repository
password: Enter password :
Login successful
Moved all targets from
oem254.ora.com:3872 to oem251.ora.com:3872
Command completed
successfully! Enter password :
Login successful
Moved all targets from
oem254.ora.com:3872 to oem251.ora.com:3872
Command completed successfully!
Repointing/Reconfiguring
(Agent) deployed target from old host to new host
Status before re-pointing to new OMS host
[oracle@vm212 bin]$ ./emctl status agent
Oracle Enterprise Manager Cloud Control
12c Release 5
Copyright (c) 1996, 2015 Oracle
Corporation. All rights reserved.
---------------------------------------------------------------
Agent
Version : 12.1.0.5.0
OMS
Version :
(unknown)
Protocol
Version : 12.1.0.1.0
Agent
Home :
/u01/app/oracle/eu/agent_inst
Agent Log Directory :
/u01/app/oracle/eu/agent_inst/sysman/log
Agent
Binaries :
/u01/app/oracle/eu/core/12.1.0.5.0
Agent Process
ID : 21126
Parent Process
ID : 21074
Agent
URL
: https://vm212.ora.com:3892/emd/main/
Local Agent URL in NAT :
https://vm212.ora.com:3892/emd/main/
Repository
URL :
https://oem254.ora.com:4903/empbs/upload
Started
at :
2016-03-03 17:45:33
Started by
user : oracle
Operating
System : Linux version
2.6.39-400.209.1.el5uek (amd64)
Last
Reload :
(none)
Last successful
upload
: (none)
Last attempted
upload
: (none)
Total Megabytes of XML files uploaded
so far : 0
Number of XML files pending
upload : 57
Size of XML files pending
upload(MB) : 0.09
Available disk space on upload
filesystem : 53.27%
Collection
Status
: Collections enabled
Heartbeat
Status
: OMS is unreachable
Last attempted heartbeat to
OMS
: 2016-03-03 18:08:01
Last successful heartbeat to
OMS :
(none)
Next scheduled heartbeat to
OMS
: 2016-03-03 18:08:31
---------------------------------------------------------------
Agent is Running and Ready
Reconfigure existing agents to re-secure them
against the new OMS. Execute below command and enter the agent
registration password when prompted
[oracle@vm212 bin]$ ./emctl secure
agent -emdWalletSrcUrl "https://oem251.ora.com:4903/em"
Oracle Enterprise Manager Cloud Control
12c Release 5
Copyright (c) 1996, 2015 Oracle
Corporation. All rights reserved.
Agent successfully
stopped... Done.
Securing agent... Started.
Enter Agent Registration Password :
Agent successfully
restarted... Done.
Securing agent...
Successful.
Step through each of your existing agents to
re-secure them against the new OMS. If you have large environment then
this can be done via shall script in one go.
Verifying the agent status after the re-pointing to
new host.
[oracle@vm212 bin]$ ./emctl status agent
Oracle Enterprise Manager Cloud Control
12c Release 5
Copyright (c) 1996, 2015 Oracle
Corporation. All rights reserved.
---------------------------------------------------------------
Agent Version
: 12.1.0.5.0
OMS
Version :
12.1.0.5.0
Protocol
Version : 12.1.0.1.0
Agent
Home :
/u01/app/oracle/eu/agent_inst
Agent Log Directory :
/u01/app/oracle/eu/agent_inst/sysman/log
Agent
Binaries : /u01/app/oracle/eu/core/12.1.0.5.0
Agent Process
ID : 25215
Parent Process
ID : 25166
Agent
URL
: https://vm212.ora.com:3892/emd/main/
Local Agent URL in NAT :
https://vm212.ora.com:3892/emd/main/
Repository
URL : https://oem251.ora.com:4903/empbs/upload
Started
at :
2016-03-03 18:18:46
Started by
user : oracle
Operating
System : Linux version
2.6.39-400.209.1.el5uek (amd64)
Last
Reload :
(none)
Last successful
upload
: 2016-03-03 18:19:56
Last attempted
upload
: 2016-03-03 18:19:56
Total Megabytes of XML files uploaded
so far : 0.14
Number of XML files pending
upload : 0
Size of XML files pending
upload(MB) : 0
Available disk space on upload
filesystem : 53.27%
Collection
Status
: Collections enabled
Heartbeat
Status
: Ok
Last attempted heartbeat to
OMS
: 2016-03-03 18:19:49
Last successful heartbeat to OMS
: 2016-03-03 18:19:49
Next scheduled heartbeat to
OMS
: 2016-03-03 18:20:51
---------------------------------------------------------------
Agent is Running and Ready
Finally its looks good. Now agent is pointing to
new host. Also verify the details on EM web console.
In the below snap you will see the updated
details. Few of them need to re-point.
Also verify the migrated Administration group from
OLD OMS to NEW OMS.
Hope this help ... :)
