Tags

, , ,

I wasn’t able to promote database targets after I applied the July bundle patch.  I quickly opened an SR and received the following excellent, if scary, advice.   The solution, as you’ll see, involves directly manipulating data in the SYSMAN schema.

The Error Message

“Plugin versions on agent https://xyz123.demo.org:1874/emd/main/ does not support target type rac_database”.

In this case the agents appeared to have a later release of the database plugin than the management server.   A quick check of the OMS and agents through the EM console showed that I was fully deployed with 12.0.8.  Everything’s fine, it just doesn’t work.

Evaluating the Problem

It seems my production OEM environment had entered some sort of quantum state, but I’m confident quantum computing wasn’t in this bundle patch.  Something must be seriously wrong.  That’s where Oracle Support came to the rescue.

The SYSMAN schema contains all sort of data and database objects behind the OEM application, including a detailed listing of acceptable plugin ranges – the min and max version your system will allow you to manage.

select * from em_target_types where type_resource_bundle like 'oracle.sysman.db.rsc';

sr313275776401_01

There’s my problem.  The max plugin version for RAC database was set to 12.1.0.7!

Action Plan (Thanks to Oracle Support)

0. Take a complete backup of repos DB  [ I chose to do a guaranteed restore point ].  The fix involves recompiling a hand-edited version of three packaged procedures, their dependent SQL, and several other objects.  Backup your OMR.

1. Navigate to <MW_HOME>/plugins/oracle.sysman.db.oms.plugin_12.1.0.8.0/sql/db_mds/12.1.0.8.0/targetType/ path on the OMS server.

i.e. ls -lrt has_data_upgrade_*

ls -lrt cluster_data_upgrade_

ls -lrt rac_database_data_upgrade_*

2. Find the last data upgrade sql file for has, cluster and rac_database.  In this case there were six SQL files for each plugin, so I edited has_data_upgrade_6.sql, etc.

 

3. Make a back up those each of those files.  We’ll edit and run these copies.

i.e. cp cluster_data_upgrade_6.sql cluster_data_upgrade_6_bkp.sql

4. Edit each backup file from step 3.  The two edits per file are shown in red below:  correct the plugin version ID and add a commit statement at the bottom of each file.

 DECLARE
 l_type_property_list SMP_EMD_NVPAIR_ARRAY;
 l_type_property SMP_EMD_NVPAIR;
 BEGIN

l_type_property_list := SMP_EMD_NVPAIR_ARRAY();
 l_type_property := SMP_EMD_NVPAIR( 'inherit_cred_metadata',
 '1');
 l_type_property_list.extend(1);
 l_type_property_list(l_type_property_list.count) :=
 l_type_property;
 l_type_property := SMP_EMD_NVPAIR(
 'inherit_cred_target_type',
 'oracle_database');
 l_type_property_list.extend(1);
 l_type_property_list(l_type_property_list.count) :=
 l_type_property;
 l_type_property := SMP_EMD_NVPAIR( 'use_parent_creds',
 '1');
 l_type_property_list.extend(1);
 l_type_property_list(l_type_property_list.count) :=
 l_type_property;
 l_type_property := SMP_EMD_NVPAIR( 'TargetVersion',
 'Version');
 l_type_property_list.extend(1);
 l_type_property_list(l_type_property_list.count) :=
 l_type_property;

mgmt_target.register_target_type(
 p_target_type_in => 'oracle_pdb',
 p_type_meta_ver_in => '1.4',
 p_parent_target_type_in => NULL,
 p_parent_type_meta_ver_in => NULL,
 p_type_property_list_in => l_type_property_list,
 p_type_display_name_in => 'Pluggable Database',
 p_type_display_nlsid_in => 'oracle_pdb_title',
 p_type_resource_bundle_in => 'oracle.sysman.db.rsc',
 p_monitored_by => '2',
 p_plugin_id => 'oracle.sysman.db',
 p_plugin_version_id => '12.1.0.8.0', 
 p_category_prop_defs => NULL,
 p_ocm_gc_merged => '1',
 p_type_relationship => 1,
 p_client => 'OTHERS',
 p_plugin_min_version_id => '12.1.0.1.0',
 p_product_id => NULL);
END;
.. lots of other stuff
 commit; 
 /

5. Run back up scripts from step 4 on the repository database as the sysman user:

@<MW_HOME>/plugins/oracle.sysman.db.oms.plugin_12.1.0.8.0/sql/db_mds/12.1.0.8.0/targetType/cluster_data_upgrade_6_bkp.sql

6. Running the following query on the repository database as the sysman user, verify that the Cluster, Cluster Database and Oracle High Availability Service targets MAX_PLUGIN_VERSION is 12.1.0.8.0:

 select * from em_target_types where type_resource_bundle like 'oracle.sysman.db.rsc';