Install Oracle Grid Infrastructure 12.1.0.1++ on Fedora 17 – 20


In my previous post i’vh described how to set up the Oracle 12c Database Software to get working on Fedora 17 up to 20. Today i’m going to to show how to setup a RAC Cluster (12.1.0.1) on Fedora 17++ (I’vh testet on Fedora 20). My enviroment: 2x VirtualBox, Fedora 20 x86_64, with 4 CPUs, 4092 MB RAM, 50GB local, 10GB shared disks, 2x NIC Paravirtualized

1. Prerequirements

First start follow the Oracle Database 12c Quick Installation Guide to setup the base for the Grid Infrastructure (see here)
During the setup of the inital stuff you should mention something like that:

Loaded plugins: langpacks, refresh-packagekit
No package libcap1 available.
Error: Nothing to do

This problem doesn’t matter, because there is a newer working version:

[root@fed01 ~]# yum list libcap
Loaded plugins: langpacks, refresh-packagekit
Installed Packages
libcap.x86_64                                             2.22-7.fc20                                             @anaconda
Available Packages
libcap.i686                                               2.22-7.fc20                                             fedora

To avoid an exception like this:

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /opt/oracle/12.1.0.1/grid/crs/install/crsconfig_params
/opt/oracle/12.1.0.1/grid/bin/crsctl query crs releaseversion ... failed rc=127 with message:
 /opt/oracle/12.1.0.1/grid/bin/crsctl.bin: error while loading shared libraries: libcap.so.1: cannot open shared object file: No such file or directory 

Failed to create keys in the OLR, rc = 127, Message:
  /opt/oracle/12.1.0.1/grid/bin/clscfg.bin: error while loading shared libraries: libcap.so.1: cannot open shared object file: No such file or directory 

2014/02/28 00:13:19 CLSRSC-188: Failed to create keys in Oracle Local Registry

Died at /opt/oracle/12.1.0.1/grid/crs/install/crsutils.pm line 6479.
The command '/opt/oracle/12.1.0.1/grid/perl/bin/perl -I/opt/oracle/12.1.0.1/grid/perl/lib -I/opt/oracle/12.1.0.1/grid/crs/install /opt/oracle/12.1.0.1/grid/crs/install/rootcrs.pl ' execution failed

… you have to relink the existing library …

[root@fed01 install]# cd /usr/lib64/
[root@fed01 lib64]# ln -s /usr/lib64/libcap.so.2 /usr/lib64/libcap.so.1

Futher i’vh disabled the firewall to avoid network problems during my tests

systemctl disable firewalld.service

2. Unpack and install the software

There will be nothing special. During linking process you will get the following error:

fedora20_12c_rac

To workaround this errors please follow my previous post. (see here)

You should be able to complete the installation up to the root.sh successfully.

3. root.sh

If you are going to execute the root.sh right now, you will get the following problem:

[root@fed01 lib64]# /opt/oracle/12.1.0.1/grid/root.sh 
Performing root user operation for Oracle 12c 

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /opt/oracle/12.1.0.1/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:  
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /opt/oracle/12.1.0.1/grid/crs/install/crsconfig_params
OLR initialization - successful
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
2014/02/28 00:16:34 CLSRSC-330: Adding Clusterware entries to file '/etc/inittab'

2014/02/28 00:18:40 CLSRSC-214: Failed to start 'ohasd'

Failed to start the Clusterware. Last 20 lines of the alert log follow: 
2014-02-28 00:13:19.635: 
[client(8796)]CRS-2101:The OLR was formatted using version 4.
2014-02-28 00:15:11.647: 
[client(9189)]CRS-2101:The OLR was formatted using version 4.

CRS-4133: Oracle High Availability Services has been stopped.
2014/02/28 00:31:53 CLSRSC-117: Failed to start Oracle Clusterware stack

Died at /opt/oracle/12.1.0.1/grid/crs/install/crsutils.pm line 8813.
The command '/opt/oracle/12.1.0.1/grid/perl/bin/perl -I/opt/oracle/12.1.0.1/grid/perl/lib -I/opt/oracle/12.1.0.1/grid/crs/install /opt/oracle/12.1.0.1/grid/crs/install/rootcrs.pl ' execution failed

Oracle’s Grid Infrastructure starts with the intitialization script:

[root@fed01 lib64]# cat /etc/inittab
# inittab is no longer used when using systemd.
#
# ADDING CONFIGURATION HERE WILL HAVE NO EFFECT ON YOUR SYSTEM.
#
# Ctrl-Alt-Delete is handled by /etc/systemd/system/ctrl-alt-del.target
#
# systemd uses 'targets' instead of runlevels. By default, there are two main targets:
#
# multi-user.target: analogous to runlevel 3
# graphical.target: analogous to runlevel 5
#
# To set a default target, run:
#
# ln -sf /lib/systemd/system/.target /etc/systemd/system/default.target
#
h1:35:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1

Fedora doesn’t use the old style and switched to systemd, therefore you have to setup the daemon with the new enviroment. Here my implementation

[root@fed01 lib64]# cat /etc/systemd/system/ohasd.service 
[Unit]
Description=ohasd daemon

[Service]
ExecStart=/etc/init.d/init.ohasd run >/dev/null 2>&1

Reload the dameon unit to inform systemd about my changes

systemctl daemon-reload

Now enable and start the daemon

systemctl enable ohasd.service
systemctl start ohasd.service

Check how it’s work?

[root@fed01 ~]# ps -ef | grep oha                                                                                                                     
root     13069     1  0 00:56 ?        00:00:00 /bin/sh /etc/init.d/init.ohasd run >/dev/null 2>&1

Now re execute the root.sh script and it should finalize successfully:

...
CRS-6016: Resource auto-start has completed for server fed01
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.

Preparing packages...
cvuqdisk-1.0.9-1.x86_64
2014/02/28 01:13:27 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

Here the cluster (first node) output:

[root@fed01 system]# crsctl stat res -t 
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       fed01                    STABLE
ora.DATA.dg
               ONLINE  ONLINE       fed01                    STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       fed01                    STABLE
ora.net1.network
               ONLINE  ONLINE       fed01                    STABLE
ora.ons
               ONLINE  ONLINE       fed01                    STABLE
ora.proxy_advm
               ONLINE  OFFLINE      fed01                    STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       fed01                    STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       fed01                    STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       fed01                    STABLE
ora.MGMTLSNR
      1        OFFLINE OFFLINE                               STABLE
ora.asm
      1        ONLINE  ONLINE       fed01                    STABLE
      2        OFFLINE OFFLINE                               STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       fed01                    STABLE
ora.fed01.vip
      1        ONLINE  ONLINE       fed01                    STABLE
ora.oc4j
      1        ONLINE  ONLINE       fed01                    STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       fed01                    STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       fed01                    STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       fed01                    STABLE
--------------------------------------------------------------------------------

4. Redo on all other nodes and complete setup

To complete your cluster setup redo all step above.

For completion the output from the second node (root.sh):

[root@fed02 lib64]# /opt/oracle/12.1.0.1/grid/root.sh 
Performing root user operation for Oracle 12c 

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /opt/oracle/12.1.0.1/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /opt/oracle/12.1.0.1/grid/crs/install/crsconfig_params

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'fed02'                                                                                                                                                                                                                                
CRS-2673: Attempting to stop 'ora.crsd' on 'fed02'                                                                                                                                                                                                                                                                           
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'fed02'                                                                                                                                                                                                                                           
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'fed02'                                                                                                                                                                                                                                                                        
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'fed02'                                                                                                                                                                                                                                                            
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'fed02'                                                                                                                                                                                                                                                            
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'fed02'                                                                                                                                                                                                                                                            
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'fed02' succeeded                                                                                                                                                                                                                                                             
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'fed02'                                                                                                                                                                                                                                                                      
CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'fed02' succeeded                                                                                                                                                                                                                                                             
CRS-2673: Attempting to stop 'ora.scan3.vip' on 'fed02'                                                                                                                                                                                                                                                                      
CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'fed02' succeeded                                                                                                                                                                                                                                                             
CRS-2673: Attempting to stop 'ora.scan2.vip' on 'fed02'                                                                                                                                                                                                                                                                      
CRS-2677: Stop of 'ora.scan1.vip' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'fed01'
CRS-2677: Stop of 'ora.scan3.vip' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.scan3.vip' on 'fed01'
CRS-2677: Stop of 'ora.scan2.vip' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.scan2.vip' on 'fed01'
CRS-2676: Start of 'ora.scan1.vip' on 'fed01' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'fed01'
CRS-2676: Start of 'ora.scan3.vip' on 'fed01' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on 'fed01'
CRS-2676: Start of 'ora.scan2.vip' on 'fed01' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'fed01'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'fed01' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on 'fed01' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'fed01' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'fed02' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'fed02'
CRS-2677: Stop of 'ora.asm' on 'fed02' succeeded
CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'fed02'
CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'fed02' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'fed02'
CRS-2677: Stop of 'ora.ons' on 'fed02' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'fed02'
CRS-2677: Stop of 'ora.net1.network' on 'fed02' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'fed02' has completed
CRS-2677: Stop of 'ora.crsd' on 'fed02' succeeded
CRS-2673: Attempting to stop 'ora.evmd' on 'fed02'
CRS-2673: Attempting to stop 'ora.storage' on 'fed02'
CRS-2673: Attempting to stop 'ora.crf' on 'fed02'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'fed02'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'fed02'
CRS-2677: Stop of 'ora.storage' on 'fed02' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'fed02'
CRS-2677: Stop of 'ora.crf' on 'fed02' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'fed02' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'fed02' succeeded
CRS-2677: Stop of 'ora.evmd' on 'fed02' succeeded
CRS-2677: Stop of 'ora.asm' on 'fed02' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'fed02'
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'fed02'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'fed02' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'fed02' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'fed02'
CRS-2677: Stop of 'ora.cssd' on 'fed02' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'fed02'
CRS-2677: Stop of 'ora.gipcd' on 'fed02' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'fed02' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'fed02'
CRS-2672: Attempting to start 'ora.evmd' on 'fed02'
CRS-2676: Start of 'ora.evmd' on 'fed02' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'fed02'
CRS-2676: Start of 'ora.gpnpd' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'fed02'
CRS-2676: Start of 'ora.gipcd' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'fed02'
CRS-2676: Start of 'ora.cssdmonitor' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'fed02'
CRS-2672: Attempting to start 'ora.diskmon' on 'fed02'
CRS-2676: Start of 'ora.diskmon' on 'fed02' succeeded
CRS-2789: Cannot stop resource 'ora.diskmon' as it is not running on server 'fed02'
CRS-2676: Start of 'ora.cssd' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'fed02'
CRS-2672: Attempting to start 'ora.ctssd' on 'fed02'
CRS-2676: Start of 'ora.ctssd' on 'fed02' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'fed02'
CRS-2676: Start of 'ora.asm' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'fed02'
CRS-2676: Start of 'ora.storage' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'fed02'
CRS-2676: Start of 'ora.crf' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'fed02'
CRS-2676: Start of 'ora.crsd' on 'fed02' succeeded
CRS-6017: Processing resource auto-start for servers: fed02
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'fed02'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'fed01'
CRS-2672: Attempting to start 'ora.ons' on 'fed02'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'fed01' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'fed01'
CRS-2677: Stop of 'ora.scan1.vip' on 'fed01' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'fed02'
CRS-2676: Start of 'ora.scan1.vip' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'fed02'
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'fed02'
CRS-2676: Start of 'ora.ons' on 'fed02' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'fed02' succeeded
CRS-2676: Start of 'ora.asm' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.proxy_advm' on 'fed01'
CRS-2679: Attempting to clean 'ora.proxy_advm' on 'fed02'
CRS-5017: The resource action "ora.proxy_advm start" encountered the following error: 
ORA-03113: end-of-file on communication channel
Process ID: 0
Session ID: 0 Serial number: 0
. For details refer to "(:CLSN00107:)" in "/opt/oracle/12.1.0.1/grid/log/fed01/agent/crsd/oraagent_oracle/oraagent_oracle.log".
CRS-2681: Clean of 'ora.proxy_advm' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.proxy_advm' on 'fed02'
CRS-2674: Start of 'ora.proxy_advm' on 'fed01' failed
CRS-2679: Attempting to clean 'ora.proxy_advm' on 'fed01'
CRS-5017: The resource action "ora.proxy_advm start" encountered the following error: 
ORA-03113: end-of-file on communication channel
Process ID: 0
Session ID: 0 Serial number: 0
. For details refer to "(:CLSN00107:)" in "/opt/oracle/12.1.0.1/grid/log/fed02/agent/crsd/oraagent_oracle/oraagent_oracle.log".
CRS-2674: Start of 'ora.proxy_advm' on 'fed02' failed
CRS-2679: Attempting to clean 'ora.proxy_advm' on 'fed02'
CRS-2681: Clean of 'ora.proxy_advm' on 'fed01' succeeded
CRS-2681: Clean of 'ora.proxy_advm' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.proxy_advm' on 'fed02'
CRS-2672: Attempting to start 'ora.DATA.dg' on 'fed02'
CRS-2672: Attempting to start 'ora.proxy_advm' on 'fed01'
CRS-2676: Start of 'ora.DATA.dg' on 'fed02' succeeded
CRS-5017: The resource action "ora.proxy_advm start" encountered the following error: 
ORA-03113: end-of-file on communication channel
Process ID: 0
Session ID: 0 Serial number: 0
. For details refer to "(:CLSN00107:)" in "/opt/oracle/12.1.0.1/grid/log/fed02/agent/crsd/oraagent_oracle/oraagent_oracle.log".
CRS-5017: The resource action "ora.proxy_advm start" encountered the following error: 
ORA-03113: end-of-file on communication channel
Process ID: 0
Session ID: 0 Serial number: 0
. For details refer to "(:CLSN00107:)" in "/opt/oracle/12.1.0.1/grid/log/fed01/agent/crsd/oraagent_oracle/oraagent_oracle.log".
CRS-2674: Start of 'ora.proxy_advm' on 'fed02' failed
CRS-2679: Attempting to clean 'ora.proxy_advm' on 'fed02'
CRS-2674: Start of 'ora.proxy_advm' on 'fed01' failed
CRS-2679: Attempting to clean 'ora.proxy_advm' on 'fed01'
CRS-2681: Clean of 'ora.proxy_advm' on 'fed02' succeeded
CRS-2672: Attempting to start 'ora.proxy_advm' on 'fed02'
===== Summary of resource auto-start failures follows =====
CRS-2807: Resource 'ora.proxy_advm' failed to start automatically.
CRS-6016: Resource auto-start has completed for server fed02
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2014/02/28 01:53:02 CLSRSC-343: Successfully started Oracle clusterware stack

Preparing packages...
cvuqdisk-1.0.9-1.x86_64
2014/02/28 01:53:18 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

The final state:

--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       fed01                    STABLE
               ONLINE  ONLINE       fed02                    STABLE
ora.DATA.dg
               ONLINE  ONLINE       fed01                    STABLE
               ONLINE  ONLINE       fed02                    STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       fed01                    STABLE
               ONLINE  ONLINE       fed02                    STABLE
ora.net1.network
               ONLINE  ONLINE       fed01                    STABLE
               ONLINE  ONLINE       fed02                    STABLE
ora.ons
               ONLINE  ONLINE       fed01                    STABLE
               ONLINE  ONLINE       fed02                    STABLE
ora.proxy_advm
               ONLINE  OFFLINE      fed01                    STABLE
               ONLINE  OFFLINE      fed02                    STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       fed02                    STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       fed01                    STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       fed01                    STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE                                STABLE
ora.asm
      1        ONLINE  ONLINE       fed01                    STABLE
      2        ONLINE  ONLINE       fed02                    STABLE
ora.cvu
      1        ONLINE  ONLINE       fed01                    STABLE
ora.fed01.vip
      1        ONLINE  ONLINE       fed01                    STABLE
ora.fed02.vip
      1        ONLINE  ONLINE       fed02                    STABLE
ora.oc4j
      1        ONLINE  ONLINE       fed01                    STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       fed02                    STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       fed01                    STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       fed01                    STABLE
--------------------------------------------------------------------------------

In my next post i’m trying to enable asmlib and acfs support. I hope i’ll get it to work.

DONE & Have Fun!

Oracle Database 12c: Apply 12.1.0.1.1 PSU (October 2013)


1. Preface

To apply PSU 12.1.0.1.1 oracle provides some new tools to reach this goal.

2. Apply PSU to single instance database

2.1 Set enviroment

[bash install]$ . oraenv
ORACLE_SID = [CDB1] ?
The Oracle base remains unchanged with value /opt/oracle/app
[bash install]$

2.2 Check OPatch version

[oracle install]$ $ORACLE_HOME/OPatch/opatch lsinventory | grep version
Oracle Interim Patch Installer version 12.1.0.1.0
OPatch version    : 12.1.0.1.0
OUI version       : 12.1.0.1.0

2.3 Upgrade OPatch

[oracle install]$ cp p6880880_121010_Linux-x86-64.zip $ORACLE_HOME
[oracle install]$ cd $ORACLE_HOME
[oracle dbhome_1]$ mv OPatch OPatch.old
[oracle dbhome_1]$ unzip p6880880_121010_Linux-x86-64.zip
Archive:  p6880880_121010_Linux-x86-64.zip
   creating: OPatch/
  inflating: OPatch/opatchauto
   creating: OPatch/jlib/
  inflating: OPatch/jlib/oracle.opatch.classpath.jar
  inflating: OPatch/jlib/oracle.opatch.classpath.windows.jar
  inflating: OPatch/jlib/opatch.jar
  inflating: OPatch/jlib/opatchsdk.jar
  inflating: OPatch/jlib/oracle.opatch.classpath.unix.jar
   creating: OPatch/oplan/
  inflating: OPatch/oplan/oplan.bat
   creating: OPatch/oplan/jlib/
  inflating: OPatch/oplan/jlib/oplan.jar
  inflating: OPatch/oplan/jlib/osysmodel-utils.jar
  inflating: OPatch/oplan/jlib/patchsdk.jar
  inflating: OPatch/oplan/jlib/JMXDrivers.jar
  inflating: OPatch/oplan/jlib/Validation.jar
  inflating: OPatch/oplan/jlib/bundle.jar
  inflating: OPatch/oplan/jlib/oracle.oplan.classpath.jar
  inflating: OPatch/oplan/jlib/OuiDriver.jar
  inflating: OPatch/oplan/jlib/automation.jar
   creating: OPatch/oplan/jlib/jaxb/
  inflating: OPatch/oplan/jlib/jaxb/jaxb-impl.jar
  inflating: OPatch/oplan/jlib/jaxb/activation.jar
  inflating: OPatch/oplan/jlib/jaxb/jaxb-api.jar
  inflating: OPatch/oplan/jlib/jaxb/jsr173_1.0_api.jar
  inflating: OPatch/oplan/jlib/EMrepoDrivers.jar
  inflating: OPatch/oplan/jlib/CRSProductDriver.jar
  inflating: OPatch/oplan/jlib/ValidationRules.jar
   creating: OPatch/oplan/jlib/apache-commons/
  inflating: OPatch/oplan/jlib/apache-commons/commons-cli-1.0.jar
  inflating: OPatch/oplan/jlib/OsysModel.jar
  inflating: OPatch/oplan/oplan
  inflating: OPatch/oplan/README.txt
  inflating: OPatch/oplan/README.html
   creating: OPatch/opatchprereqs/
  inflating: OPatch/opatchprereqs/prerequisite.properties
   creating: OPatch/opatchprereqs/opatch/
  inflating: OPatch/opatchprereqs/opatch/opatch_prereq.xml
  inflating: OPatch/opatchprereqs/opatch/runtime_prereq.xml
  inflating: OPatch/opatchprereqs/opatch/rulemap.xml
   creating: OPatch/opatchprereqs/oui/
  inflating: OPatch/opatchprereqs/oui/knowledgesrc.xml
  inflating: OPatch/emdpatch.pl
  inflating: OPatch/opatch.pl
  inflating: OPatch/opatch
  inflating: OPatch/opatch.bat
  inflating: OPatch/README.txt
  inflating: OPatch/datapatch.bat
   creating: OPatch/docs/
  inflating: OPatch/docs/Prereq_Users_Guide.txt
  inflating: OPatch/docs/Users_Guide.txt
  inflating: OPatch/docs/FAQ
   creating: OPatch/ocm/
  inflating: OPatch/ocm/ocm_platforms.txt
 extracting: OPatch/ocm/ocm.zip
   creating: OPatch/ocm/lib/
  inflating: OPatch/ocm/lib/emocmclnt.jar
  inflating: OPatch/ocm/lib/emocmclnt-14.jar
  inflating: OPatch/ocm/lib/http_client.jar
  inflating: OPatch/ocm/lib/osdt_jce.jar
  inflating: OPatch/ocm/lib/jnet.jar
  inflating: OPatch/ocm/lib/emocmcommon.jar
  inflating: OPatch/ocm/lib/xmlparserv2.jar
  inflating: OPatch/ocm/lib/log4j-core.jar
  inflating: OPatch/ocm/lib/jcert.jar
  inflating: OPatch/ocm/lib/jsse.jar
  inflating: OPatch/ocm/lib/osdt_core3.jar
  inflating: OPatch/ocm/lib/regexp.jar
   creating: OPatch/ocm/bin/
  inflating: OPatch/ocm/bin/emocmrsp
  inflating: OPatch/operr_readme.txt
 extracting: OPatch/version.txt
  inflating: OPatch/operr.bat
  inflating: OPatch/opatch.ini
  inflating: OPatch/datapatch
  inflating: OPatch/operr
  inflating: PatchSearch.xml
[oracle dbhome_1]$

2.4 Check OPatch pre apply requirements

[oracle dbhome_1]$ cd /opt/oracle/install
[oracle install]$ unzip p17027533_121010_Linux-x86-64.zip
Archive:  p17027533_121010_Linux-x86-64.zip
   creating: 17027533/
  inflating: 17027533/README.html
 extracting: 17027533/README.txt
   creating: 17027533/etc/
   creating: 17027533/etc/config/
  inflating: 17027533/etc/config/actions.xml
  inflating: 17027533/etc/config/inventory.xml
   creating: 17027533/files/
   creating: 17027533/files/lib/
  inflating: 17027533/files/lib/asmcmdexceptions.pm
  inflating: 17027533/files/lib/librs12.so
   creating: 17027533/files/lib/libgeneric12.a/
  inflating: 17027533/files/lib/libgeneric12.a/kgh.o
  inflating: 17027533/files/lib/libgeneric12.a/kgl4.o
  inflating: 17027533/files/lib/libgeneric12.a/kxdcap.o
  inflating: 17027533/files/lib/libgeneric12.a/kgl.o
  inflating: 17027533/files/lib/libgeneric12.a/kgl2.o
   creating: 17027533/files/lib/libserver12.a/
  inflating: 17027533/files/lib/libserver12.a/ksp.o
  inflating: 17027533/files/lib/libserver12.a/kpdbd.o
  inflating: 17027533/files/lib/libserver12.a/kmm.o
  inflating: 17027533/files/lib/libserver12.a/kjbm.o
  inflating: 17027533/files/lib/libserver12.a/ksb.o
  inflating: 17027533/files/lib/libserver12.a/kkm.o
  inflating: 17027533/files/lib/libserver12.a/kqr.o
  inflating: 17027533/files/lib/libserver12.a/kjcts.o
  inflating: 17027533/files/lib/libserver12.a/ktsj.o
  inflating: 17027533/files/lib/libserver12.a/opilof.o
  inflating: 17027533/files/lib/libserver12.a/kkzl.o
  inflating: 17027533/files/lib/libserver12.a/kcvfdb.o
  inflating: 17027533/files/lib/libserver12.a/kql.o
  inflating: 17027533/files/lib/libserver12.a/kjm.o
  inflating: 17027533/files/lib/libserver12.a/fbadrv.o
  inflating: 17027533/files/lib/libserver12.a/kpdba.o
  inflating: 17027533/files/lib/libserver12.a/ktm.o
  inflating: 17027533/files/lib/libserver12.a/ksl.o
  inflating: 17027533/files/lib/libserver12.a/kpolon.o
  inflating: 17027533/files/lib/libserver12.a/qesrc.o
  inflating: 17027533/files/lib/libserver12.a/kpdbe.o
  inflating: 17027533/files/lib/libserver12.a/kpdb.o
  inflating: 17027533/files/lib/libserver12.a/kzap.o
  inflating: 17027533/files/lib/libserver12.a/kzr.o
  inflating: 17027533/files/lib/libserver12.a/kjbl.o
  inflating: 17027533/files/lib/libserver12.a/qerfx.o
  inflating: 17027533/files/lib/libserver12.a/ksfd.o
  inflating: 17027533/files/lib/libserver12.a/ksfdaf.o
  inflating: 17027533/files/lib/libserver12.a/kdu.o
  inflating: 17027533/files/lib/libserver12.a/krvrd.o
  inflating: 17027533/files/lib/libserver12.a/kff.o
  inflating: 17027533/files/lib/libserver12.a/kpon.o
  inflating: 17027533/files/lib/libserver12.a/kpdbutl.o
  inflating: 17027533/files/lib/libserver12.a/knalf.o
  inflating: 17027533/files/lib/libserver12.a/jscr.o
  inflating: 17027533/files/lib/libserver12.a/kxdam.o
  inflating: 17027533/files/lib/libserver12.a/knas.o
  inflating: 17027533/files/lib/libserver12.a/kcfis.o
  inflating: 17027533/files/lib/libserver12.a/krt.o
  inflating: 17027533/files/lib/libserver12.a/ctc.o
  inflating: 17027533/files/lib/libserver12.a/ksws.o
  inflating: 17027533/files/lib/libserver12.a/aud.o
  inflating: 17027533/files/lib/libserver12.a/krvt.o
  inflating: 17027533/files/lib/libserver12.a/kwqr.o
  inflating: 17027533/files/lib/libserver12.a/kjb.o
  inflating: 17027533/files/lib/libserver12.a/kzvdve.o
  inflating: 17027533/files/lib/libserver12.a/kxes.o
  inflating: 17027533/files/lib/libserver12.a/cvw.o
  inflating: 17027533/files/lib/libserver12.a/kqlb.o
  inflating: 17027533/files/lib/libserver12.a/ktsi.o
  inflating: 17027533/files/lib/libserver12.a/kfi.o
  inflating: 17027533/files/lib/libserver12.a/knipg.o
  inflating: 17027533/files/lib/libserver12.a/jskq.o
  inflating: 17027533/files/lib/libserver12.a/kjzd.o
  inflating: 17027533/files/lib/libserver12.a/kdo.o
  inflating: 17027533/files/lib/libserver12.a/kjbr.o
  inflating: 17027533/files/lib/libserver12.a/kwqic.o
  inflating: 17027533/files/lib/libserver12.a/ksk.o
  inflating: 17027533/files/lib/libserver12.a/knl.o
  inflating: 17027533/files/lib/libserver12.a/ktfa.o
  inflating: 17027533/files/lib/libserver12.a/knahs.o
  inflating: 17027533/files/lib/libserver12.a/qol.o
  inflating: 17027533/files/lib/libserver12.a/ksu.o
  inflating: 17027533/files/lib/libserver12.a/kcl.o
  inflating: 17027533/files/lib/libserver12.a/knanr.o
  inflating: 17027533/files/lib/libserver12.a/kxdofl.o
  inflating: 17027533/files/lib/libserver12.a/k2v.o
  inflating: 17027533/files/lib/libserver12.a/kjfc.o
  inflating: 17027533/files/lib/libserver12.a/knalse.o
  inflating: 17027533/files/lib/libserver12.a/kkt.o
  inflating: 17027533/files/lib/libserver12.a/kzekm.o
  inflating: 17027533/files/lib/libserver12.a/kdn.o
  inflating: 17027533/files/lib/libserver12.a/kcf.o
  inflating: 17027533/files/lib/libserver12.a/kwqmn.o
  inflating: 17027533/files/lib/libserver12.a/ktt.o
  inflating: 17027533/files/lib/libserver12.a/kzp.o
  inflating: 17027533/files/lib/libserver12.a/kokl2.o
  inflating: 17027533/files/lib/libserver12.a/kzckmr.o
  inflating: 17027533/files/lib/libserver12.a/knasp.o
  inflating: 17027533/files/lib/libserver12.a/kokt.o
  inflating: 17027533/files/lib/libserver12.a/kfda.o
  inflating: 17027533/files/lib/libserver12.a/kfpkg.o
  inflating: 17027533/files/lib/libserver12.a/kpdbicd.o
  inflating: 17027533/files/lib/libserver12.a/opiprs.o
  inflating: 17027533/files/lib/libserver12.a/knahf.o
  inflating: 17027533/files/lib/libserver12.a/qksrc.o
  inflating: 17027533/files/lib/libserver12.a/dgls.o
  inflating: 17027533/files/lib/libserver12.a/kzmkr.o
  inflating: 17027533/files/lib/libserver12.a/kewf.o
  inflating: 17027533/files/lib/libserver12.a/kcc.o
  inflating: 17027533/files/lib/libserver12.a/qkagby.o
  inflating: 17027533/files/lib/libserver12.a/ksfq.o
  inflating: 17027533/files/lib/libserver12.a/kewm.o
  inflating: 17027533/files/lib/libserver12.a/kpdbcv.o
  inflating: 17027533/files/lib/libserver12.a/kfd.o
  inflating: 17027533/files/lib/libserver12.a/kcbo.o
  inflating: 17027533/files/lib/libserver12.a/knlogc.o
  inflating: 17027533/files/lib/libserver12.a/kpdbc.o
  inflating: 17027533/files/lib/libserver12.a/qmps.o
  inflating: 17027533/files/lib/libserver12.a/sol.o
  inflating: 17027533/files/lib/libserver12.a/dgl.o
  inflating: 17027533/files/lib/libserver12.a/krvxr.o
  inflating: 17027533/files/lib/libserver12.a/kwslb.o
  inflating: 17027533/files/lib/libserver12.a/jskm.o
  inflating: 17027533/files/lib/libserver12.a/krvxb.o
  inflating: 17027533/files/lib/libserver12.a/kcb.o
  inflating: 17027533/files/lib/libserver12.a/kfgb.o
  inflating: 17027533/files/lib/libserver12.a/knals.o
  inflating: 17027533/files/lib/libserver12.a/ktrv.o
  inflating: 17027533/files/lib/libserver12.a/kkp.o
  inflating: 17027533/files/lib/libserver12.a/knasx.o
  inflating: 17027533/files/lib/libserver12.a/knasda.o
  inflating: 17027533/files/lib/libserver12.a/kfvsu.o
  inflating: 17027533/files/lib/libserver12.a/knam.o
  inflating: 17027533/files/lib/libserver12.a/kkeaf.o
  inflating: 17027533/files/lib/libserver12.a/kwqv.o
  inflating: 17027533/files/lib/libserver12.a/kzckm.o
  inflating: 17027533/files/lib/libserver12.a/kjzn.o
  inflating: 17027533/files/lib/libserver12.a/knac.o
  inflating: 17027533/files/lib/libserver12.a/kffm.o
  inflating: 17027533/files/lib/libserver12.a/zllc.o
  inflating: 17027533/files/lib/libnnz12.so
  inflating: 17027533/files/lib/libnnzst12.a
  inflating: 17027533/files/lib/asmcmdshare.pm
  inflating: 17027533/files/lib/libzt12.a
  inflating: 17027533/files/lib/asmcmddisk.pm
  inflating: 17027533/files/lib/libosbws12.so
   creating: 17027533/files/sqlpatch/
  inflating: 17027533/files/sqlpatch/sqlpatch.pm
  inflating: 17027533/files/sqlpatch/sqlpatch.pl
   creating: 17027533/files/sqlpatch/17027533/
  inflating: 17027533/files/sqlpatch/17027533/17027533_apply.sql
  inflating: 17027533/files/sqlpatch/17027533/17027533_rollback.sql
   creating: 17027533/files/rdbms/
   creating: 17027533/files/rdbms/lib/
  inflating: 17027533/files/rdbms/lib/jox.o
   creating: 17027533/files/rdbms/lib/libknlopt.a/
  inflating: 17027533/files/rdbms/lib/libknlopt.a/jox.o
   creating: 17027533/files/rdbms/mesg/
  inflating: 17027533/files/rdbms/mesg/oraus.msg
  inflating: 17027533/files/rdbms/mesg/oraus.msb
   creating: 17027533/files/rdbms/admin/
  inflating: 17027533/files/rdbms/admin/prvtbstr.plb
  inflating: 17027533/files/rdbms/admin/prvtlmd.plb
  inflating: 17027533/files/rdbms/admin/bundledata_PSU.xml
  inflating: 17027533/files/rdbms/admin/catbundleapply.sql
  inflating: 17027533/files/rdbms/admin/prvtbxstr.plb
  inflating: 17027533/files/rdbms/admin/catuppst.sql
  inflating: 17027533/files/rdbms/admin/catbundle.sql
  inflating: 17027533/files/rdbms/admin/cdenv.sql
  inflating: 17027533/files/rdbms/admin/catbundlerollback.sql
   creating: 17027533/files/bin/
  inflating: 17027533/files/bin/asmcmdcore
   creating: 17027533/files/patch/
   creating: 17027533/files/patch/scripts/
  inflating: 17027533/files/patch/scripts/bug16825779.sql
  inflating: 17027533/files/patch/scripts/bug16286774.sql
[oracle install]$ cd 17027533
[oracle 17027533]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./
Oracle Interim Patch Installer version 12.1.0.1.2
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /opt/oracle/app/product/12.1.0/dbhome_1
Central Inventory : /opt/oracle/oraInventory
   from           : /opt/oracle/app/product/12.1.0/dbhome_1/oraInst.loc
OPatch version    : 12.1.0.1.2
OUI version       : 12.1.0.1.0
Log file location : /opt/oracle/app/product/12.1.0/dbhome_1/cfgtoollogs/opatch/opatch2013-11-27_21-48-10PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle 17027533]$

2.5 Stopping all components

[oracle 17027533]$ lsnrctl stop

LSNRCTL for Linux: Version 12.1.0.1.0 - Production on 27-NOV-2013 21:43:34

Copyright (c) 1991, 2013, Oracle.  All rights reserved.

Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
The command completed successfully
[oracle 17027533]$
[oracle 17027533]$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.1.0 Production on Wed Nov 27 21:43:07 2013

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

2.6 Apply patch

[oracle 17027533]$ $ORACLE_HOME/OPatch/opatch apply
Oracle Interim Patch Installer version 12.1.0.1.2
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

Oracle Home       : /opt/oracle/app/product/12.1.0/dbhome_1
Central Inventory : /opt/oracle/oraInventory
   from           : /opt/oracle/app/product/12.1.0/dbhome_1/oraInst.loc
OPatch version    : 12.1.0.1.2
OUI version       : 12.1.0.1.0
Log file location : /opt/oracle/app/product/12.1.0/dbhome_1/cfgtoollogs/opatch/17027533_Nov_27_2013_21_43_50/apply2013-11-27_21-43-50PM_1.log

Applying interim patch '17027533' to OH '/opt/oracle/app/product/12.1.0/dbhome_1'
Verifying environment and performing prerequisite checks...
All checks passed.
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  y

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/opt/oracle/app/product/12.1.0/dbhome_1')

Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...

Patching component oracle.rdbms, 12.1.0.1.0...

Patching component oracle.rdbms.dbscripts, 12.1.0.1.0...

Patching component oracle.rdbms.rsf, 12.1.0.1.0...

Patching component oracle.ldap.rsf, 12.1.0.1.0...

Patching component oracle.ldap.rsf.ic, 12.1.0.1.0...

Verifying the update...
Patch 17027533 successfully applied
Log file location: /opt/oracle/app/product/12.1.0/dbhome_1/cfgtoollogs/opatch/17027533_Nov_27_2013_21_43_50/apply2013-11-27_21-43-50PM_1.log

OPatch succeeded.
[oracle 17027533]$

2.7 Start components

[oracle 17027533]$ lsnrctl start

LSNRCTL for Linux: Version 12.1.0.1.0 - Production on 27-NOV-2013 21:44:48

Copyright (c) 1991, 2013, Oracle.  All rights reserved.

Starting /opt/oracle/app/product/12.1.0/dbhome_1/bin/tnslsnr: please wait...

TNSLSNR for Linux: Version 12.1.0.1.0 - Production
Log messages written to /opt/oracle/app/diag/tnslsnr/oel12ctest/listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oel12ctest)(PORT=1521)))

Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 12.1.0.1.0 - Production
Start Date                27-NOV-2013 21:44:48
Uptime                    0 days 0 hr. 0 min. 0 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Log File         /opt/oracle/app/diag/tnslsnr/oel12ctest/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oel12ctest)(PORT=1521)))
The listener supports no services
The command completed successfully
[oracle 17027533]$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.1.0 Production on Wed Nov 27 21:44:50 2013

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ORACLE instance started.

Total System Global Area 1269366784 bytes
Fixed Size                  2287912 bytes
Variable Size             855639768 bytes
Database Buffers          402653184 bytes
Redo Buffers                8785920 bytes
Database mounted.
Database opened.
SQL> alter pluggable database all open;

Pluggable database altered.

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

2.7 Check applied patches

cd $ORACLE_HOME/OPatch
[oracle OPatch]$ ./opatch lsinventory
Oracle Interim Patch Installer version 12.1.0.1.2
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

Oracle Home       : /opt/oracle/app/product/12.1.0/dbhome_1
Central Inventory : /opt/oracle/oraInventory
   from           : /opt/oracle/app/product/12.1.0/dbhome_1/oraInst.loc
OPatch version    : 12.1.0.1.2
OUI version       : 12.1.0.1.0
Log file location : /opt/oracle/app/product/12.1.0/dbhome_1/cfgtoollogs/opatch/opatch2013-11-27_21-52-12PM_1.log

Lsinventory Output file location : /opt/oracle/app/product/12.1.0/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2013-11-27_21-52-12PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Database 12c                                                  12.1.0.1.0
There are 1 products installed in this Oracle Home.

Interim patches (1) :

Patch  17027533     : applied on Wed Nov 27 21:49:35 CET 2013
Unique Patch ID:  16677152
Patch description:  "Database Patch Set Update : 12.1.0.1.1 (17027533)"
   Created on 27 Sep 2013, 05:30:33 hrs PST8PDT
   Bugs fixed:
     17034172, 16694728, 16448848, 16863422, 16634384, 16465158, 16320173
     16313881, 16910734, 16816103, 16911800, 16715647, 16825779, 16707927
     16392068, 14197853, 16712618, 17273253, 16902138, 16524071, 16856570
     16465149, 16705020, 16689109, 16372203, 16864864, 16849982, 16946613
     16837842, 16964279, 16459685, 16978185, 16845022, 16195633, 14536110
     16964686, 16787973, 16850996, 16674842, 16838328, 16178562, 15996344
     16503473, 16842274, 16935643, 17000176, 14355775, 16362358, 16994576
     16485876, 16919176, 16928832, 16864359, 16617325, 16921340, 16679874
     16788832, 16483559, 16733884, 16784167, 16286774, 15986012, 16660558
     16674666, 16191248, 16697600, 16993424, 16946990, 16589507, 16173738
     16784143, 16772060, 16991789, 17346196, 16495802, 16859937, 16590848
     16910001, 16603924, 16427054, 16730813, 16227068, 16663303, 16784901
     16836849, 16186165, 16457621, 16007562, 16170787, 16663465, 16524968
     16543323, 17027533, 16675710, 17005047, 16795944, 16668226, 16070351
     16212405, 16523150, 16698577, 16621274, 16930325, 17330580, 16443657

--------------------------------------------------------------------------------

OPatch succeeded.

2.8 Postpatch work
Here comes the new section. On 11g and earlier you have to use @catbundle.sql psu apply. In 12g the new tool datapatch will do this work you you.

[oracle 17027533]$ cd $ORACLE_HOME/OPatch
[oracle OPatch]$ ./datapatch -verbose
SQL Patching tool version 12.1.0.1.0 on Wed Nov 27 21:46:28 2013
Copyright (c) 2013, Oracle.  All rights reserved.

Connecting to database...OK
Determining current state...
Currently installed SQL Patches:
  PDB CDB$ROOT:
  PDB PDB$SEED:
  PDB PDB1:
Currently installed C Patches: 17027533
For the following PDBs: CDB$ROOT
  Nothing to roll back
  The following patches will be applied: 17027533
For the following PDBs: PDB$SEED
  Nothing to roll back
  The following patches will be applied: 17027533
For the following PDBs: PDB1
  Nothing to roll back
  The following patches will be applied: 17027533
Adding patches to installation queue...
Installing patches...
Validating logfiles...
Patch 17027533 apply (pdb CDB$ROOT): SUCCESS
  logfile: /opt/oracle/app/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_apply_CDB1_CDBROOT_2013Nov27_21_46_32.log (no errors)
Patch 17027533 apply (pdb PDB$SEED): SUCCESS
  logfile: /opt/oracle/app/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_apply_CDB1_PDBSEED_2013Nov27_21_46_41.log (no errors)
Patch 17027533 apply (pdb PDB1): SUCCESS
  logfile: /opt/oracle/app/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_apply_CDB1_PDB1_2013Nov27_21_46_44.log (no errors)
SQL Patching tool complete on Wed Nov 27 21:46:53 2013

Problems while patching with datapatch

If you rollback an online patch before you apply the psu you may get an error during datapatch:

[oracle OPatch]$ export OPATCH_DEBUG=true
[oracle OPatch]$ ./datapatch -verbose -debug
SQL Patching tool version 12.1.0.1.0 on Tue Oct 29 13:25:17 2013
Copyright (c) 2013, Oracle. All rights reserved.

Command line arguments:
db:
apply_list:
rollback_list:
force: 0
prereq: 0
oh:
Connecting to database...OK
Determining current state...
CDB! pdbs: CDB$ROOT PDB$SEED PDB1
Currently installed SQL Patches:
PDB CDB$ROOT:
PDB PDB$SEED:
PDB PDB1:
DBD::Oracle::st execute failed: ORA-20001: Latest xml inventory is not loaded into table
ORA-06512: at "SYS.DBMS_QOPATCH", line 1011
ORA-06512: at line 4 (DBD ERROR: OCIStmtExecute) [for Statement "DECLARE
x XMLType;
BEGIN
x := dbms_qopatch.get_pending_activity;
? := x.getStringVal();
END;" with ParamValues: :p1=undef] at /opt/oracle/app/product/12.1.0/dbhome_1/sqlpatch/sqlpatch.pm line 824.
[oracle OPatch]$

This is a problem with directories created during patching …
To solve this issue you can workaround with the following

Check parameter:

SELECT a.ksppinm “Parameter”,
b.ksppstvl “Session Value”,
c.ksppstvl “Instance Value”
FROM x$ksppi a,
x$ksppcv b,
x$ksppsv c
WHERE a.indx = b.indx
AND a.indx = c.indx
AND a.ksppinm LIKE ‘/_disable_direc%’ escape ‘/’

if _disable_directory_link_check is set to FALSE I would suggest you set to TRUE and try to execute datapatch and update again. It should work now.

3. Apply PSU to Grid Infrastruce and RAC database

3.1 Check GI Patches

[oracle@node0 ~]$ opatch lsinventory
Oracle Interim Patch Installer version 12.1.0.1.2
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

Oracle Home       : /opt/oracle/12.1.0.1/grid
Central Inventory : /opt/oracle/oraInventory
   from           : /opt/oracle/12.1.0.1/grid/oraInst.loc
OPatch version    : 12.1.0.1.2
OUI version       : 12.1.0.1.0
Log file location : /opt/oracle/12.1.0.1/grid/cfgtoollogs/opatch/opatch2013-11-24_22-37-14PM_1.log

Lsinventory Output file location : /opt/oracle/12.1.0.1/grid/cfgtoollogs/opatch/lsinv/lsinventory2013-11-24_22-37-14PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Grid Infrastructure 12c                                       12.1.0.1.0
There are 1 products installed in this Oracle Home.

There are no Interim patches installed in this Oracle Home.

Patch level status of Cluster nodes :

 Patching Level                  Nodes
 --------------                  -----
 0                               node1,node0,node2,node3

--------------------------------------------------------------------------------

3.2 Check Databases Patches

[oracle@node0 ~]$ opatch lsinventory
Oracle Interim Patch Installer version 12.1.0.1.2
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

Oracle Home       : /opt/oracle/app/12.1.0.1/dbhome_1
Central Inventory : /opt/oracle/oraInventory
   from           : /opt/oracle/app/12.1.0.1/dbhome_1/oraInst.loc
OPatch version    : 12.1.0.1.2
OUI version       : 12.1.0.1.0
Log file location : /opt/oracle/app/12.1.0.1/dbhome_1/cfgtoollogs/opatch/opatch2013-11-24_22-39-13PM_1.log

Lsinventory Output file location : /opt/oracle/app/12.1.0.1/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2013-11-24_22-39-13PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Database 12c                                                  12.1.0.1.0
There are 1 products installed in this Oracle Home.

There are no Interim patches installed in this Oracle Home.

Rac system comprising of multiple nodes
  Local node = node0
  Remote node = node1
  Remote node = node2
  Remote node = node3

--------------------------------------------------------------------------------

3.3 Create response file

[oracle@node0 tmp]$ cd /opt/oracle/12.1.0.1/grid/OPatch/ocm/bin
[oracle@node0 bin]$ ls -l
total 12
-rwxr----- 1 oracle oinstall 9063 Nov 27  2009 emocmrsp
[oracle@node0 bin]$ ./emocmrsp
OCM Installation Response Generator 10.3.7.0.0 - Production
Copyright (c) 2005, 2012, Oracle and/or its affiliates.  All rights reserved.

Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  y
The OCM configuration response file (ocm.rsp) was successfully created.
[oracle@node0 bin]$ ls -l
total 16
-rwxr----- 1 oracle oinstall 9063 Nov 27  2009 emocmrsp
-rw-r--r-- 1 oracle oinstall  623 Nov 24 22:42 ocm.rsp
[oracle@node0 bin]$ cp ocm.rsp /opt/oracle/install/

3.4 Patch GI and Databases with opatchauto
opatchauto is also a new tool to apply patches automated to gi and database at one step. You have to run this as ROOT user:

[root@node0 install]# opatchauto apply /opt/oracle/install/17272829 -ocmrf /opt/oracle/install/ocm.rsp
OPatch Automation Tool
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

OPatchauto version : 12.1.0.1.2
OUI version        : 12.1.0.1.0
Running from       : /opt/oracle/12.1.0.1/grid

opatchauto log file: /opt/oracle/12.1.0.1/grid/cfgtoollogs/opatchauto/17272829/opatch_gi_2013-11-24_22-43-51_deploy.log

Parameter Validation: Successful

Grid Infrastructure home:
/opt/oracle/12.1.0.1/grid
RAC home(s):
/opt/oracle/app/12.1.0.1/dbhome_1

Configuration Validation: Successful

Patch Location: /opt/oracle/install/17272829
Grid Infrastructure Patch(es): 17027533 17077442 17303297
RAC Patch(es): 17027533 17077442

Patch Validation: Successful

Stopping RAC (/opt/oracle/app/12.1.0.1/dbhome_1) ... Successful
Following database(s) were stopped and will be restarted later during the session: cdb1

Applying patch(es) to "/opt/oracle/app/12.1.0.1/dbhome_1" ...
Patch "/opt/oracle/install/17272829/17027533" successfully applied to "/opt/oracle/app/12.1.0.1/dbhome_1".
Patch "/opt/oracle/install/17272829/17077442" successfully applied to "/opt/oracle/app/12.1.0.1/dbhome_1".

Stopping CRS ... Successful

Applying patch(es) to "/opt/oracle/12.1.0.1/grid" ...
Patch "/opt/oracle/install/17272829/17027533" successfully applied to "/opt/oracle/12.1.0.1/grid".
Patch "/opt/oracle/install/17272829/17077442" successfully applied to "/opt/oracle/12.1.0.1/grid".
Patch "/opt/oracle/install/17272829/17303297" successfully applied to "/opt/oracle/12.1.0.1/grid".

Starting CRS ... Successful

Starting RAC (/opt/oracle/app/12.1.0.1/dbhome_1) ... Successful

SQL changes, if any, are applied successfully on the following database(s): CDB1

Apply Summary:
Following patch(es) are successfully installed:
GI Home: /opt/oracle/12.1.0.1/grid: 17027533, 17077442, 17303297
RAC Home: /opt/oracle/app/12.1.0.1/dbhome_1: 17027533, 17077442

opatchauto succeeded.
[root@node0 install]#

3.5 Check patch on local node

[root@node0 install]# su - oracle
[oracle@node0 ~]$ opatch lsinventory
Oracle Interim Patch Installer version 12.1.0.1.2
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

Oracle Home       : /opt/oracle/12.1.0.1/grid
Central Inventory : /opt/oracle/oraInventory
   from           : /opt/oracle/12.1.0.1/grid/oraInst.loc
OPatch version    : 12.1.0.1.2
OUI version       : 12.1.0.1.0
Log file location : /opt/oracle/12.1.0.1/grid/cfgtoollogs/opatch/opatch2013-11-24_22-57-20PM_1.log

Lsinventory Output file location : /opt/oracle/12.1.0.1/grid/cfgtoollogs/opatch/lsinv/lsinventory2013-11-24_22-57-20PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Grid Infrastructure 12c                                       12.1.0.1.0
There are 1 products installed in this Oracle Home.

Interim patches (3) :

Patch  17303297     : applied on Sun Nov 24 22:50:27 CET 2013
Unique Patch ID:  16881795
Patch description:  "ACFS Patch Set Update 12.1.0.1.1"
   Created on 14 Oct 2013, 07:25:50 hrs US/Central
   Bugs fixed:
     14487556, 16398970, 16552813, 16930184, 16420645, 16170117, 16436434
     16476044, 16458315, 16463033, 16095100, 16545876, 16429953, 14826673
     16001893, 16482869, 16371746, 16435343, 14476443, 16294308, 16671486
     16386110, 15978267, 16085530, 16347837, 16814544, 16022372, 16167084
     14510092, 16450287, 16399406

Patch  17077442     : applied on Sun Nov 24 22:49:35 CET 2013
Unique Patch ID:  16881794
Patch description:  "Oracle Clusterware Patch Set Update 12.1.0.1.1"
   Created on 12 Oct 2013, 06:33:53 hrs US/Central
   Bugs fixed:
     16505840, 16505255, 16390989, 16399322, 16505617, 16505717, 17486244
     16168869, 16444109, 16505361, 13866165, 16505763, 16208257, 16904822
     17299876, 16246222, 16505214, 16505540, 15936039, 16580269, 16838292
     16505449, 16801843, 16309853, 16505395, 17507349, 17475155, 16493242
     17039197, 16196609, 17463260, 16505667, 15970176, 16488665, 16670327

Patch  17027533     : applied on Sun Nov 24 22:49:04 CET 2013
Unique Patch ID:  16677152
Patch description:  "Database Patch Set Update : 12.1.0.1.1 (17027533)"
   Created on 27 Sep 2013, 05:30:33 hrs PST8PDT
   Bugs fixed:
     17034172, 16694728, 16448848, 16863422, 16634384, 16465158, 16320173
     16313881, 16910734, 16816103, 16911800, 16715647, 16825779, 16707927
     16392068, 14197853, 16712618, 17273253, 16902138, 16524071, 16856570
     16465149, 16705020, 16689109, 16372203, 16864864, 16849982, 16946613
     16837842, 16964279, 16459685, 16978185, 16845022, 16195633, 14536110
     16964686, 16787973, 16850996, 16674842, 16838328, 16178562, 15996344
     16503473, 16842274, 16935643, 17000176, 14355775, 16362358, 16994576
     16485876, 16919176, 16928832, 16864359, 16617325, 16921340, 16679874
     16788832, 16483559, 16733884, 16784167, 16286774, 15986012, 16660558
     16674666, 16191248, 16697600, 16993424, 16946990, 16589507, 16173738
     16784143, 16772060, 16991789, 17346196, 16495802, 16859937, 16590848
     16910001, 16603924, 16427054, 16730813, 16227068, 16663303, 16784901
     16836849, 16186165, 16457621, 16007562, 16170787, 16663465, 16524968
     16543323, 17027533, 16675710, 17005047, 16795944, 16668226, 16070351
     16212405, 16523150, 16698577, 16621274, 16930325, 17330580, 16443657

Patch level status of Cluster nodes :

 Patching Level                  Nodes
 --------------                  -----
 1650217826                      node0
 0                               node1,node2,node3

--------------------------------------------------------------------------------

OPatch succeeded.
[oracle@node0 ~]$

3.6 Repeat the opatchauto process on each node
3.7 Check opatch inventory again now

[root@node0 install]# su - oracle
[oracle@node0 ~]$ opatch lsinventory
Oracle Interim Patch Installer version 12.1.0.1.2
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

Oracle Home       : /opt/oracle/12.1.0.1/grid
Central Inventory : /opt/oracle/oraInventory
   from           : /opt/oracle/12.1.0.1/grid/oraInst.loc
OPatch version    : 12.1.0.1.2
OUI version       : 12.1.0.1.0
Log file location : /opt/oracle/12.1.0.1/grid/cfgtoollogs/opatch/opatch2013-11-24_22-57-20PM_1.log

Lsinventory Output file location : /opt/oracle/12.1.0.1/grid/cfgtoollogs/opatch/lsinv/lsinventory2013-11-24_22-57-20PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Grid Infrastructure 12c                                       12.1.0.1.0
There are 1 products installed in this Oracle Home.

Interim patches (3) :

Patch  17303297     : applied on Sun Nov 24 22:50:27 CET 2013
Unique Patch ID:  16881795
Patch description:  "ACFS Patch Set Update 12.1.0.1.1"
   Created on 14 Oct 2013, 07:25:50 hrs US/Central
   Bugs fixed:
     14487556, 16398970, 16552813, 16930184, 16420645, 16170117, 16436434
     16476044, 16458315, 16463033, 16095100, 16545876, 16429953, 14826673
     16001893, 16482869, 16371746, 16435343, 14476443, 16294308, 16671486
     16386110, 15978267, 16085530, 16347837, 16814544, 16022372, 16167084
     14510092, 16450287, 16399406

Patch  17077442     : applied on Sun Nov 24 22:49:35 CET 2013
Unique Patch ID:  16881794
Patch description:  "Oracle Clusterware Patch Set Update 12.1.0.1.1"
   Created on 12 Oct 2013, 06:33:53 hrs US/Central
   Bugs fixed:
     16505840, 16505255, 16390989, 16399322, 16505617, 16505717, 17486244
     16168869, 16444109, 16505361, 13866165, 16505763, 16208257, 16904822
     17299876, 16246222, 16505214, 16505540, 15936039, 16580269, 16838292
     16505449, 16801843, 16309853, 16505395, 17507349, 17475155, 16493242
     17039197, 16196609, 17463260, 16505667, 15970176, 16488665, 16670327

Patch  17027533     : applied on Sun Nov 24 22:49:04 CET 2013
Unique Patch ID:  16677152
Patch description:  "Database Patch Set Update : 12.1.0.1.1 (17027533)"
   Created on 27 Sep 2013, 05:30:33 hrs PST8PDT
   Bugs fixed:
     17034172, 16694728, 16448848, 16863422, 16634384, 16465158, 16320173
     16313881, 16910734, 16816103, 16911800, 16715647, 16825779, 16707927
     16392068, 14197853, 16712618, 17273253, 16902138, 16524071, 16856570
     16465149, 16705020, 16689109, 16372203, 16864864, 16849982, 16946613
     16837842, 16964279, 16459685, 16978185, 16845022, 16195633, 14536110
     16964686, 16787973, 16850996, 16674842, 16838328, 16178562, 15996344
     16503473, 16842274, 16935643, 17000176, 14355775, 16362358, 16994576
     16485876, 16919176, 16928832, 16864359, 16617325, 16921340, 16679874
     16788832, 16483559, 16733884, 16784167, 16286774, 15986012, 16660558
     16674666, 16191248, 16697600, 16993424, 16946990, 16589507, 16173738
     16784143, 16772060, 16991789, 17346196, 16495802, 16859937, 16590848
     16910001, 16603924, 16427054, 16730813, 16227068, 16663303, 16784901
     16836849, 16186165, 16457621, 16007562, 16170787, 16663465, 16524968
     16543323, 17027533, 16675710, 17005047, 16795944, 16668226, 16070351
     16212405, 16523150, 16698577, 16621274, 16930325, 17330580, 16443657

Patch level status of Cluster nodes :

 Patching Level                  Nodes
 --------------                  -----
 1650217826                      node0,node1,node2,node3

--------------------------------------------------------------------------------

OPatch succeeded.

3.8 And on database home

[oracle@node0 ~]$ . oraenv
ORACLE_SID = [+ASM1] ? CDB1
The Oracle base has been set to /opt/oracle/app
[oracle@node0 ~]$ opatch lsinventory
Oracle Interim Patch Installer version 12.1.0.1.2
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

Oracle Home       : /opt/oracle/app/12.1.0.1/dbhome_1
Central Inventory : /opt/oracle/oraInventory
   from           : /opt/oracle/app/12.1.0.1/dbhome_1/oraInst.loc
OPatch version    : 12.1.0.1.2
OUI version       : 12.1.0.1.0
Log file location : /opt/oracle/app/12.1.0.1/dbhome_1/cfgtoollogs/opatch/opatch2013-11-24_22-58-04PM_1.log

Lsinventory Output file location : /opt/oracle/app/12.1.0.1/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2013-11-24_22-58-04PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Database 12c                                                  12.1.0.1.0
There are 1 products installed in this Oracle Home.

Interim patches (2) :

Patch  17077442     : applied on Sun Nov 24 22:46:01 CET 2013
Unique Patch ID:  16881794
Patch description:  "Oracle Clusterware Patch Set Update 12.1.0.1.1"
   Created on 12 Oct 2013, 06:33:53 hrs US/Central
   Bugs fixed:
     16505840, 16505255, 16390989, 16399322, 16505617, 16505717, 17486244
     16168869, 16444109, 16505361, 13866165, 16505763, 16208257, 16904822
     17299876, 16246222, 16505214, 16505540, 15936039, 16580269, 16838292
     16505449, 16801843, 16309853, 16505395, 17507349, 17475155, 16493242
     17039197, 16196609, 17463260, 16505667, 15970176, 16488665, 16670327

Patch  17027533     : applied on Sun Nov 24 22:45:55 CET 2013
Unique Patch ID:  16677152
Patch description:  "Database Patch Set Update : 12.1.0.1.1 (17027533)"
   Created on 27 Sep 2013, 05:30:33 hrs PST8PDT
   Bugs fixed:
     17034172, 16694728, 16448848, 16863422, 16634384, 16465158, 16320173
     16313881, 16910734, 16816103, 16911800, 16715647, 16825779, 16707927
     16392068, 14197853, 16712618, 17273253, 16902138, 16524071, 16856570
     16465149, 16705020, 16689109, 16372203, 16864864, 16849982, 16946613
     16837842, 16964279, 16459685, 16978185, 16845022, 16195633, 14536110
     16964686, 16787973, 16850996, 16674842, 16838328, 16178562, 15996344
     16503473, 16842274, 16935643, 17000176, 14355775, 16362358, 16994576
     16485876, 16919176, 16928832, 16864359, 16617325, 16921340, 16679874
     16788832, 16483559, 16733884, 16784167, 16286774, 15986012, 16660558
     16674666, 16191248, 16697600, 16993424, 16946990, 16589507, 16173738
     16784143, 16772060, 16991789, 17346196, 16495802, 16859937, 16590848
     16910001, 16603924, 16427054, 16730813, 16227068, 16663303, 16784901
     16836849, 16186165, 16457621, 16007562, 16170787, 16663465, 16524968
     16543323, 17027533, 16675710, 17005047, 16795944, 16668226, 16070351
     16212405, 16523150, 16698577, 16621274, 16930325, 17330580, 16443657

Rac system comprising of multiple nodes
  Local node = node0
  Remote node = node1
  Remote node = node2
  Remote node = node3

--------------------------------------------------------------------------------

OPatch succeeded.
[oracle@node0 ~]$

All done

Oracle Security Patch October 2013 has been released


On 15. October 2013 Oracle released the quarterly Security Patch for October 2013.

How to patch, see here.

At same time the following PSUs for Database and Clusterware/Grid Infrastructure has been released:

Unix/Linux Systems:

  • 12.1.0.1.0 PSU 1 (12.1.0.1.1)  (DB: 17027533, GI: 17272829)
    actual available for:
    • Linux x86-64
    • Solaris x86-64
    • Solaris SPARC (64Bit)
  • 11.2.0.3.0 PSU 8 (11.2.0.3.8)  (DB:  16902043, GI: 17272731)
  • 11.2.0.2.0 PSU 12 (11.2.0.2.12)  (DB:  17082367, GI: 17272753)
  • 11.1.0.7.0 PSU 16 (11.1.0.7.16) (DB:  17082366, CRS: 11724953)

Windows Systems:

For 10g Customers:

The PSU July 2013 was the final PSU for Oracle 10gR2 Database (10.2.0.4.0 & 10.2.0.5.0). So the final version is:

Unix/Linux

  • 10.2.0.5.0 PSU 12
  • 10.2.0.4.0 PSU 17

Windows

  • 10.2.0.5 BP 23 (32 Bit)
  • 10.2.0.5 BP 22 (64 Bit) (Maybe there will be a BP 23, Documentation is inconsistent)
  • 10.2.0.4 BP 50 (32 Bit)
  • 10.2.0.4 BP 49 (64 Bit) (Maybe there will be a BP 50, Documentation is inconsistent)

Common Vulnerabilities and Exposures (CVE) fixed in these patches:

  • CVE-2011-3389
  • CVE-2013-0169
  • CVE-2013-3762
  • CVE-2013-5766
  • CVE-2013-5827
  • CVE-2013-5828

Limitations:

  • Patch Set Update (PSU) patches are cumulative.
  • This patch is Oracle RAC Rolling Installable.
  • This patch is Data Guard Standby-First Installable.

IMPORTANT:
This patch contains a security fix due to which a SELECT query’s plan MAY change under the following conditions:

  • The SELECT queries a table protected with a Fined Grained Auditing policy
  • And the policy condition is NULL

Refer to My Support notice for more information:

  • Bug 17027533 – 12.1.0.1.1 (Oct 2013) Database Patch Set Update (PSU) [ID 17027533.8]
  • Oracle Database Patch Set Update 12.1.0.1.1 Known Issues (Doc ID 1571651.1)
  • Bug 16902043 – 11.2.0.3.8 (Oct 2013) Database Patch Set Update (PSU) [ID 16902043.8]
  • Oracle Database Patch Set Update 11.2.0.3.8 Known Issues (Doc ID 1571650.1)
  • Patch Set Update and Critical Patch Update October 2013 Availability Document (Doc ID 1571391.1)

The new release, new Bugs here the list of the Multitenant Bugs, which has been fixed:

16427054 - SR12.1.0.2PX_HYBRID_LOAD - TRC - KPDBIDTONAME
16443657 - CDB: OCITRANSCOMMIT() IS ABLE TO COMMIT FROM WRONG CONTAINER
16457621 - W2K8_12.1_CDB: ORA-600 [KKAEGEN_GET_EDITION_NAME_3] TERMINATE INSTANCE
16459685 - CDB (NON RAC) : ORA-44310 AND ORA-07445:[KSPGIP()+106] [SIGSEGV]
16483559 - CDB:COMMON USER NOT SYNCED ON PDB OPEN WITH FORCE OPTION
16485876 - FIRE LOGON TRIGGER DOING DDL IN OTHER CONTAINER GIVES ORA-600 [KTSSCNI1],
16603924 - XSTRM CDB W/ UPG'D PDBS, CREATE_OUTBOUND => ORA-600 [KKAEGEN_GET_EDITION_NAME_1]
16660558 - CDB: ORA-7445 [KSFD_IO] & [KSLWS_DMP_SESS_WAITSTACK] IN CREATE PDB UNDO CALLBACK
16663303 - SR12.1UPD-PLUGIN:DBMS_EDITIONS_UTILITIES FAILS WITH ORA-38817: INSUFFICIENT PRIV
16663465 - SR12.1UPD-PLUGIN -TRC -ORA-600KKDLGETBASEUSER2:AUTHIDTYPE/ORA-04024 SELF-DEADLOC
16675710 - CDB: ORA-7445 [KSUSDIINPROGRESS()+47] [SIGSEGV] [ADDR:0X18] [PC:0XB35190F]
16689109 - INVALID OBJECTS OCCUR WHEN UPGRADING IN A CDB
16697600 - CDB: "ALTER PDB ALL OPEN INSTANCES = ALL" ERRORS IF ALL OPEN ALREADY
16698577 - FA + REDACT: ORA-10387 AND ORA-600 [KGLRELEASEHANDLEREFERENCE1]
16705020 - LNX64-12.1-CDB: HIT ORA-7445 [KSP_PDB_SPFILE_INSERT] WHEN CREATING PDB
16707927 - PKT : ORA-600 [2130] - TRC - KCCUGG
16712618 - CDB:ORPHANED USER CAN BE UNLOCKED
16715647 - CDB-ADG:RESTRICTED OPEN FORCE FOR MULTIPLE PDBS DOES NOT WORK
16730813 - CDB:ORA-65144 WHEN DISABLING RESTRICTED SESSION IN ROOT
16772060 - TT12.1SQLFUZZ2: DBMS_PDB.SYNC_PDB THROWS ORA-600 [KGHSTACK_UNDERFLOW_INTERNAL_1]
16784167 - CDB(NON-RAC):ORA-00600: INTERNAL ERROR CODE, ARGUMENTS: [2801], [], [], [], [],
16784901 - TT12.1SQLFUZZ2: DESC ON RECREATED SUPPLIED OBJ THROWS ORA-7445 [KQLPRFD()+121]
16795944 - TT12.1SQLFUZZ2: PDB OPEN CRASHES AFTER LICENSE_MAX_USERS SET TO LESS THAN CURREN
16825779 - COMMON PROFILE RESOURCE LIMIT FOR PASSWORD VERIFY FUNCTION DISPLAYS FROM ROOT
16836849 - PHSB: CDB ORA-00600:[KTCALLOCXCB] DURING DB OPEN
16859937 - CDB: LOCAL USER CONVERTED TO COMMON
16902138 - RAC: ORA-7445 [RPIDRV] AFTER DROPPING A PDB
16921340 - CDB EXIT: NON-CDB TO PDB PLUGIN HANGS
16935643 - GOT ORA-600 [KQLUDP2] , [0X16EB753A8], [4] WHILE UPGRADING A PDB
16946613 - TT12.1SQLFUZZ2: CMN USER WHICH WAS LOCAL PRIOR TO SYNC NOT SYNCED ON PDB OPEN
16946990 - UNABLE TO INSTALL APEX IN LOCAL PDB 12C DATABASE.RAISES ORA-65050
16993424 - CDB: ORA-600 [KKAEGEN_GET_EDITION_NAME_3], [3]
16994576 - CDB(NON-RAC): ORA-600 [KQLBBOTADD:3]
17000176 - HANG INSIDE CATCDB_INT.SQL CALLED USING CATCON.PL
16911800 - Fix for bug 16911800
16919176 - Fix for bug 16919176

Oracle RAC with OCR mirrored by ASM pitfalls


Since 11gR2 OCR and Voting files can placed into ASM. Here an overview:

ASM Reduncancy Level  OCR Mirrors  Votingdisks  Failgroups  Min. count of disks
EXTERNAL 1 1 1 1
NORMAL 2 3 3 3
HIGH 3 5 5 5

The most common configuration of RAC is a 2 node RAC. For example Oracle database applicance.
If a RAC has one storage the will be no problem and ASM mirror is not nessecary, EXTERNAL redundancy can be used. If the RAC has two storages there is a problem with the Votingdisks. Therefore a third location will be needed. In most configurations two locations are expensive enough and a third location is not available.

Here a real situation:
Customer has two nodes and two storages. All files are mirrored on ASM with normal redundancy, also OCR diskgroup:
Site A: 2x voting, secondary RAC node
Site B: 1x voting, master RAC node

Suddenly site A will break down due to a site desaster. Some seconds later the RAC node on Site B will shutdown due to OCR errors.
Why does this happens:
ASM mirroring is done at block/extent level.

  • EXTERNAL mirroring does mean no mirror
  • NORMAL = extent will be located in one other failgroup
  • HIGH = extent will be located in two other failgroups

Build up on the count of disks needed and the mirror copies witch should be used?

  1. EXTERNAL = not usable for two storages
  2. NORMAL = 3 Disks with 3 voting disks and an OCR mirror
    therefore segmentation of disks is 2:1, but only 2 mirrored blocks of OCR maybe all OCR blocks are on Site A
  3. HIGH = 5 Disks with 5 voting disks and an HIGH OCR mirror
    therefore segmentation of disks is 3:2, but only 3 mirrored blocks of OCR maybe all OCR blocks are on Site A

So what can be done. The solution: NORMAL redundancy with HIGH redundancy OCR mirror. The following construct will be created:
Disk segmentation is 2:1 and 3 mirrored block of OCR. All blocks of the OCR will be mirrored on every Disk. Whatever witch Site on the disaster will happen, at least one OCR mirror copy will be available.

Here the demonstration on 12.1.0.1 GI:
1. Create Cluster with normal redundancy cluster diskgroup DG_CLUSTER
2. Check asm template of OCR asm diskgroup:

SQL> select * from v$asm_template where group_number=1;

GROUP_NUMBER ENTRY_NUMBER REDUND STRIPE S NAME                           PRIM MIRR     CON_ID
------------ ------------ ------ ------ - ------------------------------ ---- ---- ----------
           1          123 MIRROR COARSE Y VOTINGFILE                     COLD COLD          0
           1          343 MIRROR COARSE Y OCRFILE                        COLD COLD          0

3. Check mirror on OCR:

ASMCMD> ls -l +DG_CLUSTER/vmsvr-clu2/OCRFILE
Type     Redund  Striped  Time             Sys  Name
OCRFILE  MIRROR  COARSE   JUL 23 23:00:00  Y    REGISTRY.255.821572803

4. Check ASM extent distribution

SQL>select g.name
2          ,d.path
3          ,e.XNUM_KFFXP extent
4          ,decode(e.lxn_kffxp,0,'primary',1,'mirror-normal','mirror-high') mirrormeta
5  from x$kffxp        e
6        ,v$asm_alias  a
7        ,v$asm_disk   d
8        ,v$asm_diskgroup  g
9  where e.number_kffxp=a.file_number
10       and e.disk_kffxp=d.disk_number
11       and d.group_number = g.group_number
12       and a.name='REGISTRY.255.821572803'
13*      order by 3,4 desc

NAME                         PATH                               EXTENT MIRRORMETA
---------------------------------------------------------------------- -------------
DG_CLUSTER                   ORCL:ORA_DISK_2                         0 primary
DG_CLUSTER                   ORCL:ORA_DISK_1                         0 mirror-normal
DG_CLUSTER                   ORCL:ORA_DISK_3                         1 primary
DG_CLUSTER                   ORCL:ORA_DISK_1                         1 mirror-normal
DG_CLUSTER                   ORCL:ORA_DISK_1                         2 primary
DG_CLUSTER                   ORCL:ORA_DISK_3                         2 mirror-normal
DG_CLUSTER                   ORCL:ORA_DISK_2                         3 primary
DG_CLUSTER                   ORCL:ORA_DISK_3                         3 mirror-normal
DG_CLUSTER                   ORCL:ORA_DISK_3                         4 primary
DG_CLUSTER                   ORCL:ORA_DISK_2                         4 mirror-normal
DG_CLUSTER                   ORCL:ORA_DISK_1                         5 primary
DG_CLUSTER                   ORCL:ORA_DISK_2                         5 mirror-normal
DG_CLUSTER                   ORCL:ORA_DISK_2                         6 primary
DG_CLUSTER                   ORCL:ORA_DISK_1                         6 mirror-normal
DG_CLUSTER                   ORCL:ORA_DISK_3                         7 primary
DG_CLUSTER                   ORCL:ORA_DISK_1                         7 mirror-normal
DG_CLUSTER                   ORCL:ORA_DISK_1                         8 primary
DG_CLUSTER                   ORCL:ORA_DISK_3                         8 mirror-normal
DG_CLUSTER                   ORCL:ORA_DISK_2                         9 primary
DG_CLUSTER                   ORCL:ORA_DISK_3                         9 mirror-normal
DG_CLUSTER                   ORCL:ORA_DISK_3                        10 primary
DG_CLUSTER                   ORCL:ORA_DISK_2                        10 mirror-normal
...

As you can see the diskgroup is made up of 3 disks (ORA_DISK_1 – 3). Further there are only two mirrors of each extent.

5. Backup OCR

[root ~]# ocrconfig -manualbackup
vmsvredu3     2013/07/24 23:51:17     /opt/oracle/12.1/grid/cdata/vmsvr-clu2/backup_20130724_235117.ocr
vmsvredu3     2013/07/23 22:52:10     /opt/oracle/12.1/grid/cdata/vmsvr-clu2/backup_20130723_225210.ocr
vmsvredu3     2013/07/23 22:45:11     /opt/oracle/12.1/grid/cdata/vmsvr-clu2/backup_20130723_224511.ocr

6. To correct this problem if don’t have an OCR mirror, stop cluster and start one node exclusiv and without crsd

crsctl start crs -excl -nocrs

7. Change asm template

SQL> alter diskgroup dg_cluster modify template OCRFILE attributes (HIGH);

Diskgroup altered.

SQL> select * from v$asm_template where group_number=1;

GROUP_NUMBER ENTRY_NUMBER REDUND STRIPE S NAME                           PRIM MIRR     CON_ID
------------ ------------ ------ ------ - ------------------------------ ---- ---- ----------
           1          120 MIRROR COARSE Y PARAMETERFILE                  COLD COLD          0
           1          121 MIRROR COARSE Y ASMPARAMETERFILE               COLD COLD          0
           1          123 MIRROR COARSE Y VOTINGFILE                     COLD COLD          0
           1          124 MIRROR COARSE Y DUMPSET                        COLD COLD          0
           1          125 HIGH   FINE   Y CONTROLFILE                    COLD COLD          0
           1          126 MIRROR COARSE Y FLASHFILE                      COLD COLD          0
           1          127 MIRROR COARSE Y ARCHIVELOG                     COLD COLD          0
           1          128 MIRROR COARSE Y ONLINELOG                      COLD COLD          0
           1          129 MIRROR COARSE Y DATAFILE                       COLD COLD          0
           1          230 MIRROR COARSE Y TEMPFILE                       COLD COLD          0
           1          231 MIRROR COARSE Y BACKUPSET                      COLD COLD          0
           1          232 MIRROR COARSE Y XTRANSPORT BACKUPSET           COLD COLD          0
           1          233 MIRROR COARSE Y INCR XTRANSPORT BACKUPSET      COLD COLD          0
           1          234 MIRROR COARSE Y AUTOBACKUP                     COLD COLD          0
           1          235 MIRROR COARSE Y XTRANSPORT                     COLD COLD          0
           1          237 MIRROR COARSE Y CHANGETRACKING                 COLD COLD          0
           1          238 MIRROR COARSE Y FLASHBACK                      COLD COLD          0
           1          239 MIRROR COARSE Y KEY_STORE                      COLD COLD          0
           1          340 MIRROR COARSE Y AUTOLOGIN_KEY_STORE            COLD COLD          0
           1          341 MIRROR COARSE Y AUDIT_SPILLFILES               COLD COLD          0
           1          342 MIRROR COARSE Y DATAGUARDCONFIG                COLD COLD          0
           1          343 HIGH   COARSE Y OCRFILE                        COLD COLD          0

22 rows selected.

SQL>

8. Remove old OCR

ASMCMD> ls -l
Type     Redund  Striped  Time             Sys  Name
OCRFILE  MIRROR  COARSE   JUL 24 10:00:00  Y    REGISTRY.255.821572803
ASMCMD> rm -f REGISTRY.255.821572803

9. Restore OCR

[root ~]# ocrconfig -restore /opt/oracle/12.1/grid/cdata/vmsvr-clu2/backup_20130724_235117.ocr

10. Check new OCR

ASMCMD> ls -l
Type     Redund  Striped  Time             Sys  Name
OCRFILE  HIGH    COARSE   JUL 24 10:00:00  Y    REGISTRY.255.821615711
ASMCMD>

11. Check crsd starts

[oracle ~]$ crsctl start res ora.crsd -init
CRS-2672: Attempting to start 'ora.crf' on 'vmsvredu3'
CRS-2672: Attempting to start 'ora.storage' on 'vmsvredu3'
CRS-2676: Start of 'ora.storage' on 'vmsvredu3' succeeded
CRS-2676: Start of 'ora.crf' on 'vmsvredu3' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'vmsvredu3'
CRS-2676: Start of 'ora.crsd' on 'vmsvredu3' succeeded

12. Restart cluster normal

[root ~]# crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'vmsvredu3'
CRS-2673: Attempting to stop 'ora.crsd' on 'vmsvredu3'
CRS-2677: Stop of 'ora.crsd' on 'vmsvredu3' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'vmsvredu3'
CRS-2673: Attempting to stop 'ora.evmd' on 'vmsvredu3'
CRS-2673: Attempting to stop 'ora.storage' on 'vmsvredu3'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'vmsvredu3'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'vmsvredu3'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'vmsvredu3'
CRS-2677: Stop of 'ora.storage' on 'vmsvredu3' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'vmsvredu3'
CRS-2677: Stop of 'ora.drivers.acfs' on 'vmsvredu3' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'vmsvredu3' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'vmsvredu3' succeeded
CRS-2677: Stop of 'ora.evmd' on 'vmsvredu3' succeeded
CRS-2677: Stop of 'ora.asm' on 'vmsvredu3' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'vmsvredu3'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'vmsvredu3' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'vmsvredu3' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'vmsvredu3'
CRS-2677: Stop of 'ora.cssd' on 'vmsvredu3' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'vmsvredu3'
CRS-2677: Stop of 'ora.crf' on 'vmsvredu3' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'vmsvredu3'
CRS-2677: Stop of 'ora.gipcd' on 'vmsvredu3' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'vmsvredu3' has completed
CRS-4133: Oracle High Availability Services has been stopped.

[root@vmsvredu3 ~]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root ~]#

13. Check ASM mirroring again:

SQL> select g.name
  2          ,d.path
  3          ,e.XNUM_KFFXP extent
  4          ,decode(e.lxn_kffxp,0,'primary',1,'mirror-normal','mirror-high') mirrormeta
  5  from x$kffxp        e
  6        ,v$asm_alias  a
  7        ,v$asm_disk   d
  8        ,v$asm_diskgroup  g
  9  where e.number_kffxp=a.file_number
 10        and e.disk_kffxp=d.disk_number
 11        and d.group_number = g.group_number
 12        and a.name='REGISTRY.255.821615711'
 13        order by 3,4 desc
 14  ;

NAME                           PATH                               EXTENT MIRRORMETA
------------------------------ ------------------------------ ---------- -------------
DG_CLUSTER                     ORCL:ORA_DISK_1                         0 primary
DG_CLUSTER                     ORCL:ORA_DISK_3                         0 mirror-normal
DG_CLUSTER                     ORCL:ORA_DISK_2                         0 mirror-high
DG_CLUSTER                     ORCL:ORA_DISK_2                         1 primary
DG_CLUSTER                     ORCL:ORA_DISK_3                         1 mirror-normal
DG_CLUSTER                     ORCL:ORA_DISK_1                         1 mirror-high
DG_CLUSTER                     ORCL:ORA_DISK_3                         2 primary
DG_CLUSTER                     ORCL:ORA_DISK_1                         2 mirror-normal
DG_CLUSTER                     ORCL:ORA_DISK_2                         2 mirror-high
DG_CLUSTER                     ORCL:ORA_DISK_1                         3 primary
DG_CLUSTER                     ORCL:ORA_DISK_3                         3 mirror-normal
DG_CLUSTER                     ORCL:ORA_DISK_2                         3 mirror-high
DG_CLUSTER                     ORCL:ORA_DISK_2                         4 primary
DG_CLUSTER                     ORCL:ORA_DISK_3                         4 mirror-normal
DG_CLUSTER                     ORCL:ORA_DISK_1                         4 mirror-high
DG_CLUSTER                     ORCL:ORA_DISK_3                         5 primary
DG_CLUSTER                     ORCL:ORA_DISK_1                         5 mirror-normal
DG_CLUSTER                     ORCL:ORA_DISK_2                         5 mirror-high
DG_CLUSTER                     ORCL:ORA_DISK_1                         6 primary
DG_CLUSTER                     ORCL:ORA_DISK_3                         6 mirror-normal
DG_CLUSTER                     ORCL:ORA_DISK_2                         6 mirror-high
...

All done. Now a disaster can come.

References: