Database Configuration

Additional Steps for Informix

Logon as root to the primary host of the database where the package is running in debug mode.

Perform the following steps as an INFORMIX user:

su - informix

Comment out the remsh sections in the files called .dbenv.csh and .dbenv.sh in the home directory. If they are missing, check for alternative files with hostnames in them:

.dbenv_<local>.csh and .dbenv_<local>.sh.

The remsh section looks similar to:

# remsh <local> date >& /dev/null 
# if ( $status <= 0 ) then 
#   echo Remote login check to dbserver <local> failed. 
#   echo Check if <local> is alive and file ... 
#   echo on <local> contains an entry for ... 
# endif 

Copy the two .dbenv files over to the INFORMIX home directory on all cluster nodes, all external application server hosts, and MC/ServiceGuard by using ftp.If you do not do this, switching time increases dramatically; each remsh command to the original database host causes delay if the node is not available.

In Step Title not available you copied two files to the INFORMIX home directory of the primary node. At this time, still as an INFORMIX user, customize these files by replacing the string relocdb with your individual <relocdbci> (or <relocdb> in case of two package concept). For example:

.customer.sh:

.customer.csh:

Copy the .customer.sh and .customer.csh to all INFORMIX home directories on all nodes including the application servers outside of the cluster.

Perform the following steps as <sid>adm user:

su - <sid>adm

Copy the files that were manipulated in Steps Title not available and Title not available to the home directory:

cp /home/informix/.dbenv* ~
cp /home/informix/.customer.* ~

Copy the files over to the home directories of <sid>adm on all cluster nodes and all external application server hosts.

Perform the following steps as an INFORMIX user.

Modify the parameters DBSERVERNAME and DBSERVERALIAS in the ONCONFIG file:

/informix/<SID>/etc/onconfig.<primary>.<DBID>.

The parameter default to:

<local><INFORMIX_DBID><shm|tcp> 

Set them to:

<relocdb><INFORMIX_DBID><shm|tcp> 

For example:

DBSERVERNAME relocdb<sid>shm 
DBSERVERALIASES relocdb<sid>tcp

Rename the ONCONFIG file to:

/informix/<SID>/etc/onconfig.<relocdb>.<DBID> 

Add a line with rel-IP-name of the database package to the file:

/informix/<SID>/etc/sqlhosts.soc. 

After SAP R/3 installation this file will be similar to:

demo_on onipcshm on_hostname on_servername 
demo_se seipcpip se_hostname sqlexec 
<local><sid>shm onipcshm <local> sapinf<SID> 
<local><sid>tcp onsoctcp <local> sapinf<SID> 

Change the <local> entries to <relocdb> of the database package. For example:

<relocdb><sid>shm onipcshm <relocdb> sapinf<SID> 
<relocdb><sid>tcp onsoctcp <relocdb> sapinf<SID>

If you are using SAP R/3 version 4.0x:

Create additional entries in the INSTVERS table of SAP R/3. Print out the content of the table. Note the ID column. This is the primary key of the table. All new entries need a unique key.

Review the STATUS column. There should be one row containing a 0 as status. This indicates the current, successful installation.

Copy this row using a unique key as well as secondary hostnames for HOSTNAME and DBHOSTNAME.

For the two package concept or more than two cluster nodes include entries for all possible combinations. For example, you need two additional entries for a two-package concept with two cluster nodes to reflect that the two packages can run on different nodes. Refer to Step Title not available for more information.

Additional Steps for Oracle

With the introduction of Oracle 8.i Enterprise a new 2-phase instance and crash recovery mechanism is available that enables a faster and predictable recovery time after a crash.

The instance and crash recovery is initiated automatically and consists of two phases:

Roll-forward phase: Oracle applies all committed and uncommitted changes in the redo log files to the affected datablocks. Following parameters can be used to tune the roll forward phase:

Roll-back phase: Oracle applies information in the rollback segments to undo changes made by uncommitted transactions to the data blocks. Following parameters can be used to tune the roll-back phase:

All these parameters can be used to tune the duration of Instance / Crash recovery.

More details on these useful new High-Availability features can be found in the Oracle 8.1.7 documentation "Oracle8i Designing and Tuning for Performance" (Chapter 24: Tuning Instance recovery Performance).

The following steps have to be performed in order to adjust the Oracle DB setting to the HA configuration.

Logon as root to the primary host of the database where the package is running in debug mode.

Perform the following step as <sid>adm.

To ensure that a database that crashed during an online backup starts correctly after the crash, all datafiles that were in 'begin backup' state need to be altered with an ''end backup' statement. Adjust the required steps in /sapmnt/<SID>/exe/startdb.

Therefore, 'startdb' needs to be adjusted accordingly. Insert / Change the following code within the '/sapmnt/<SID>/exe/startdb' file. The sample code can be found in the file: /opt/cmcluster/sap/SID/startdb.sql

# 
# Startup the database without changing the ARCHIVELOG state 
# 
echo "connect internal;" > $SRVMGRDBA_CMD_FILE 
echo "startup;" >> $SRVMGRDBA_CMD_FILE 
echo "exit;" >> $SRVMGRDBA_CMD_FILE 
eval $SRVMGRDBA command=@$SRVMGRDBA_CMD_FILE >> $LOG 2>&1

#
# Startup the database without changing the ARCHIVELOG state 
# alter datafile 'end backup' when instance crashed during 
# backup 
echo "connect internal;" > $SRVMGRDBA_CMD_FILE 
echo "startup mount;" >> $SRVMGRDBA_CMD_FILE 
echo "spool endbackup.log" >> $SRVMGRDBA_CMD_FILE 
echo "select 'alter database datafile '''||f.name||''' end backup;'" >> $SRVMGRDBA_CMD_FILE 
echo "from v\$datafile f, v\$backup b" >> $SRVMGRDBA_CMD_FILE 
echo "where b.file# = f.file# and b.status = 'ACTIVE'" >> $SRVMGRDBA_CMD_FILE 
echo "/" >> $SRVMGRDBA_CMD_FILE 
echo "spool off" >> $SRVMGRDBA_CMD_FILE 
echo "!grep '^alter' endbackup.log >endbackup.sql" >> $SRVMGRDBA_CMD_FILE 
echo "@endbackup.sql" >> $SRVMGRDBA_CMD_FILE 
echo "!rm endbackup.*" >> $SRVMGRDBA_CMD_FILE 
echo "alter database open;" >> $SRVMGRDBA_CMD_FILE 
echo "exit;" >> $SRVMGRDBA_CMD_FILE eval $SRVMGRDBA command=@$SRVMGRDBA_CMD_FILE >> $LOG 2>&1 

If you are using SAP R/3 version 3.x:

Perform the following steps as ora<sid>.

Configure the listener to listen on the relocatable name of the database package. To do this, change all references from <local> to the relocatable name <relocdb> in the files. For example:

/usr/sap/trans/listener.ora 
/usr/sap/trans/tnsnames.ora

If you are using SAP R/3 version 4.0A or higher:

Perform the following steps as ora<sid>.

Configure the listener to listen on the relocatable name of the database package. To do this, change all references from <local> to the relocatable name <relocdb> in the files on the shared volume group. Be careful if these files were customized after SAP R/3 installation. For example:

$ORACLE_HOME/network/admin/listener.ora 
$ORACLE_HOME/network/admin/tnsnames.ora 

If you are using SAP R/3 version 4.0A or higher:

Copy $ORACLE_HOME/network/admin/tnsnames.ora to all additional application server hosts. Be careful if these files were customized after SAP R/3 installation.

If you are using SAP R/3 version 4.0A or higher and you are using the two package concept:

As with all application servers, the Central Instance needs a subset of the files in $ORACLE_HOME to work properly. Verify if your Central Instance can run on any host without an application server installed. These hosts need a local copy of the filetree.

If you have access to the SAP R/3 install disks you can get the files from the SAP R/3 archive ORACLI.CAR or OCL<oclrel>.CAR on the kernel CD. The extractor program CAR functions the same as the tar(1m) command. It is shipped with SAP R/3.

If the Central Instance was installed on a node different from the database node, you can get the files there. Or you can use any additional application server host as source. Copy the filetree, for example, by using the procedure explained in Step Title not available. The filetree should contain the following subdirectories: lib, network ocommon, and rdbms.

Do not copy $ORACLE_HOME from the database host or you will copy much more than you want. Refer to SAP R/3 OSS Note 180430 for more details.

If you are using SAP R/3 version 3.x:

If existing locally, copy /etc/listener.ora to all adoptive nodes in the cluster.

If you are using SAP R/3 version 3.x and plan to use more than one SAP R/3 system inside of your cluster:

It is possible that more than one database is running on the same node. Even though one listener is capable of serving many database instances problems can occur in switchover environments because needed filesystems may not be available at the startup time of the listener. Use a dedicated listener process for each database.

You can use the standard listener.ora file that is created during the installation of one SAP R/3 system <SID1> as a template. Double its contents. For example:

cat listener.ora listener.ora >listener.ora.new
mv listener.ora.new listener.ora

Now the file consists of two identical parts.

Test your setup by starting the listeners as ora<sid1/2>:

lsnrctl start LISTENER<SID1/2>

Create an /etc/services entry for the new port you specified above. Use tlisrv<SID2> as service name. The name is not needed anyhow. This entry has to be made on all hosts that run an instance that belongs to the system. This includes all external application server hosts outside of the cluster.

If you use the two package concept:

Set the optional parameter SQLNET.EXPIRE_TIME in sqlnet.ora to a reasonable value in order to take advantage of the Dead Connection Detection feature of ORACLE. The parameter file sqlnet.ora resides either in /usr/sap/trans or in $ORACLE_HOME/network/admin.

The value of SQLNET.EXPIRE_TIME determines how often (in seconds) SQL*Net sends a probe to verify that a client-server connection is still active. If the Central Instance switches, the application servers may crash, thereby leaving shadow processes running on the database host. While the CI package cleans up the application server hosts, it does not touch the ORACLE shadow processes running on the database host. Remove them, because their number increases with every CI package switch. After an application server crash, a connection to the database shadow process may be left open indefinitely. If the SQLNET.EXPIRE_TIME parameter is specified, SQL*Net sends a probe periodically to determine whether there is an invalid connection that should be terminated. It finds the dead connections and returns an error, causing the server process to exit.

If you are using SAP R/3 version 4.0A or higher:

Perform the following step as <ora>sid. Start the listener if it is not already running:

lsnrctl start <listenername>

Start svrmgrl and type:

> connect internal;
> startup;
> select * from sapr3.instvers;

The output is similar to:

If you use one package concept with a two node cluster, enter the following line in the svrmgrl session and you are done:

> insert into sapr3.instvers values
(<key>,'40B',<SID>,<INSTNR>,'HP-UX',<sec_host>,
<sec_host>,'ORACLE','19980527','094252','N','0','MCSG',
'[RFCRADDBDIF_IND_IND]CRFCJOB-RADDBDIF');

For two package concept or more cluster nodes add entries for all possible combinations. For example, you need two additional entries for a two-package concept with two cluster nodes to reflect that the packages can run on different nodes:

> insert into sapr3.instvers values
  (<key>+1,'40B',<SID>,<INSTNR>,'HP-UX',<pri_host>,
  <sec_host>,'ORACLE','19980527','094252','N','0','MCSG',
  '[RFCRADDBDIF_IND_IND]CRFCJOB-RADDBDIF');
> insert into sapr3.instvers values
  (<key>+2,'40B',<SID>,<INSTNR>,'HP-UX',<sec_host>,
  <pri_host>,'ORACLE','19980527','094252','N','0','MCSG',
  '[RFCRADDBDIF_IND_IND]CRFCJOB-RADDBDIF');
> Refer to SAP OSS note number 144978 for more information about the INSTVERS table.

Stop the database before continuing.

Additional steps for Oracle 8.1.x DB:

For the SGeSAP <=V3.02 the sapbackup directory needs to be in $ORACLE_HOME/sapbackup. If you are using an Oracle 8.1.x DB the $ORACLE_HOME directory is set to /oracle/<SID>/8.1.x whereas the sapbackup directory is still placed /oracle/<SID>/sapbackup. It's best practice to create the symbolic link $ORACLE_HOME/sapbackup -> /oracle/<SID>/sapbackup.

Additional steps for Application server and Oracle 8.1.x DB:

If you run an Oracle DB >= 8.1.x and install additional R/3 Application Server follow the procedure described in OSS303238 to install the ORACLE client libraries.

If you configure multiple Application servers to be started with the "parallel" startup option in sap.conf, make sure the "tcp_conn_request_max" ndd parameter on the DB-nodes (primary and backup) are configured with an appropriate value:

Example: tcp_conn_request_max = 1024

If this parameter is set too low, incoming tcp connections from starting SAP Application servers that want a connect to the DB via the Oracle Listener may halt. This will hang the starting process of the SAP Application server.