Application Notes

This section includes notes about certain ‘applications’ that have been encountered in the SavWareHA installations to date. ‘Applications’ refers to both hardware and software devices and programs. We invite the reader to e-mail your own unique ‘Application Notes’ to ‘SavWareHA@avnet.com’ if they might assist others in implementing SavWareHA.

 
SavWareHA Configuration Worksheet

In planning a SavWareHA installation, we have found that the following form has helped to identify major components, configuration needs, and serves as a hard copy document of the installation. It is placed on a sheet by itself for your use.

SavWareHA Configuration Worksheet Customer: System Serial #:

Mirror #0:		Mirror #1:
Primary Device: Mounts as:	Standby Device:	Primary Device: Mounts as:	Standby Device:

 
SCO LLI Drivers

Only recommended for servers running SavWareHA version 2.04c or earlier on Intel 486 or slower processors.

SavWareHA can be configured to use the SCO LLI Drivers to improve the performance of the data link, eliminate the long TCP/IP timeout in the case of a data link failure and reduce the CPU overhead required for TCP/IP transfers. Use the following procedure:

Issue the command ‘llistat’ from the ‘root’ account on both the Primary and Standby systems and note the device name of the LLI network device. Using a text editor, modify the file ‘/etc/default/netdisk ’ . Add the LLI device names found previously of the data link network card on the Primary and Standby System to the device line.

For example, change:

standby:/dev/u -      0     0      0

To:

standby:/dev/u -      0      0      0 /dev/wdn0     /dev/wdn0

NOTE: Use [TAB] between each field.

Shutdown and reboot the Primary System for the change to take effect. This change has no meaning or effect on the Standby System .

If you notice performance degradation and possibly many ‘<defunct>‘ processes, then the passing of UDP’s over the LLI link has become unreliable. Revert to the TCP/IP method, or investigate you cabling / network cards closely.

When a second network is defined to SCO, a separate Internet address and machine name are used to identify the network.

It is recommended that the network card for the data link on the primary system be named “neptune” and the network card for the data link on the standby system be named “pluto”. This must be done with netconfig before SavWareHA is configured.

Install SavWareHA as usual: Specify “neptune” and “pluto” as the system names.

This will route data over the data link network card when SavWareHA starts up.

On Line Backups with ‘XA ’

Occasionally, a client requires 100% uptime, but also wants checksum verified backups each night. This is a common, but often overlooked, need for SavWareHA customers. It also points out another benefit of having SavWareHA installed on a mission-critical system.

The SavWareHA package is NOT a substitute for frequent, checksum-verified data backups to removable media, such as those provided by ‘XA ’ from Avnet Technology Solutions. Please secure a copy of ‘XA’ if you do not already have one for all of your sites, SavWareHA or not. This discussion will center around the scripts needed to implement ‘XA’.

The idea here is to form a script command file that will be called from ‘cron’ on the Primary System. The script file will issue commands to both the Primary and Standby Systems to:

1. Turn off the slave side of the mirror;

2. Clean and mount (read-only) the slave side of the mirror;

3. Fire off ‘XA’ from the command line;

4. Unmount the slave side and start regeneration.

The following script segment makes several assumptions, as listed below. Please keep in mind that any of the assumptions may be subject to change on particular systems:

• Mirrored Unit is mirror #0, the first mirror
• The Standby System is named ‘standby’
• Slave sub-component is named ‘/dev/u’
• Slave sub-component can be mounted on directory ‘/u’
• A backup of the entire Standby System is desired
• Output redirection would be specified by the calling routine

Given the above assumptions, the script might look similar to the following:

# /usr/local/bin/backup.slv - Backup Slave Mirror

# Turn off slave side, mirror #0:

/etc/mirror -m0 -s1

# Cause slave sub-component to be cleaned:

rcmd standby /etc/fsck -y -s /dev/u

# Cause slave sub-component to be mounted, read only:

rcmd standby /etc/mount -r /dev/u /u

# Cause ‘XA’ to fire off and backup entire Standby System:

rcmd standby /usr/xa/bin/start.backupx

# Now that ‘XA’ has finished (any result), unmount slave:

rcmd standby /etc/umount /dev/u

# And bring mirror back into compliance (note CAPITAL ‘G’):

/etc/mirror -m0 –G

 
MRACDB Root DataBase Consistency

Available on SavWareHA Release 2.02f and later.

The program /etc/mracbd is provided in the directory /etc/sentinel.d to allow important configuration information on the root filesystem of the Primary system to be periodically copied to the Standby system. This allows the system administrator to make changes to the Primary system, such as adding a new user , and have those changes made automatically on the Standby system.

The mracdb facility is not configured during a normal installation. The installer must copy the file mracdb from /etc/sentinel.d to /etc and then run the program /etc/mracdb. This first time this program is executed it will add the proper entry to /etc/crontab for the periodic scheduling. It will also create the file /etc/sentinel.d/cdb which contains a list of files that will be copied from the Primary system to the Standby system. The default file will contain the basic entries required to maintain users , passwords and permissions . Additional entries can be made to the file to provide additional ease of maintenance.

The mracdb functionality can be disabled at any time by using the command /etc/mracdb -r from the root account.

 
Switching Network Devices (IP Matching)

Applies to SCO Unix Release 3.2v4.x, ODT Release 3 and OpenServer5.

Note: Please review the complete capabilities of this technique and of IP Aliasing to determine which technique will work better in your application. Typically IP Aliasing is a better technique.

Available on SavWareHA Release 2.02f and later.

The normal installation of SavWareHA allows for the switching of serial devices in the event of a fallback. This application note describes the configuration changes that can be made to allow for network devices , such as PCs or X-Terminals , to be automatically switched between systems in the event of a fallback.

Requirements and Limitations

This application note only applies to SCO Unix Release 3.2v4.x and ODT Relase 3.

There must be at least two network links between the Primary and Standby system. One dedicated Data Link for mirroring disk data and one dedicated IO Link for connecting user PCs and other network devices.

In addition, there must be a serial link between the Primary and Standby system. This will be used by the monitor program to pass control information between the systems. Note that the IO Link can not be used for the monitor program because it will be disabled on the system that is not “in control”.

The IO Link will be disabled on the system that is not “in control”. This means that in normal mode the IO Link on the Standby system is disabled. This will prevent any network functionality (i.e. rlogin, rcp, etc..) on that network interface. Note that network functionality will be available to the Standby system on the Data Link.

Theory of Operation

The MAC (Hardware Address) of the IO Link network interface card (NIC) for both systems will be set to the same pre-defined value and the interfaces will be disabled during the boot to multi-user mode (/etc/tcp ). Later during the boot sequence, the program /etc/sentinel.d/S01tcpa will determine which system is “in control” and enable that IO Link interface at a common IP address and with a common system name. The system that is not “in control” will remain connected to the IO Link network with it’s network disabled.

In the case of a fallback, the Primary system will disable the IO Link interface (if possible) and the Standby system will enable it’s IO Link interface at the same IP address and system name that the Primary had previously used. Any connection oriented sessions that were already established will be lost. New connections can be made from a PC or X-Terminal to the Standby with no configuration changes because the Standby now has the same address that the Primary used to have.

Installation

1. Create the file /etc/sentinel.d/iolink on both systems. It should be created as a text file that only contains the network interface name of the IO Link (i.e. wdn0).

2. Run the following program on both systems. This will modify /etc/tcp.

/etc/sentinel.d/tcpopts -install

The first hook into /etc/tcp sets the MAC address to a predefined value before any of the TCP daemons start so they will see the correct MAC address.

The second hook immediately disables the iolink before any service daemons are started.

3. Important Note: The interface for the Data Link must be listed before the interface for the IO Link in the file /etc/strcf . The installer may have to manually modify the file /etc/strcf to insure this sequencing.

4. The IP address and system name configured in /etc/hosts and in /etc/tcp for the IO Link must be set to the same value on both the Primary and Standby systems.

5. The systems names listed in /etc/default/netdisk, /etc/sentinel.d/roles and /.rhosts on both systems must be modified to refer to the Data Link instead of the IO Link.

6. Reboot both the Primary and Standby systems.

7. Use the common IP address and system name when referring to the SavWareHA system from a network device.

 
Switching Network Devices (IP Aliasing)

Applies to SavWareHA for OpenServer5 Release 2.04c and greater and SavWareHA for UnixWare 7 release 2.05x6 and greater.

IP Aliasing is a simpler form of network device switching. It uses a new (third) IP number that always floats to the system in control.

Configuration is done by creating the file /etc/sentinel.d/ipalias on both systems with a line that has the floater IP number and rebooting both systems. This number must be on one of the subnets configured on the system (See Network field in the netstat –in command).

SavWareHA will always leave each machine at its own IP address, and just add / delete the floater IP address to the system in control. This means that both machines are always accessible to the network via their fixed IP address. In addition, a user can access the IP Alias number and automatically connect to the system in control.

Example:

Result of netstat –in command:

Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll

Net0 1500 192.168.20 192.168.20.7 171 0 171 0 0

Use a text editor to add the following entry into /etc/sentinel.d/ipalias and reboot

192.168.20.207

You can then access the system as 192.168.20.7. If it is the system in control, you can also access it as 192.168.20.207.

 
Using an Equinox SST with SavWareHA

The Equinox SST can be used for serial ports that must be switched between two computers under the control of SavWareHA.

Requirements:

SST host cards, one for each computer.
Equinox SST Driver with redundancy (Rel 2 or higher).
Equinox SST redundancy cable.
External power supply for SST panel.
SavWareHA revision 2.02f or greater.

There must be a power supply for the SST pods, as power is normally derived from the primary system through the original cable. Power can not come from the standby system through the redundancy cable, so it must be supplied from the external power supply.

After installing the SST driver on both systems, you must edit the file /etc/conf/pack.d/eqnx/space.c. The line of int EQNft must be changed from ‘=0’ to’ =1’. After the changes have been made, the kernel must be rebuilt and the system rebooted.

The file /etc/sentinel.d/X65eqnx must be copied to /etc/sentinel.d/S65eqnx.

The file /etc/sentinel.d/S65eqnx must be linked to /etc/sentinel.d/K65eqnx.

If more than one “SST Chain” is established between the systems then the file /etc/sentinel.d/S65eqnx must be modified to include additional lines that call /etc/eqnx/sstft with the -s and -q options to start and stop the interfaces. The parameter specified for -s and -q must be the TTY device name of the first TTY device on the SST chain.

The file /etc/sentinel.d/terminals contains a list of TTY device names that will be enabled during a switchover.

 
S75inittab

S75inittab speeds the process of enabling and disabling terminals during a switchover. It should be used in place of /etc/sentinel.d/terminals when more than three terminals need to be switched.

/etc/sentinel.d/S75inittab will cause the file /etc/inittab to be recreated from the kernel environment each time the system is booted.

/etc/sentinel.d/S75inittab is enabled by default on SavWareHA release 2.03a.

The device driver names for all serial multiplexor devices that are to be switched must be manually entered into the “SHARED_DEV” line in /etc/sentinel.d/S75inittab and a line must be added to /etc/sentinel.d/cdb for the switchover to work properly. (mracdb must have been previously configured).

The device driver name is the name of the file in the /etc/conf/init.d directory that contains the inittab entries for the devices.

A new inittab file is created for each system during the system boot or switchover. The new inittab that is created will have all of the “shared devices” turned off on the system that is not in control.

This functionality can be disabled by renaming /etc/sentinel.d/S75inittab to /etc/sentinel.d/X75inittab and removing /etc/sentinel.d/K32inittab on both systems.

 
SavWareHA Release Notes 2.03a

A problem has been corrected that would cause a SavWareHA system that was configured to operate in LLI mode to revert back to TCP/IP mode in certain circumstances. This could have caused the system to hang during the long TCP/IP time-out in the event of a standby or data link failure.

A positive indication is now available to allow the system administrator to determine if the data link is operating in TCP/IP mode or LLI mode. This is displayed in the SavWareHA mirror menu option “Display Mirror Status”. The indication is only displayed when the data link is in use (Both sides of the mirror are active).

The file /etc/inittab is now regenerated during each system boot from the kernel environment files in /etc/conf/init.d and /etc/conf/cf.d/init.base. Any changes to /etc/inittab that are not also reflected in the corresponding kernel environment files will be lost during the system boot. Previously, the file /etc/inittab was only regenerated when a new kernel was linked. This functionality is accomplished by /etc/sentinel.d/S75inittab and can be disabled by renaming the file to /etc/sentinel.d/X75inittab or removing the file.

Monitor Link Failure Notifications are now displayed on the console and a message is mailed to the system administrator(s) in addition to the previous action which was to record the event the log file

Data Link Failure Notifications are now displayed on the console and a message is mailed to the system administrator(s) in addition to the previous action which was to record the event in the log file.

A problem has been corrected that would have allowed a regeneration to a drive that had reads enabled. Regeneration is now prohibited to a drive with reads enabled.

A problem has been corrected that could have caused the system to force reads to a drive with an incomplete filesystem under certain failure conditions. Error and Warning messages were generated to the console in the previous versions. In this new release, a PANIC is generated and the primary system is halted under these circumstances to prevent possible data corruption.

The periodic monitor program has now been modified to check the system status every 2 hours. In addition, inconsistency notices are displayed on the system console as well as mail messages to the system administrator(s).

The system time and date on the standby system are now synchronized to the primary system.

A utility program has been added to the system that will accumulate all of the support logs and statistics from both systems into a single file that can be transmitted to customer support for review. The program is /etc/sentinel.d/makelog.

 
SavWareHA Release Notes 2.04b

Because of a change in the default service number of netdsk and smon, Upgrades to 2.04 from any/all lesser version must be done to both the primary and standby systems at the same time.

1. SavWareHA Release 2.04b dramatically improves the netdisk throughput running in TCP/IP mode on SCO OpenServer 5.0.x. Now running either TCP/IP or LLI mode we have seen from 500KB to 2.5MB / second throughput across a 100BaseT network. This is the throughput used for disk mirroring. LLI mode is preferred as it does not incur the CPU overhead that TCP/IP mode does. However, if the boxes are geographically separated and the data (mirror) link must be routed then use TCP/IP mode as LLI packets are not routeable.

2. Single install Diskette has both SCO 4.x and OpenServer 5 versions as well as Extended Monitor

3. Auto Recognize / Operation of Shared Disk Array (SDA)

4. Role Reversal of Sentinel System (Primary <<==>> Standby).

NOTICE: Important Information

Because of the ability to now do Role Reversal, it is best to not name the machines 'primary' and 'standby' or else you will confuse operators when the system named 'standby' becomes the primary system. Therefore, the new default names for the two systems is 'Pluto' and 'Neptune' (two planets that because of their elliptical orbit, alternate having the distinction of being the furthest planet from the sun).

5. Host equivalency not needed for normal operations or switch-over, only for (re-)configuration, or if using 'mracdb'.

6. Evaluation Period of Hot Root Mirror now works instead of displaying "Registration Time Expired".

7. Configure of Sentinel Mirrors automatically setup for LLI mode of operation for the NetDisk Data Link.

8. Configure of Sentinel Mirrors automatically setup to permit Role Reversal.

9. Command 'sentinel.sh -setslave' to setup to bring fresh Operating System on-line as a replacement standby system. Drive Partitions need to be pre-allocated but can't have the filesystems configured.

10. Command 'sentinel.sh -setlli' to setup for LLI mode of operation.

11. Command 'sentinel.sh -setrev'. Must be run in advance on the primary system to permit Role Reversal.

 
SavWareHA Release Notes 2.04c

1. Add IP Alias method for OpenServer 5

2. Correct problem with a false detection of a Shared Disk Array.

3. Correct problem with smon on serial ports after a role reversal.

4. Correct problem with smon not disarming if loss of network link.

 
SavWareHA Release 2.05x6 (UnixWare7)

Differences between SavWareHA for OpenServer5 and UnixWare7

1. Install procedure uses ‘pkgadd’ instead of ‘custom’.

2. IP Aliasing is used instead of IP Matching (See separate application note)

3. Device names for the mirrors have 3 digits instead of 2 (i.e /dev/mirror000 instead of /dev/mirror00).

4. MRACDB functionality is not currently supported.

5. Sub-division of a mirror into partitions is not currently supported.

6. EMON functionality is not currently supported

7. Use of F3 key to list available disk devices during install is not currently supported

8. The Equinox SST switchover functionality is not currently supported (Equinox Issue).