Abnormal System Shutdowns

In addition to being able to choose “dump everything” or “dump nothing,” as of HP-UX Release 11.0 you have the ability to determine which classes of memory pages get dumped.

You are reading this section because system recovery time is critical to you. Obviously, the fewer pages your system needs to dump to disk (and on reboot, copy to the HP-UX file system area), the faster your system can be back up and running. Therefore, if possible, avoid using the full dump option.

When you define your dump devices, whether in a kernel build or at run time, you can list which classes of memory must always get dumped, and which classes of memory should not be dumped. If you leave both of these lists empty, HP-UX will decide for you which parts of memory should be dumped based on what type of error occurred. In nearly all cases, this is the best thing to do.

	NOTE: Even if you have defined (in the kernel or at run time) that you do not want a full dump to be performed, an operator at the system console at the time of a crash can override those definitions and request a full dump. Likewise, if at the time of a crash you know what caused it (and therefore do not need the system dump), but have previously defined a full or selective dump, an operator at the system console at the time of a crash can override those definitions and request no dump be performed.

Compressed dump is a feature available on systems running HP-UX 11i version 1 (or subsequent releases). Following a system crash, the HP-UX operating system can use the feature to compress data from memory before it writes the data to the dump device. Compression decreases the volume of crash data, making the writes to disk faster.

By reducing the time required to store the entire dump, the recovery period is shorter and your system will be up and running much more quickly. Dump compression provides a greater time saving on systems that have large amounts of memory.

The following features are available in this version of dump compression:

The following restrictions apply to this release:

If either of the preceding conditions apply, the following messages are displayed at the console:

In HP-UX Release 11i version 1.0, compression is enabled in the kernel by default. You can disable (and enable) the feature by using the crashconf dump configuration utility. Use the -c option with the on argument to control the status of compressed dumps, as follows:

Use the -v option to examine the status of compressed dumps, as follows:

$ crashconf -v
{Lines omitted from display}
Dump compressed:     ON

You can disable compression by using the crashconf -c command with the off argument, as follows:

$ crashconf -v -c off
{Lines omitted from display}
Dump compressed:     OFF

Any changes that you make to the dump configuration take effect immediately but will persist only until the next reboot or the next invocation of the crashconf command. To make changes persist across reboots, use the -t option. To make changes persist across kernel rebuilds, use SAM or the kctune command.

In HP-UX 11i v2, you can use the persistent dynamic tunable dump_compress_on to set compression on or off as required. Set this tunable by using the crashconf command with the -t option.

Prior to HP-UX 11i v2, you can also edit the /etc/rc.config.d/crashconf initialization script to set the compression option for every subsequent reboot. Open this file with a text editor and modify the value of the CRASHCONF_COMPRESS variable to 1 (enable, on) or 0 (disable, off). If the CRASHCONF_COMPRESS variable does not exist in /etc/rc.config.d/crashconf, the default behavior is to compress the dump. Beginning with HP-UX 11i v2, this configuration is handled by the dynamic tunable dump_compress_on, and is not controlled by crashconf.

Programs can use the pstat_getcrashinfo() function to query the current crash dump configuration. See pstat(2) for more information. The psc_flags data field shows whether dump compression is enabled or not. For example:

/*
* This structure describes the system crash dump configuration.
* It is only available as 64-bit data (_PSTAT64 defined).
*/
struct pst_crashinfo {
  int64_t  psc_flags;	  /* Dump config. flags, see below */
  struct   __psdev psc_headerdev;    /* Device containing dump header */
  int64_t  psc_headeroffset;/* Byte Offset of dump hdr on device */
  int64_t  psc_ncrashdevs;  /* Number of dump devices */
  int64_t  psc_totalsize;   /*Total amount of dump space (kB) */
  int64_t  psc_included;    /* Page classes to be included */
  int64_t  psc_excluded;    /* Page classes to be excluded */
  int64_t  psc_default;     /*Defaults for unspec’d classes*/
  int64_t  psc_nclasses;    /* Number of classes */
  int64_t  psc_pgcount[PST_MAXCLASSES]; /* Number of pages in each class */
  int64_t  psc_reserved;               /* Reserved for future use */
};
/* Flag values for psc_flags: */
#define PS_EARLY_DUMP 0x1   /* An early dump was taken */
#define PS_CONF_CHANGED 0x2 /* Config. changed since boot */
#define PS_HEADER_VALID 0x4 /* headerdev and headeroffset valid */
#define PS_COMPRESS 0x8     /* Compress dump state*/

System dumps can be very large, so large that your ability to store them in your HP-UX file system area can be taxed.

The boot time utility called savecrash can be configured (by editing the file /etc/rc.config.d/savecrash) to compress or not compress the data as it copies the memory image from the dump devices to the HP-UX file system area during the reboot process. This has system recovery time implications in that compressing the data takes longer. So, if you have the disk space and require that your system be back up and running as quickly as possible, configure savecrash to not compress the data.

It is possible to use a specific device for both paging (swapping) and as a dump device. If system recovery time is critical to you, do not configure the primary paging device as a dump device. From the savecrash(1M) manpage:

Another advantage to keeping your paging and dump devices separate is that paging will not overwrite information stored on a dump device, no matter how long the system has been up or how much activity has taken place. Therefore, you can prevent savecrash processing at boot time (by editing the file /etc/rc.config.d/savecrash). This can save you a lot of time when you are trying to get your system back up in a hurry. After the system is up and running, you can run savecrash manually to copy the memory image from the dump area to the HP-UX file system area.

If a memory dump resides partially on dedicated dump devices and partially on devices that are also used for paging, you can choose to save (to the HP-UX file system) only those pages that are endangered by paging activity. Pages residing on the dedicated dump devices can remain there. If you know how to analyze memory dumps, it is even possible to analyze them directly from the dedicated dump devices using a debugger that supports this feature.

Before sending your memory dump to someone else for analysis, you must move the dumped pages from the dedicated dump devices to the HP-UX file system. You can then use a utility such as tar to bundle them up for shipment. To move the dumped pages, use the command /usr/sbin/crashutil to complete the copy instead of savecrash.

You have chosen this section because it is most important to you to capture the specific instruction or piece of data that caused your system crash. The only way to guarantee that you have it is to capture everything. This means selecting to do a full dump of memory.

Be aware, however, that this can be a costly procedure from both a time and a disk space perspective. From the time perspective, it can take quite a while to dump the entire contents of memory in a system with very large amounts of memory. It can take an additional large amount of time to copy that memory image to the HP-UX file system area during the reboot process.

From the disk space perspective, if you have large amounts of memory (some HP-UX systems can now have terabytes of memory), you will need an amount of dump area at least equal to the amount of memory in your system; and, depending on a number of factors, you will need additional disk space in your HP-UX file system area equaling the amount of physical memory in your system, in the worst case.

There are now a number of places that you can define which devices are to be used as dump devices:

Definitions at each of these places add to or replace any previous definitions from the other sources. However, consider the following situation:

	NOTE: The preceding example is presented for completeness. The actual amount of time between the point where kernel dump devices are activated, and the point where runtime dump devices are activated is very small (a few seconds), so the window of vulnerability for this situation is practically nonexistent.

It is possible to use a specific device for both paging purposes and as a dump device. But, if crash dump integrity is critical to you, this is not recommended. From the savecrash(1M) manpage:

So, if possible, avoid using a given device for both paging and dumping: particularly the primary paging device!

Systems configured with small amounts of memory and using only the primary swap device as a dump device are in danger of not being able to preserve the dump (copy it to the HP-UX file system area) before paging activity destroys the data in the dump area. Larger memory systems are less likely to need paging (swap) space during startup, and are therefore less likely to destroy a memory dump on the primary paging device before it can be copied.

You are reading this section because disk space is a limited resource on your system. Obviously, the fewer pages that you have to dump, the less space is required to hold them. Therefore, a full dump is not recommended. If disk space is very limited, you can always choose no dump at all.

However, there is a happy medium, and it happens to be the default dump behavior; it is called a selective dump. HP-UX can do a pretty good job of determining which pages of memory are the most critical for a given type of crash, and save only those. By choosing this option, you can save a lot of disk space on your dump devices, and again later, in your HP-UX file system area. For instructions on how to do this, see “Defining Dump Devices”.

Regardless of whether you choose to do a full or selective save, whatever is saved on the dump devices needs to be copied to your HP-UX file system area before you can use it.

	NOTE: Beginning with HP-UX Release 11.0, it is possible to analyze a crash dump directly from dump devices using a debugger that supports this feature (see the caution in the section called “Analyzing Crash Dumps”). But if you need to save it to removable media, or send it to someone, you will first need to copy the memory image to the HP-UX file system area.

If the disk space shortage on your system is in the HP-UX file system area (not in the dump devices), you can choose to have savecrash (the boot time utility that does the copy) compress your data as it makes the copy.

If you have plenty of dump device space but are limited on space in your HP-UX file system, you can use the -p option to the savecrash command. This command copies only those pages on dump devices that are endangered by paging activity (the pages residing on devices that are being used for both paging and dumping). Pages that are on dedicated dump devices are not copied.

To configure this option into the boot process, edit the file /etc/rc.config.d/savecrash and uncomment the line that sets the environment variable SAVE_PART=1.