Jump to content United States-English
HP.com Home Products and Services Support and Drivers Solutions How to Buy
» Contact HP
More options
HP.com home
HP-UX Event ManagerAdministrator's Guide: HP-UX 11i v3 Edition 1 > Chapter 2 Using Event Manager

Monitoring Events

» 

Technical documentation

Complete book in PDF
» Feedback
Content starts here

 » Table of Contents

 » Index

The following sections discuss the commands you can use to monitor and review event activity.

Displaying Events Using evmshow

An Event Manager event is a binary data package, because it must be converted to text before you can display it on a terminal. The evmshow command reads binary events from its stdin stream or from a named file, and outputs the same events in text form to stdout. To display the contents of a file containing Event Manager events, enter the following command at the HP-UX prompt:

 # cat my_events | evmshow | more

This command displays the events from the log file in the default format. It takes the format data item from each event, expands it with the values of any variables it references, and displays it. References to variables are identified by a dollar sign ($). Therefore, if the my_events file contains an event with a format data item of ProcSM:A category "$_catname" has been added , and the event also contains a variable named _catname with a value of esmd, the corresponding line of the output is as follows:

ProcSM : A category "esmd" has been added

This information indicates what happened. However, not when it happened, or the importance of the event. You can modify the output of the evmshow command to include any data items in the event, including its timestamp and priority, by using the -t option to specify a show-template. A show-template is a text string that indicates which data items you want to be displayed for an event, and how you want them to be displayed.

The following example illustrates the use of a show-template to display an event with a timestamp, a priority, and the formatted event message. In the show-template, the names of the items to be displayed are each preceded by an at sign (@) . Two at signs (@@) indicate that the event's format item must be expanded and displayed. The second line shows the output for the domain full event. In the output, the event priority is enclosed within brackets, and there are two spaces before the message text, exactly as specified in the following show-template:

 # cat my_events | evmshow -t "@timestamp [@priority]  @@" | 	 	 	 	 	 		 more
03-Aug-2006 21:06:14 [200]  ProcSM : A category "esmd" has been added

You can set up your own show-template to display the items that are important to you, in any format you want. For more information about the data items, see EvmEvent(5). After you determine your preferred style, you can set a default show-template in the environment variable EVM_SHOW_TEMPLATE and use fewer keystrokes at the command line. The following Korn shell (ksh) commands are equivalent to those in the previous example:

	 # export EVM_SHOW_TEMPLATE="@timestamp [@priority]  @@"
# cat my_events | evmshow | more

If you want more information about an event, you can request a detailed display, including an explanation and a full dump of its contents, by using the command with the -d option. The following example shows a detailed display of the category esmd add event:

 #cat my_events | evmshow -d  | more============================ EVM Log event ===========================EVM event name: sys.unix.procsm.category.create._catname.esmdThis informational event is posted by ProcSM module to indicate that a new category has been added to Process Set Manager.  [1]======================================================================
Formatted Message:ProcSM: A category "esmd" has been added.  [2] Event Data Items:    [3]  Event Name:sys.unix.procsm.category.create._catname.esmdPriority          : 200PID               : -1
PPID              : -1Event Id          : 28Timestamp         : 03-Aug-2006 21:06:14Format            : ProcSM: A category "$_catname" has been added. [4]Reference         : cat:evmexp.cat:2300Variable Items:   [5]
_catname (STRING) = "esmd"
 Where:
  1. The explanation of the event. In some cases, this data field contains a recommended action to rectify a problem.

  2. The Formatted Message section.

  3. The Event Data Items section, which lists all the standard data items contained in the event. For description of each of these items, see EvmEvent()

    The items shown here are typical of many events. However, some of these may be missing, or occasionally you may see additional items.

  4. The Format data item is almost the same as the content of the Formatted Message data item, but it includes a reference to a variable called _catname, indicated by the $ symbol preceding it.

  5. The Variable Items section, which contains the value of the domain variable.

For more information on how to select events for detailed display, see “Using the Event-Id to Select Events for Detailed Display”.

You can use the evmshow -x command to display the explanation alone. Alternatively, use the -x and -t options together to obtain a summary of the event followed by its explanation. For example:

	# cat my_events | evmshow -x -t "@timestamp [@priority]  @@"		 |	 more \
03-Aug-2006 21:06:14 [200] ProcSM : A category "esmd" has been added
This informational event is posted by ProcSM module to indicate that a new category has been added to Process Set Manager. 

You can display events that are stored in the various system log files, or monitor them as they occur by using the evmget and evmwatch commands. For more information about these commands, see “Retrieving Stored Events Using evmget” and “Monitoring Events”.

Most systems produce a large number of events, many of which report normal operation. Use event filters to limit the display to a set of events that you consider to be important. The section “Introduction to Event Filters” introduces the filtering facilities.

Regardless where the events come from, you can use the evmshow command to format them for display. For more information about show-template, see evmshow(1).

Retrieving Stored Events Using evmget

You can use the evmget command to obtain the following:

  • An ordered view by retrieving events from each of the various log files.

  • Convert the events to Event Manager events if they are not already in that form.

  • Return a single stream of Event Manager events.

Using the evmshow command, you can convert the Event Manager event stream into a display format.

The following command pipeline uses the evmget command to retrieve all system events, and passes them to the evmshow command for display:

 # evmget | evmshow -t "@timestamp [@priority]  @@" | more

The evmget command makes a service connection to the Event Manager daemon, which starts a new copy of the get-server program, /usr/sbin/evm_getsrv. The get-server program reads the channel configuration file, and runs the get function, usually a shell script, for each channel configured in the channel configuration file, /etc/evmchannel.conf. This configuration file is described in ““Configuring Event Manager Channel””.

The get function performs the following functions:

  • Reads the channel log file

  • Converts the events into EVM format

  • Feeds events back to the evmget command which writes them to its stdout stream

After all the channel get functions run and all the events are returned, both get-server daemon and the evmget command terminate.

NOTE: Though events may be stored in log files as lines of text, or in a special binary format, the evmget command returns all events in the form of binary events, which can be passed to evmshow for display. If you send the output of evmget directly to your terminal, the command displays an error message because the binary output cannot be displayed properly and can affect the settings of your terminal. If you pipe the output into another command, such as more or pg, the evmget command fails to detect the error, and random characters are displayed.

Similar to the evmshow command, the evmget command supports a filter option to enables you to limit the events it returns. For example, the following command displays only high-priority events:

	 # evmget -f '[pri >= 600]' | evmshow | more

It is more efficient to specify a filter with the evmget command than with the evmshow command. This is because the evmget command passes its filter string to the event channel get function, which only returns events that match the filter. Fewer events are passed back through the get-server daemon to the evmget command, and the commands operate faster because they transfer and process fewer events.

If you want to save retrieved events for later analysis, or to copy them to another system, you can redirect the output of the evmget command into a file. For example:

	 # evmget -f '[pri >= 600]' > my_events

At a later time, you can sort and filter the binary file and pass it to the evmshow command to view it in any format you like.

Each get function feeds its events back to the evmget command in turn, and the evmget command outputs them in the order in which it receives them. You must pipe the events using the evmsort command to view the events in a particular sort order. For more information about using the evmsort command, see “Sorting Events Using evmsort”.

“Using the -A Option to Simplify the Command String” introduces using the evmget command with the -A option, which makes it possible to retrieve, sort, and display events without building a pipeline.

Depending on the size and type of your system and the number of events being logged, event retrieval can take a noticeably long time. This is because each retrieval operation requires every channel's get function to read through its log files, convert its events to Event Manager events, and then apply the filter string (if any) to determine whether the event is passed back to the evmget command. The larger the log files, the longer this process takes. Careful log file management helps to speed up the process. If you know that you want to display events that belong to a particular event channel, you can shorten the process by using the evmget -C command to display only the specified channel. For example:

	 	# evmget -f '[pri >= 600]' -C evmlog | evmshow | more

In this example, the get function runs only on the evmlog channel. As a result, the command completes its task quickly. A filter string is specified to return events that have a priority greater than 600. You can determine what channels are configured by using the evminfo-lc command, or by examining the channel configuration file. For more information about the channel configuration file, see evminfo(1) .

NOTE: By default, only the evmlog channel is provided.

Sorting Events Using evmsort

The evmsort command takes a stream of Event Manager events as input, sorts them into the requested order, and writes them to its stdout stream. This command enables you to sort the output from the evmget command, but it can be used to sort Event Manager events from any source. For more information, see evmsort(1) .

The following example shows a typical command sequence:

	 # export EVM_SHOW_TEMPLATE="@timestamp [@priority]  @@"
# evmget -f '[pri >= 600]' | evmsort | evmshow | more

By default, the evmsort command sorts events in the chronological order, so the previous command is suitable for most cases. You use the s option to declare a sort specification if you want the events sorted differently. A sort specification is a text string that defines one or more sort keys, which are the data items on which you want to sort the events. The specification is a list of data item names, separated by colons (:). For example:

	 priority:timestamp

The preceding specification sorts events by timestamp within priority, so the first group of events that are returned are those with the lowest priority, sorted in their order of occurrence. You may use this specification as follows:

	 # evmget -f '[pri >= 600]' | evmsort -s "priority:timestamp" | 	 	 	 	evmshow | more

The default sort order is ascending, but you can change it to descending for an individual item specifier by appending a minus sign (-). You can explicitly request ascending order by specifying a plus sign (+). For example, the following command displays the highest priority events first (descending order), but within each priority range, the events are sorted oldest first (ascending order):

	 # evmget -f '[pri >= 600]' | evmsort -s "priority-:timestamp+" 	 	 		 | evmshow | more

For consistency with the show-template syntax, the evmsort command enables you to precede each item specifier with an at (@) character, as described in “Displaying Events Using evmshow”. There is no requirement to do this, and it does not affect the operation.

When you establish your sorting preferences, you can create a new default sort sequence by setting the environment variable EVM_SORT_SPEC. The following Korn shell (ksh) commands are equivalent to the previous example:

	 # export EVM_SORT_SPEC="priority-:timestamp+"
# evmget -f '[pri >= 600]' | evmsort | evmshow | more

You can override the value of the EVM_SORT_SPEC variable at any time by supplying a different sort specification with the s option.

Monitoring Events Using evmwatch

You can use the evmwatch command to monitor event activity through a terminal window. This command is an Event Manager subscribing client. It makes a connection to the daemon, sends it a subscription request, and waits to receive events. As events arrive, the evmwatch command writes them to the standard out stream (stdout) as binary Event Manager events. You cannot display the output of the evmwatch command because it is a stream of binary events. You must use the evmshow command to format the events. The following example monitors all events, and displays them on your terminal as they occur: # evmwatch | evmshow -t "@timestamp [@priority] @@"

Depending on your system type, and the level of event activity, this command may run for a while before any events are displayed. The command continues to run until you terminate it to regain control of your terminal, usually by pressing Ctrl/C. When a system is operating correctly, many of the events posted are low-priority informational events. You may want to filter these events out, particularly if your system has a high level of event activity. You can do this by supplying a filter to the evmwatch command:

# evmwatch -f "[priority >= 400]" | evmshow -t "@timestamp [@priority] @@"

This example watches for events with a priority of error equal to 400 or higher. You can change the filter string to exclude any set of events that occur regularly and are uninteresting. Alternatively, you may need to watch for a particular set of events. The preceding examples do not show the output of evmshow piped into more for display, because evmwatch is a realtime monitor. The evmwatch command displays events as they occur, rather than displaying them from a file. A command like pg or more may wait for the operator to intervene before reading more data from its input pipe; over time, this could lead to congestion in the pipeline. The Event Manager daemon cannot wait for its client (the evmwatch command) to clear its backlog; this results in the evmwatch command missing events. You should display the output from the evmwatch command directly on a terminal window, instead of piping commands to more or pg.

Avoid piping the output of the evmwatch command into the evmsort command because the evmsort command cannot sort events until it reads to the end of its input. As a monitoring program, the evmwatch command usually waits for input until it is killed explicitly. As a result, if you pipe the output of the evmwatch command directly into the evmsort command, there is no output from the evmsort command.The -A option simplifies the command string by running the evmsort command and the evmshow command automatically. The evmwatch command also supports the -A option and automatically runs the evmshow command when you use it. You can specify a show-template as an option to the evmwatch command as follows:

# evmwatch -A -f "[priority >= 400]" -t \"@timestamp \ [@priority] @@"

As with the evmget command, you can capture a set of interesting events in a file, to review later. It is more useful to store events in binary form than in text form, so you should send the output of the evmwatch command directly to a file, as shown in the following example, rather than piping it into the evmshow command first.

# evmwatch -f "[priority >= 400]" > my_events

The evmwatch command supports additional options that are useful for monitoring events from within a shell script. Refer evmwatch (1) for more information.

Posting Events Using evmpost

Although most events are likely to be posted by system and application software, there may be times when you want to post an event from the command line or from a shell script. For example, you may want to post a message event in the evmlog to note that a task is complete, or that you noticed something interesting. Making an entry in the evmlog makes it easy to establish when other events occurred relative to your entry.

You can post an event by using the evmpost command. The simplest form of this command is the quick message form, which you can specify by using the -a (administrator) or -u (user) option. To post a message, you must supply the message on the command line as a quoted string:

	 # evmpost -a "Fire drill started - evacuating computer room"

Administrative quick messages are posted with the name sys.unix.evm.msg.admin, so that you can search for them with a name filter:

	 # evmget -f '[name *.msg.admin]' | 
evmshow -t 'timestamp [@priority] @@'
27-Jun-2000 15:40:49 [200] EVM admin msg: Fire drill started - evacuating computer room

By default, the message is posted as a notice event, with a priority of 200. You can change the priority with the -p option. For example, setting the priority to 400 categorizes the message as an error event as follows:

	 # evmpost -p 400 -a  \
"Users reporting possible network problems"

By default, only the superuser or members of the adm group can post events with the -a option, although you can make it available to other privileged users by editing the authorization file, /etc/evm.auth, as described in “User Authorization”. Any user can specify the -u option to post messages in the same way. If on necessary, you can restrict this privilege to trusted users by editing the authorization file.

Posting Events from a Shell Script

Use the evmpost command to post a newly registered event, by passing the event information to the command in source (text) format. For more information about the event syntax, see evmpost(1). Source-level posting is useful in a shell script that performs a routine operation, where the event may indicate success or failure of the operation. This section describes the steps to create and post a new event that informs when a backup is complete. To create and post new events, complete the following steps:

  1. Create a template file, and verify its syntax.

  2. Install the template file, and make it known to the Event Manager daemon.

  3. Update the authorization file to allow the events to be posted.

  4. Write shell script commands to post the event.

For more information about event design guidelines, see the Event Manager Programmer’s Guide. You should be familiar with the concepts described in that book before you begin designing a new event. In which example, the backup script posts one of two events, local.admin.backup.ok with a priority of 200 (notice) and local.admin.backup.failed, with a priority of 400 (error). The failure event includes a variable item named result_code, to hold the exit code returned by the backup program. The variable is an 8-bit unsigned integer, and in the template it has a dummy value of zero. This dummy value is replaced with an actual value when the event is posted. The template file syntax is described in the evmtemplate(4) .

To create and post a new event, complete the following steps:

  1. Create the /var/evm/adm/templates/local directory if it does not exist.

  2. Use a text editor, such as vi, to create the following text file:

    		 	 	 	 		 # This file contains EVM event templates for local
    # backup notification events.
    event {
    name local.admin.backup.ok
    format "BACKUP: Backup completed OK"
    priority 200
    }

    event {
    name local.admin.backup.failed
    format "BACKUP: Backup failed - code $result_code"
    var {name result_code type UINT8 value 0}
    priority 400
    }
  3. Save the file in the /var/evm/adm/templates/local directory with the name backup.evt.

    You can install new template files in any directory under /var/evm/adm/templates, but name subdirectories and template files according to the names of your events for ease of identification. Keeping a small number of closely-related event templates in a single template file simplifies maintenance.

  4. Verify the template syntax. The syntax of a template file is identical to the syntax used to post an event. Hence, you can use the evmpost -r command to verify the syntax. The -r option instructs the evmpost command not to post the event, but to validate the syntax, convert the input into binary events, and then write the events to its standard output (stdout) stream. Use the evmpost -M command option to prevent the merging of template items in to the event, or to add any environmental items such as a timestamp.

    As with any stream of binary Event Manager events, you can use the evmshow command to verify the output of the evmpost command. To do this, enter the following command at the HP-UX prompt:

    	 					# cat /var/evm/adm/templates/local/backup.evt |
    evmpost -r -M | evmshow -t "@priority @@"

    If you created the file correctly, the following output is displayed:

    	 			200 BACKUP: Backup completed OK
    400 BACKUP: Backup failed - code 0
  5. Verify that the file is owned by root or bin, and that its permissions are set to 0400, 0600, 0440, or 0640. Correct the permissions by using the chown command and the chmod command, if necessary.

  6. Enter the following command to instruct the Event Manager daemon to reload its configuration: # evmreload -d

    If the command displays an error message, correct the problem and re-enter the command. The most common problem is that the ownership or permissions of the file is incorrect.

  7. Verify template registration by using the evmwatch -i command option, which retrieves templates from the daemon's database. The evmwatch command outputs the templates in the form of binary Event Manager events. You can use the evmshow command to display the binary Event Manager events. You need to show only the names of the events to ensure that they are registered correctly, as shown in the following example:

    	 	# evmwatch -i -f "[name local.admin.backup]" |
    evmshow -t "@name"
    local.admin.backup.ok
    local.admin.backup.failed
  8. Update the authorization file, /etc/evm.auth, to allow the events to be posted. Add the following lines to ensure that only the superuser can post the events and any user can see the events:

    	 	# Local backup events:
    event_rights {
    class local.admin.backup
    post root
    access +
    }

    Only the first three components of the name are specified. These components are common to the two new events, and when either of the events is posted its name matches this entry.

  9. Enter the evmreload -d command option, so that the daemon recognizes the new authorizations.

  10. Verify that the events are logged correctly by entering the following commands at the HP-UX prompt:

    		 	 		 	 	 			# echo 'event {name local.admin.backup.ok}' | evmpost
    # echo 'event {name local.admin.backup.failed}' | evmpost
    # evmget -f '[name local.admin.backup]' |
    evmshow -t '@timestamp [@priority] @@'

    28-Jun-2002 15:21:39 [200] BACKUP: Backup completed OK
    28-Jun-2002 15:21:40 [400] BACKUP: Backup failed - code 0

    In the previous example, the evmpost command reads the source input from its standard input (stdin) stream, converts it to an event, and posts it. The output from the final command shows the posted events. It includes the priorities specified in the template file, because the daemon merges the template information into each event as it is posted. The value of the code in the second event is zero, because it is the dummy value supplied in the template file, and that value was not overridden in the posted event. In the backup script, the value is set to a value other than zero.

  11. Add the posting commands to your backup script, as shown in the following example:

    	 	 	 	 		#! /bin/sh
    # This shell script runs the backup operation
    # and posts an event to indicate success
    #or failure.

    do_backups # Performs the backup operation
    if [ $? -eq 0 ]
    then
    # success
    echo 'event {name local.admin.backup.ok}'| evmpost
    else
    # failure
    RES=$?
    evmpost << END
    event {
    name local.admin.backup.failed
    var { name result_code type UINT8 value $RES }
    }
    END
    fi

    In the previous example, the input to the evmpost command for the success event is simple, so it is supplied on the same line by using the echo command. For the failure event, the value of the result_code variable must also be supplied. To supply this value, the shell's << syntax provides a more structured multiline form of input. Both forms of input supply source code input to the evmget command through its standard input (stdin) stream.

For more information about posting events from the command line, or from within a shell script, see evmpost(1).

Understanding the Event Manager Mark Event

When you review or monitor event activity, you observe the following event that occurs every 15 minutes:

		26-Jun-2000 08:57:45 [200] EVM: Mark event

The evmlog event channel posts this event to ensure that there is periodic event activity. If your system has a problem and you need to determine when it was last operational, you can look for mark commands in the evm log by using the following command:

# evmget -f "[name *.evm.mark]" | evmshow -t "@timestamp @last_timestamp @@"

26-Jun-2000 00:57:35 26-Jun-2000 04:42:40  [16 times] EVM: 	 	 		 	 	 Mark event	26-Jun-2000 04:57:41 -  EVM: Mark event	26-Jun-2000 05:12:41 -  EVM: Mark event	26-Jun-2000 05:27:41 -  EVM: Mark event	26-Jun-2000 05:42:41 26-Jun-2000 09:12:45  [15 times] EVM: 	 	 	 	 	 	Mark event

If the default logger configuration file is in use, you usually see three individual mark events, followed by a single event preceded by [n times], where n is a number less than or equal to 16. This is the result of the logger's suppression facility, which minimizes wasted space by combining multiple events over a period of up to four hours. The normal timestamp value shows the first occurrence of a combined event, and the last_timestamp data item shows the time of the last occurrence. The example includes the last_timestamp data item in the show-template, which displays the last mark event, posted at 09:12:45. This mark event tells you that the system was operational at that time.

Printable version
Privacy statement Using this site means you accept its terms Feedback to webmaster
© 2007 Hewlett-Packard Development Company, L.P.