Jump to content United States-English
HP.com Home Products and Services Support and Drivers Solutions How to Buy
» Contact HP
More options
HP.com home
NFS Services Administrator's Guide: HP-UX 11i version 3 > Chapter 5 Troubleshooting NFS Services

Logging and Tracing of NFS Services

» 

Technical documentation

Complete book in PDF
» Feedback
Content starts here

 » Table of Contents

 » Index

This section tells you how to start the following tools:

AutoFS Logging

AutoFS logs messages through /usr/sbin/syslogd. By default, syslogd writes messages to the file /var/adm/syslog/syslog.log. See the syslogd (1M) manpage for more information.

To Start AutoFS Logging

  1. Log in as root to the NFS client.

  2. Enter the following command to get a list of all the automounted directories on the client:

    for FS in $(grep autofs /etc/mnttab | awk ‘{print $2}’) do   grep ‘nfs’ /etc/mnttab | awk ‘{print $2}’ | grep ^${FS}   done
  3. For every automounted directory listed by the grep command, enter the following command to determine whether the directory is currently in use:

    /usr/sbin/fuser -cu local_mount_point

    This command lists the process IDs and user names of everyone using the mounted directory.

  4. Warn any users to cd out of the directory, and kill any processes that are using the directory, or wait until the processes terminate. You can enter the following command to kill all the processes using the mounted directory:

    /usr/sbin/fuser -ck local_mount_point
  5. Enter the following commands to kill the AutoFS:

    /sbin/init.d/autofs stop 
    CAUTION: Do not kill the automountd daemon with the kill command. It does not unmount AutoFS mount points before it dies. Use the nfs.client stop script to ensure that automountd dies.
  6. Add -v to AUTOMOUNTD_OPTIONS variable in the /etc/rc.config.d/nfsconf file, as in the following example:

    AUTOMOUNTD_OPTIONS = “-v”

    This change will enable AutoFS logging.

  7. Enter the following commands to start AutoFS:

    /sbin/init.d/nfs.client start

To Stop AutoFS Logging

To stop AutoFS logging, kill AutoFS and restart it after removing “-v” option from AUTOMOUNTD_OPTIONS.

AutoFS Tracing

Two levels of AutoFS tracing are available:

Detailed (level 3)

Includes traces of all AutoFS requests and replies, mount attempts, timeouts, and unmount attempts. You can start level 3 tracing while AutoFS is running.

Basic (level 1)

Includes traces of all AutoFS requests and replies. You must restart AutoFS to start level 1 tracing.

To Start and Stop AutoFS Detailed Tracing

  1. Log in as root to the NFS client.

  2. Enter the following commands (PID is the process ID returned by the ps command):

    ps -ef | grep automoutd
    kill -SIGUSR2 PID

Level 3 tracing is appended to the file /var/adm/automount.log.

NOTE: The command, kill -SIGUSR2 PID, works if tracing is not already on.

To stop level 3 tracing, enter the same commands listed above to send the SIGUSR2 signal to automountd. The SIGUSR2 signal is a toggle that turns tracing on or off depending on its current state.

If you have basic (level 1) tracing turned on when you send the SIGUSR2 signal to automountd, the SIGUSR2 signal turns tracing off.

To Start AutoFS Basic Tracing

  1. Log in as root to the NFS client.

  2. Add -T to the AUTOMOUNTD_OPTIONS variable in the /etc/rc.config.d/nfsconf file, as in the following example:

    AUTOMOUNTD_OPTIONS=”-T” 

    This change puts AutoFS basic tracing messages into the file /var/adm/automount.log.

  3. Enter the following command to get a list of all the automounted directories on the client:

    for FS in $(grep autofs /etc/mnttab | awk ‘{print $2}’)
    do 
     grep ‘nfs’ /etc/mnttab | awk ‘{print $2}’ | grep ^${FS}
    done
  4. For every automounted directory listed by the grep command, enter the following command to determine whether the directory is currently in use:

    /usr/sbin/fuser -cu local_mount_point

    This command lists the process IDs and user names of everyone using the mounted directory.

  5. Warn any users to cd out of the directory, and kill any processes that are using the directory, or wait until the processes terminate. You can enter the following command to kill all the processes using the mounted directory:

    /usr/sbin/fuser -ck local_mount_point
  6. Enter the following command to kill the AutoFS:

    /sbin/init.d/autofs stop 
    CAUTION: Do not kill the automountd daemon with the kill command. It does not unmount AutoFS mount points before it dies. Use the nfs.client stop script to ensure that automountd dies.
  7. Enter the following command to start AutoFS with tracing enabled:

    /abin/init.d/autofs start 

To Stop AutoFS Basic Tracing

To stop AutoFS tracing, kill AutoFS and restart it only with removing -T from AUTOMOUNTD_OPTIONS.

AutoFS Tracing Output

The most interesting events to the users are the tracing output when mounting and unmounting of file systems occur.

Mount Event Tracing Output

The general format of a mount event trace is:MOUNT REQUEST: <time stamp>

    <mount information> <other tracing> ... <other tracing>

MOUNT REPLY: <status>=mount statusThe <mount information> trace has the following format (all on oneline):

name=<key>[<subdirectory>] <map>= map name <opts>=mount options <path>=mount path <other tracing>

where:

<key> = the key value from the map

<subdirectory> = subdirectory (may be blank)

<map> = name of map

<opts> = mount options

<path> = mount path

<other tracing> = other trace information

The mount status in the mount reply contains 0 if the mount is successful, it has non-zero value when the mount is not successful.

The following is an example of a typical mount trace:May 13 18:45:09 t5   MOUNT REQUEST: Tue May 13 18:45:09 2003May 13 18:45:09 t5 name=nfs127[/tmp] map=auto.indirect opts=path=/n2ktmp_8264/nfs127/tmp direct=1May 13 18:45:09 t5        PUSH /etc/auto.indirectMay 13 18:45:09 t5 POP /etc/auto.indirectMay 13 18:45:09 t5 mapline: hpnfs127:/ /tmp hpnfs127:/tmpMay 13 18:45:09 t5 do_mount1:May 13 18:45:09 t5 (nfs,nfs) /n2ktmp_8264/nfs127/tmp hpnfs127:/tmp penalty=0May 13 18:45:09 t5 nfsmount: input: hpnfs127[other]May 13 18:45:09 t5 nfsmount: standard mount on/n2ktmp_8264/nfs127/tmp :May 13 18:45:09 t5        hpnfs127:/tmpMay 13 18:45:09 t5 nfsmount: v3=1[0],v2=0[0] => v3.May 13 18:45:09 t5 nfsmount: Get mount version: request vers=3min=3May 13 18:45:09 t5 nfsmount: mount version=3May 13 18:45:09 t5   Port numbers are 937, 937May 13 18:45:09 t5   Port matchMay 13 18:45:09 t5 mount hpnfs127:/tmp /n2ktmp_8264/nfs127/tmp()May 13 18:45:09 t5 nfs_args: hpnfs127, , 0x2004060, 0, 0, 0, 0,0, 0, 0, 0,May 13 18:45:09 t5 args_temp: hpnfs127, , 0x3004060, 0, 0, 0, 0,0, 0, 0, 0, hpnfs127:/tmpMay 13 18:45:09 t5 mount hpnfs127:/tmp dev=44000004 rdev=0 OKMay 13 18:45:09 t5   MOUNT REPLY: status=0, AUTOFS_DONE

Unmount Event Tracing Output

The general format of an unmount event trace is:

UNMOUNT REQUEST: <time stamp>

   <other tracing>

         ...

<other tracing>

UNMOUNT REPLY: <status>=unmount status

The unmount status in the unmount reply contains 0 if the unmount is successful, it has a non-zero value when the unmount is not successful.

The following is an example of a typical unmount trace event:

May 13 18:46:27 t1      UNMOUNT REQUEST: Tue May 13 18:46:27 2003
May 13 18:46:27 t1 dev=44000004 rdev=0 direct
May 13 18:46:27 t1 ping: hpnfs127 request vers=3 min=2
May 13 18:46:27 t1 pingnfs OK: nfs version=3
May 13 18:46:27 t1 nfsunmount: umount /n2ktmp_8264/nfs127/tmp
May 13 18:46:27 t1 Port numbers are 937, 937
May 13 18:46:27 t1 Port match
May 13 18:46:27 t1 nfsunmount: umount /n2ktmp_8264/nfs127/tmp OK
May 13 18:46:27 t1 unmount /n2ktmp_8264/nfs127/tmp OK
May 13 18:46:27 t1 UNMOUNT REPLY: status=0




 

Logging for the Other NFS Services

You can configure logging for the following NFS services:

  • rpc.rexd

  • rpc.rstatd

  • rpc.rusersd

  • rpc.rwalld

  • rpc.sprayd

Logging is not available for the rpc.quotad daemon.

Each message logged by these daemons can be identified by the date, time, host name, process ID, and name of the function that generated the message. You can direct logging messages from all these NFS services to the same file.

To Control the Size of Log Files

Log files grow without bound, using up disk space. You might want to create a cron job to truncate your log files regularly. Following is an example crontab entry that empties the log file at 1:00 AM every Monday, Wednesday, and Friday:

0 1 * * 1,3,5 cat /dev/null > log_file

For more information, type man 1M cron or man 1 crontab at the HP-UX prompt.

To Configure Logging for the Other NFS Services

  1. Add the -l logfile option to the lines in /etc/inetd.conf for the services you want to log. In the following example, logging is turned on for rpc.rexd and rpc.rstatd:

    rpc stream tcp nowait root /usr/sbin/rpc.rexd 100017 1 \
    rpc.rexd -l /var/adm/rpc.log

    rpc dgram udp wait root /usr/lib/netsvc/rstat/rpc.rstatd \
    100001 1-3 rpc.rstatd -l /var/adm/rpc.log
  2. Enter the following command to restart inetd:

    /usr/sbin/inetd -c

If you do not specify a log file for the other NFS services (with the -l option), they do not log any messages. The NFS services can all share the same log file.

For more information, see rexd(1M), rstatd(1M), rusersd(1M), rwalld(1M) , and sprayd(1M).

Logging With nettl and netfmt

  1. Enter the following command to make sure nettl is running:

    /usr/bin/ps -ef | grep nettl

    If nettl is not running, enter the following command to start it:

    /usr/sbin/nettl -start
  2. Enter the following command to start logging:

    /usr/sbin/nettl -l i w e d -e all 

    The logging classes are specified following the -l option. They are i (informational), w (warning), e (error), and d (disaster). Disaster logging is always on. You cannot turn it off. Information logging (i) fills up your log file faster than the other classes, so you might want to leave it off.

  3. Recreate the event you want to log.

  4. Enter the following command to turn logging off:

    /usr/sbin/nettl -l d -e all

    This command changes the logging class back to disaster only for all subsystems.

  5. Enter the following command to format the binary log file:

    /usr/sbin/netfmt -lN -f /var/adm/nettl.LOG00 > formatted_file

    where formatted_file is the name of the file where you want the formatted output from netfmt. The default log file, /var/adm/nettl.LOGnn, is specified in the nettl configuration file, /etc/nettlgen.conf. If the file /var/adm/nettl.LOG00 does not exist on your system, the default log file may have been changed in /etc/nettlgen.conf.

For more information, see nettl(1M) and netfmt(1M).

Tracing With nettl and netfmt

  1. Enter the following command to make sure nettl is running:

    /usr/bin/ps -ef | grep nettl

    If nettl is not running, enter the following command to start it:

    /usr/sbin/nettl -start
  2. Enter the following command to start tracing:

    /usr/sbin/nettl -tn pduin pduout loopback -e all -s 1024 \
    -f tracefile
  3. Recreate the event you want to trace.

  4. Enter the following command to turn tracing off:

    /usr/sbin/nettl -tf -e all
  5. Create the following filter file for netfmt:

    filter ip_saddr remote_host_IP_address
    filter ip_daddr remote_host_IP_address
    filter rpcprogram nfs
    filter rpcprogram nlockmgr
    filter rpcprogram llockmgr
    filter rpcprogram status
    filter rpcprogram mount
    filter rpcprogram rpcbind

    remote_host_IP_address is the IP address of the host with which your host was communicating when the event you want to trace occurred.

  6. Enter the following command to format the binary trace file:

    /usr/sbin/netfmt -c filter_file -lN -f tracefile.TRC0 > formatted_file

    where tracefile is the name of the file you specified when you started tracing, and formatted_file is the name of the file where you want the formatted output from netfmt.

For more information, type man 1M nettl or man 1M netfmt.

Printable version
Privacy statement Using this site means you accept its terms Feedback to webmaster
© 2007 Hewlett-Packard Development Company, L.P.