Logging and Tracing of NFS Services

This section tells you how to start the following tools:

NFS Logging

You can configure logging for the following NFS daemons:

rpc.mountd
rpc.statd
rpc.lockd

Each message logged by these daemons can be identified by the date, time, host name, process ID, and name of the daemon that generated the message. You can direct logging messages from all these NFS daemons to the same file.

To Control the Size of Log Files

Log files grow without bound, using up disk space. You might want to create a cron job to truncate your log files regularly. Following is an example crontab entry that empties the log file at 1:00 AM every Monday, Wednesday, and Friday:

0 1 * * 1,3,5 cat /dev/null > log_file

For more information, type man 1M cron or man 1 crontab at the HP-UX prompt.

To Start and Stop `rpc.mountd` Logging

Issue the following commands to kill the rpc.mountd process and restart it with logging turned on (PID is a process ID returned by the ps command):
ps -ef | grep mountd kill PID /usr/sbin/rpc.mountd -l /var/adm/mountd.log
If you want rpc.mountd to log mount requests and mount failures as well as errors, add the -t2 option to the rpc.mountd command, as in the following example:
/usr/sbin/rpc.mountd -l /var/adm/mountd.log -t2
To stop logging, kill rpc.mountd and restart it without the -l logfile and -t2 options.

If you do not specify the -l or-t option, rpc.mountd logs only errors to /var/adm/mountd.log. If this file does not exist, rpc.mountd creates it. rpc.mountd can share the same log file with the other NFS daemons.

For more information, type man 1M mountd at the HP-UX prompt.

To Start and Stop Detailed Logging of `rpc.statd` and `rpc.lockd`

To start detailed logging of rpc.statd and rpc.lockd while they are running, issue the following commands (PID is a process ID returned by the ps command):

/usr/bin/ps -ef | /usr/bin/grep rpc.statd
/usr/bin/kill -SIGUSR2 PID
/usr/bin/ps -ef | /usr/bin/grep rpc.lockd
/usr/bin/kill -SIGUSR2 PID

The SIGUSR2 signal sets the logging to level 3 (the most detailed level).

The logging for rpc.statd is appended to the file /var/adm/rpc.statd.log. The logging for rpc.lockd is appended to the file /var/adm/rpc.lockd.log.

To stop detailed logging of rpc.statd and rpc.lockd, issue the same commands listed above to send the SIGUSR2 signal to the processes. The SIGUSR2 signal is a toggle that turns logging on or off, depending on its current state.

For more information, type man 1M statd or man 1M lockd at the HP-UX prompt.

To Start and Stop Basic Logging of `rpc.statd` and `rpc.lockd`

To start basic logging of rpc.statd and rpc.lockd (just errors, warnings, startup, and shutdown), issue the following commands (PID is a process ID returned by the ps command):

ps -ef | grep lockd
kill PID
ps -ef | grep statd
kill PID
/usr/sbin/rpc.statd -l /var/adm/rpc.statd.log
/usr/sbin/rpc.lockd -l /var/adm/rpc.lockd.log




	NOTE: Always start `rpc.statd` before starting `rpc.lockd`.

To stop basic logging of rpc.statd and rpc.lockd, kill them and restart them without the -l logfile option.

The rpc.statd and rpc.lockd daemons can share the same log file with the other NFS daemons.

Please see the following man pages for more information: lockd (1M) or statd (1M).

AutoFS Logging

AutoFS logs messages through /usr/sbin/syslogd. By default, syslogd writes messages to the file /var/adm/syslog/syslog.log. See the syslogd (1M) man page for more information.

To Start AutoFS Logging

Issue the following command to get a list of all the automounted directories on the client:

for FS in $(grep autofs /etc/mnttab | awk ‘{print $2}’) do   grep ‘nfs’ /etc/mnttab | awk ‘{print $2}’ | grep ^${FS}   done

For every automounted directory listed by the grep command, issue the following command to determine whether the directory is currently in use:
/usr/sbin/fuser -cu local_mount_point
This command lists the process IDs and user names of everyone using the mounted directory.
Warn any users to cd out of the directory, and kill any processes that are using the directory, or wait until the processes terminate. You can issue the following command to kill all the processes using the mounted directory:
/usr/sbin/fuser -ck local_mount_point

Issue the following commands to kill the AutoFS:

/sbin/init.d/nfs.client stop




	CAUTION: Do not kill the automountd daemon with the kill command, it does not die gracefully. It does not unmount AutoFS mount points before it dies. Use the nfs.client stop script to ensure that automountd dies cleanly.

Add -v to AUTOMOUNTD_OPTIONS variable in the /etc/rc.config.d/nfsconf file, as in the following example:
AUTOMOUNTD_OPTIONS = “-v”
This change will enable AutoFS logging.
Issue the following commands to start AutoFS:
/sbin/init.d/nfs.client start

To Stop AutoFS Logging

To stop AutoFS logging, kill AutoFS and restart it (as described in the previous section),only with removing “-v” option from AUTOMOUNTD_OPTIONS.

AutoFS Tracing

Two levels of AutoFS tracing are available:

Detailed (level 3): Includes traces of all AutoFS requests and replies, mount attempts, timeouts, and unmount attempts. You can start level 3 tracing while AutoFS is running.
Basic (level 1): Includes traces of all AutoFS requests and replies. You must restart AutoFS to start level 1 tracing.

To Start and Stop AutoFS Detailed Tracing

Log in as root to the NFS client.
Issue the following commands (PID is the process ID returned by the ps command):
ps -ef | grep automoutd
kill -SIGUSR2 PID

Level 3 tracing is appended to the file /var/adm/automount.log.




	NOTE: The command, kill -SIGUSR2 PID, works if tracing is not already on.

To stop level 3 tracing, issue the same commands listed above to send the SIGUSR2 signal to automountd. The SIGUSR2 signal is a toggle that turns tracing on or off depending on its current state.

If you have basic (level 1) tracing turned on when you send the SIGUSR2 signal to automountd, the SIGUSR2 signal turns tracing off.

To Start AutoFS Basic Tracing

Log in as root to the NFS client.
Add -T to the AUTOMOUNTD_OPTIONS variable in the /etc/rc.config.d/nfsconf file, as in the following example:
AUTOMOUNTD_OPTIONS=”-T”
This change puts AutoFS basic tracing messages into the file /var/adm/automount.log.

Issue the following command to get a list of all the automounted directories on the client:

for FS in $(grep autofs /etc/mnttab | awk ‘{print $2}’)

do

grep ‘nfs’ /etc/mnttab | awk ‘{print $2}’ | grep ^${FS}

done

For every automounted directory listed by the grep command, issue the following command to determine whether the directory is currently in use:
/usr/sbin/fuser -cu local_mount_point
This command lists the process IDs and user names of everyone using the mounted directory.
Warn any users to cd out of the directory, and kill any processes that are using the directory, or wait until the processes terminate. You can issue the following command to kill all the processes using the mounted directory:
/usr/sbin/fuser -ck local_mount_point

Issue the following commands to kill the AutoFS:

/sbin/init.d/nfs.client stop




	CAUTION: Do not kill the `automountd` daemon with the kill command, it does not die gracefully. It does not unmount AutoFS mount points before it dies. Use the nfs.client stop script to ensure that automountd dies cleanly.

Issue the following command to start AutoFS with tracing enabled:
/abin/init.d/nfs.client start

To Stop AutoFS Basic Tracing

To stop AutoFS tracomg, kill AutoFS and restart it (as described in the previous section), only with removing -T from AUTOMOUNTD_OPTIONS.

AutoFS Tracing Output

The most interesting events to the users are the tracing output when mounting and unmounting of file systems occur.

Mount Event Tracing Output

The general format of a mount event trace is:MOUNT REQUEST: <time stamp>

<mount information> <other tracing> ... <other tracing>

MOUNT REPLY: <status>=mount statusThe <mount information> trace has the following format (all on oneline):

name=<key>[<subdirectory>] <map>= map name <opts>=mount options <path>=mount path <other tracing>

where:

<key> = the key value from the map

<subdirectory> = subdirectory (may be blank)

<map> = name of map

<opts> = mount options

<path> = mount path

<other tracing> = other trace information

The mount status in the mount reply contains 0 if the mount is successful, it has non-zero value when the mount is not successful.

The following is an example of a typical mount trace:May 13 18:45:09 t5 MOUNT REQUEST: Tue May 13 18:45:09 2003May 13 18:45:09 t5 name=nfs127[/tmp] map=auto.indirect opts=path=/n2ktmp_8264/nfs127/tmp direct=1May 13 18:45:09 t5 PUSH /etc/auto.indirectMay 13 18:45:09 t5 POP /etc/auto.indirectMay 13 18:45:09 t5 mapline: hpnfs127:/ /tmp hpnfs127:/tmpMay 13 18:45:09 t5 do_mount1:May 13 18:45:09 t5 (nfs,nfs) /n2ktmp_8264/nfs127/tmp hpnfs127:/tmp penalty=0May 13 18:45:09 t5 nfsmount: input: hpnfs127[other]May 13 18:45:09 t5 nfsmount: standard mount on/n2ktmp_8264/nfs127/tmp :May 13 18:45:09 t5 hpnfs127:/tmpMay 13 18:45:09 t5 nfsmount: v3=1[0],v2=0[0] => v3.May 13 18:45:09 t5 nfsmount: Get mount version: request vers=3min=3May 13 18:45:09 t5 nfsmount: mount version=3May 13 18:45:09 t5 Port numbers are 937, 937May 13 18:45:09 t5 Port matchMay 13 18:45:09 t5 mount hpnfs127:/tmp /n2ktmp_8264/nfs127/tmp()May 13 18:45:09 t5 nfs_args: hpnfs127, , 0x2004060, 0, 0, 0, 0,0, 0, 0, 0,May 13 18:45:09 t5 args_temp: hpnfs127, , 0x3004060, 0, 0, 0, 0,0, 0, 0, 0, hpnfs127:/tmpMay 13 18:45:09 t5 mount hpnfs127:/tmp dev=44000004 rdev=0 OKMay 13 18:45:09 t5 MOUNT REPLY: status=0, AUTOFS_DONE

Unmount Event Tracing Output

The general format of an unmount event trace is:

UNMOUNT REQUEST: <time stamp>

<other tracing>

...

<other tracing>

UNMOUNT REPLY: <status>=unmount status

The unmount status in the unmount reply contains 0 if the unmount is successful, it has a non-zero value when the unmount is not successful.

The following is an example of a typical unmount trace event:

May 13 18:46:27 t1      UNMOUNT REQUEST: Tue May 13 18:46:27 2003
May 13 18:46:27 t1      dev=44000004 rdev=0 direct
May 13 18:46:27 t1      ping: hpnfs127 request vers=3 min=2
May 13 18:46:27 t1      pingnfs OK: nfs version=3
May 13 18:46:27 t1      nfsunmount: umount /n2ktmp_8264/nfs127/tmp
May 13 18:46:27 t1      Port numbers are 937, 937
May 13 18:46:27 t1      Port match
May 13 18:46:27 t1      nfsunmount: umount /n2ktmp_8264/nfs127/tmp OK
May 13 18:46:27 t1      unmount /n2ktmp_8264/nfs127/tmp OK
May 13 18:46:27 t1      UNMOUNT REPLY: status=0

Logging for the Other NFS Services

You can configure logging for the following NFS services:

rpc.rexd
rpc.rstatd
rpc.rusersd
rpc.rwalld
rpc.sprayd

Logging is not available for the rpc.quotad daemon.

Each message logged by these daemons can be identified by the date, time, host name, process ID, and name of the function that generated the message. You can direct logging messages from all these NFS services to the same file.

To Control the Size of Log Files

0 1 * * 1,3,5 cat /dev/null > log_file

For more information, type man 1M cron or man 1 crontab at the HP-UX prompt.

To Configure Logging for the Other NFS Services

Add the -l logfile option to the lines in /etc/inetd.conf for the services you want to log. In the following example, logging is turned on for rpc.rexd and rpc.rstatd:

rpc stream tcp nowait root /usr/sbin/rpc.rexd 100017 1 \
rpc.rexd -l /var/adm/rpc.log
 
rpc dgram udp wait root /usr/lib/netsvc/rstat/rpc.rstatd \
100001 1-3 rpc.rstatd -l /var/adm/rpc.log

Issue the following command to restart inetd:
/usr/sbin/inetd -c

If you do not specify a log file for the other NFS services (with the -l option), they do not log any messages. The NFS services can all share the same log file.

See the man page, rexd(1M), for descriptions of the messages logged by the rpc.rexd daemon.

For more information, see the following man pages: rexd(1M), rstatd(1M), rusersd(1M), rwalld(1M), and sprayd(1M).

NIS Logging

You can configure logging for the following NIS processes:

ypxfr
ypserv
ypbind
yppasswdd

To Control the Size of Log Files

0 1 * * 1,3,5 cat /dev/null > log_file

For more information, please see the man page, cron(1M) or crontab, at the HP-UX prompt.

To Stop and Start Logging of `ypxfr`

If ypxfr is run interactively from the command line, it logs messages to standard output. If ypxfr is run by cron or by yppush, it logs messages to the file /var/yp/ypxfr.log, if the file exists. To start logging of ypxfr, issue the following command to make sure the /var/yp/ypxfr.log file exists:

/usr/bin/touch /var/yp/ypxfr.log

To stop logging of ypxfr, remove the ypxfr.log file:

/usr/bin/rm /var/yp/ypxfr.log

You cannot redirect the logging output of ypxfr.

For more information, see the following man pages: ypxfr(1M), cron(1M), and yppush(1M).

To Start and Stop Logging of `ypserv`

By default, the ypserv daemon logs messages to the file /var/yp/ypserv.log, if it exists. To start logging of ypserv, issue the following command to make sure the /var/yp/ypserv.log file exists:

/usr/bin/touch /var/yp/ypserv.log

To stop logging of ypserv, remove the ypserv.log file:

/usr/bin/rm /var/yp/ypserv.log

If you want to direct ypserv logging to a different file, follow these steps:

Add the -l logfile option to the YPSERV_OPTIONS variable in /etc/rc.config.d/namesvrs, as in the following example:
YPSERV_OPTIONS=”-l /var/yp/nis_log”
Issue the following commands to restart ypserv (PID is the process ID returned by the ps command):
ps -ef | grep ypserv kill PID /usr/lib/netsvc/yp/ypserv options
options is the list of options configured in the YPSERV_OPTIONS variable in the /etc/rc.config.d/namesvrs file. You can also source the /etc/rc.config.d/namesvrs file, and then enter the ypserv command as follows:
/usr/lib/netsvc/yp/ypserv $YPSERV_OPTIONS

If you specify a log file with the -l option, ypserv can share the same log file with the other NIS daemons.

For more information, type man 1M ypserv at the HP-UX prompt.

To Configure `ypbind` Logging

Add the -l logfile option to the YPBIND_OPTIONS variable in /etc/rc.config.d/namesvrs, as in the following example:
YPBIND_OPTIONS=”-l /var/yp/nis_log”
Issue the following commands to restart ypbind (PID is the process ID returned by the ps command):
ps -ef | grep ypbind kill PID /usr/lib/netsvc/yp/ypbind options
options is the list of options configured in the YPBIND_OPTIONS variable in the /etc/rc.config.d/namesvrs file. You can also source the /etc/rc.config.d/namesvrs file, and then enter the ypbind command as follows:
/usr/lib/netsvc/yp/ypbind $YPBIND_OPTIONS

If you do not specify a log file for ypbind (with the -l option), it logs messages to the system console, /dev/console. The ypbind daemon can share the same log file with the other NIS daemons.

For more information, type man 1M ypbind at the HP-UX prompt.

To Configure `yppasswdd` Logging

Add the -l logfile option to the YPPASSWDD_OPTIONS variable in /etc/rc.config.d/namesvrs, as in the following example:
YPPASSWDD_OPTIONS=”-l /var/yp/nis_log”
Issue the following commands to restart yppasswdd (PID is the process ID returned by the ps command):
ps -ef | grep yppasswdd kill PID /usr/lib/netsvc/yp/rpc.yppasswdd options
options is the list of options configured in the YPPASSWDD_OPTIONS variable in the /etc/rc.config.d/namesvrs file. You can also source the /etc/rc.config.d/namesvrs file, and then enter the yppasswdd command as follows:
/usr/lib/netsvc/yp/rpc.yppasswdd $YPPASSWDD_OPTIONS

For more information, type man 1M yppasswdd at the HP-UX prompt.

NIS+ Logging

You can log the activities of the NIS+ rpc.nisd daemon with the -A and -v options.

On the NIS+ server, add the -A or -v option to the RPC_NISD_OPTIONS variable, as in the following example:
RPC_NISD_OPTIONS=”$EMULYP -v”

Issue the following commands to restart rpc.nisd:

/sbin/init.d/nisplus.server stop
/sbin/init.d/nisplus.server start

To stop NIS+ logging, remove the -v and -A options from the RPC_NISD_OPTIONS variable, and issue the commands in step 2 to restart rpc.nisd.

The -v option causes rpc.nisd to send a running narration of what it is doing to syslogd. Messages are logged at LOG_INFO priority.

The -A option logs NIS+ authentication activities to syslogd with LOG_INFO priority.

You might have to modify your /etc/syslog.conf file to allow messages of LOG_INFO priority to be logged.

For more information, type man 1M syslogd or man 1M nisd at the HP-UX prompt.

Logging With `nettl` and `netfmt`

Issue the following command to make sure nettl is running:
/usr/bin/ps -ef | grep nettl
If nettl is not running, issue the following command to start it:
/usr/sbin/nettl -start
Issue the following command to start logging:
/usr/sbin/nettl -l i w e d -e all
The logging classes are specified following the -l option. They are i (informational), w (warning), e (error), and d (disaster). Disaster logging is always on. You cannot turn it off. Information logging (i) fills up your log file faster than the other classes, so you might want to leave it off.
Recreate the event you want to log.
Issue the following command to turn logging off:
/usr/sbin/nettl -l d -e all
This command changes the logging class back to disaster only for all subsystems.
Issue the following command to format the binary log file:
/usr/sbin/netfmt -lN -f /var/adm/nettl.LOG00 > formatted_file
where formatted_file is the name of the file where you want the formatted output from netfmt. The default log file, /var/adm/nettl.LOGnn, is specified in the nettl configuration file, /etc/nettlgen.conf. If the file /var/adm/nettl.LOG00 does not exist on your system, the default log file may have been changed in /etc/nettlgen.conf.

NIS+ logging is not supported by nettl.

For more information, type man 1M nettl or man 1M netfmt.

Tracing With `nettl` and `netfmt`

Issue the following command to make sure nettl is running:
/usr/bin/ps -ef | grep nettl
If nettl is not running, issue the following command to start it:
/usr/sbin/nettl -start

Issue the following command to start tracing:

/usr/sbin/nettl -tn pduin pduout loopback -e all -s 1024 \
-f tracefile

Recreate the event you want to trace.
Issue the following command to turn tracing off:
/usr/sbin/nettl -tf -e all

Create the following filter file for netfmt:

filter ip_saddr remote_host_IP_address
filter ip_daddr remote_host_IP_address
filter rpcprogram nfs
filter rpcprogram nlockmgr
filter rpcprogram llockmgr
filter rpcprogram status
filter rpcprogram mount
filter rpcprogram rpcbind

remote_host_IP_address is the IP address of the host with which your host was communicating when the event you want to trace occurred.

Issue the following command to format the binary trace file:
/usr/sbin/netfmt -c filter_file -lN -f tracefile.TRC0 > formatted_file
where tracefile is the name of the file you specified when you started tracing, and formatted_file is the name of the file where you want the formatted output from netfmt.

NIS+ tracing is not supported by nettl.

For more information, type man 1M nettl or man 1M netfmt.

Logging and Tracing of NFS Services

Technical documentation

» Table of Contents

» Index

NFS Logging

To Control the Size of Log Files

To Start and Stop `rpc.mountd` Logging

To Start and Stop Detailed Logging of `rpc.statd` and `rpc.lockd`

To Start and Stop Basic Logging of `rpc.statd` and `rpc.lockd`

AutoFS Logging

To Start AutoFS Logging

To Stop AutoFS Logging

AutoFS Tracing

To Start and Stop AutoFS Detailed Tracing

To Start AutoFS Basic Tracing

To Stop AutoFS Basic Tracing

AutoFS Tracing Output

Mount Event Tracing Output

Unmount Event Tracing Output

Logging for the Other NFS Services

To Control the Size of Log Files

To Configure Logging for the Other NFS Services

NIS Logging

To Control the Size of Log Files

To Stop and Start Logging of `ypxfr`

To Start and Stop Logging of `ypserv`

To Configure `ypbind` Logging

To Configure `yppasswdd` Logging

NIS+ Logging

Logging With `nettl` and `netfmt`

Tracing With `nettl` and `netfmt`

Logging and Tracing of NFS Services

Technical documentation

» Table of Contents

» Index

NFS Logging

To Control the Size of Log Files

To Start and Stop rpc.mountd Logging

To Start and Stop Detailed Logging of rpc.statd and rpc.lockd

To Start and Stop Basic Logging of rpc.statd and rpc.lockd

AutoFS Logging

To Start AutoFS Logging

To Stop AutoFS Logging

AutoFS Tracing

To Start and Stop AutoFS Detailed Tracing

To Start AutoFS Basic Tracing

To Stop AutoFS Basic Tracing

AutoFS Tracing Output

Mount Event Tracing Output

Unmount Event Tracing Output

Logging for the Other NFS Services

To Control the Size of Log Files

To Configure Logging for the Other NFS Services

NIS Logging

To Control the Size of Log Files

To Stop and Start Logging of ypxfr

To Start and Stop Logging of ypserv

To Configure ypbind Logging

To Configure yppasswdd Logging

NIS+ Logging

Logging With nettl and netfmt

Tracing With nettl and netfmt

To Start and Stop `rpc.mountd` Logging

To Start and Stop Detailed Logging of `rpc.statd` and `rpc.lockd`

To Start and Stop Basic Logging of `rpc.statd` and `rpc.lockd`

To Stop and Start Logging of `ypxfr`

To Start and Stop Logging of `ypserv`

To Configure `ypbind` Logging

To Configure `yppasswdd` Logging

Logging With `nettl` and `netfmt`

Tracing With `nettl` and `netfmt`