Creating a Storage Infrastructure with VERITAS Cluster File System (CFS)

First, be sure the cluster is running:

# cmviewcl

If it is not, start it:

# cmruncl

If you have not initialized your disk groups, or if you have an old install that needs to be re-initialized, use the vxinstall command to initialize VxVM/CVM disk groups. “Initializing the VERITAS Volume Manager ”.

The VERITAS cluster volumes are managed by a Serviceguard-supplied system multi-node package which runs on all nodes at once, and cannot failover. In CVM 4.1, which is required for the Cluster File System, Serviceguard supplies the SG-CFS-pkg template. (In CVM 3.5, Serviceguard supplies the VxVM-CVM-pkg template)

The CVM 4.1 package has the following responsibilities:

	NOTE: Do not edit system multi-node package configuration files, such as VxVM-CVM-pkg.conf and SG-CFS-pkg.conf. Create and modify configuration using the cfs admin commands listed in Appendix A.

Activate the SG-CFS-pkg and start up CVM with the cfscluster command; this creates SG-CFS-pkg, and also starts it.

This example, for the cluster file system, uses a timeout of 900 seconds; if your CFS cluster has many disk groups and/or disk LUNs visible to the cluster nodes, you may need to a longer timeout value. Use the -s option to start the CVM package in shared mode:

# cfscluster config -t 900 -s

Verify the system multi-node package is running and CVM is up, using the cmviewcl or cfscluster command. Following is an example of using the cfscluster command. In the last line, you can see that CVM is up, and that the mount point is not yet configured:

# cfscluster status

Node              :  ftsys9
Cluster Manager   :  up
CVM state         :  up (MASTER)
MOUNT POINT   TYPE    SHARED VOLUME   DISK GROUP    STATUS
 
Node              :  ftsys10
Cluster Manager   :  up
CVM state         :  up
MOUNT POINT   TYPE    SHARED VOLUME   DISK GROUP    STATUS

Find the master node using vxdctl or cfscluster status

Initialize a new disk group, or import an existing disk group, in shared mode, using the vxdg command.

Verify the disk group. The state should be enabled and shared: # vxdg list

NAME       STATE                     ID       
logdata   enabled, shared, cds      11192287592.39.ftsys9

Use the cfsdgadm command to create the package SG-CFS-DG-ID#, where ID# is an automatically incremented number, assigned by Serviceguard when it creates the package. In this example, the SG-CFS-DG-ID# package will be generated to control the disk group logdata, in shared write mode:
# cfsdgadm add logdata all=sw

With VERITAS CFS, you can verify the package creation with the cmviewcl command, or with the cfsdgadm display command. An example of cfsdgadm output is is shown below:
#cfsdgadm display

 Node Name : ftsys9 (MASTER)
 DISK GROUP        ACTIVATION MODE
  logdata           off    (sw)
 
 Node Name :  ftsys10
  DISK GROUP       ACTIVATION MODE
   logdata          off    (sw)

Activate the disk group and Start up the Package
# cfsdgadm activate logdata

To verify, you can use cfsdgadm or cmviewcl This example shows the cfsdgadm output:
# cfsdgadm display -v logdata

NODE NAME       ACTIVATION MODE
  ftsys9        sw (sw)
     MOUNT POINT        SHARED VOLUME     TYPE
  ftsys10       sw (sw)
      MOUNT POINT           SHARED VOLUME       TYPE

To view the package name that is monitoring a disk group, use the cfsdgadm show_package command:
# cfsdgadm show_package logdata

sg_cfs_dg-1

Make log_files volume on the logdata disk group:
# vxassist -g logdata make log_files 1024m

Use the vxprint command to verify:
# vxprint log_files

disk group: logdata
TY NAME      ASSOC      KSTATE     LENGTH    PLOFFS  STATE  TUTIL0   PUTIL0
v  log_files    fsgen   ENABLED    1048576     -    ACTIVE   -       -
pl log_files-01 fsgen   ENABLED    1048576     -    ACTIVE   -       -
sd ct4t0d6-01   fsgen   ENABLED    1048576     -    ACTIVE   -       -

Create a filesystem:
# newfs -F vxfs /dev/vx/rdsk/logdata/log_files

version 6 layout
1-048576 sectors, 1048576 blocks of size 1024, log size 16384 blocks
largefiles supported

Create the cluster mount point:
# cfsmntadm add logdata log_files /tmp/logdata/log_files all=rw

Package name “SG-CFS-MP-1” is generated to control the resource.

You do not need to create the directory. The command creates one on each of the nodes, during the mount.

	CAUTION: Once you create the disk group and mount point packages, it is critical that you administer the cluster with the cfs commands, including cfsdgadm, cfsmntadm, cfsmount, and cfsumount. These non-cfs commands could cause conflicts with subsequent command operations on the file system or Serviceguard packages. Use of these other forms of mount will not create an appropriate multi-node package which means that the cluster packages are not aware of the file system changes.

	NOTE: Please note that the disk group and mount point multi-node packages do not monitor the health of the disk group and mount point. They check that the packages that depend on them have access to the disk groups and mount points. If the dependent application package loses access and cannot read and write to the disk, it will fail; however that will not cause the DG or MP multi-node package to fail.

Verify with cmviewcl or cfsmntadm display. This example uses the cfsmntadm command:
# cfsmntadm display

Cluster Configuration for Node: ftsys9
MOUNT POINT             TYPE    SHARED VOLUME    DISK GROUP    STATUS
/tmp/logdata/log_files  regular  log_files       logdata       NOT MOUNTED
 
Cluster Configuration for Node: ftsys10
MOUNT POINT             TYPE    SHARED VOLUME   DISK GROUP    STATUS
/tmp/logdata/log_files  regular  log_files      logdata       NOT MOUNTED

Mount the filesystem:
# cfsmount /tmp/logdata/log_files

This starts up the multi-node package and mounts a cluster-wide filesystem.

Verify that multi-node package is running and filesystem is mounted:
#cmviewcl

CLUSTER   STATUS
cfs_cluster   up
NODE   STATUS   STATE
ftsys9   up     running
ftsys10  up     running
 
MULTI_NODE_PACKAGES
PACKAGE      STATUS     STATE     AUTO_RUN    SYSTEM
SG-CFS-pkg   up         running    enabled     yes
SG-CFS-DG-1  up         running    enabled     no
SG-CFS-MP-1  up         running    enabled     no

# ftsys9/etc/cmcluster/cfs> bdf

Filesystem                kbytes used avail  %used  Mounted on
/dev/vx/dsk/logdata/log_files  10485 17338 966793   2%     tmp/logdata/log_files

# ftsys10/etc/cmcluster/cfs> bdf

Filesystem                kbytes used avail  %used  Mounted on
/dev/vx/dsk/logdata/log_files  10485 17338 966793   2%     tmp/logdata/log_files

To view the package name that is monitoring a mount point, use the cfsmntadm show_package command:
# cfsmntadm show_package /tmp/logdata/log_files

SG-CFS-MP-1

After creating your mount point packages for the cluster file system, you can configure your application package to depend on the mount points. In the ASCII configuration file, specify the dependency triplet, specifying DEPENDENCY_CONDITION SG-mp-pkg-#=UP and DEPENDENCY_LOCATION SAME_NODE. For more information about the DEPENDENCY parameter, see “Package Configuration File Parameters ”.

	NOTE: Unlike LVM volume groups, CVM disk groups are not entered in the cluster configuration file, they are entered in the package configuration file only.