FastResync




	NOTE: You may need an additional license to use this feature.

The FastResync feature (previously called fast mirror resynchronization or FMR) performs quick and efficient resynchronization of stale mirrors (a mirror that is not synchronized). This increases the efficiency of the VxVM snapshot mechanism, and improves the performance of operations such as backup and decision support. Typically, these operations require that the volume is quiescent, and that they are not impeded by updates to the volume by other activities on the system. To achieve these goals, the snapshot mechanism in VxVM creates an exact copy of a primary volume at an instant in time. After a snapshot is taken, it can be accessed independently of the volume from which it was taken. In a clustered VxVM environment with shared access to storage, it is possible to eliminate the resource contention and performance overhead of using a snapshot simply by accessing it from a different node.

For details of how to enable FastResync on a per-volume basis, see “Enabling FastResync on a Volume”.

FastResync Enhancements

FastResync provides two fundamental enhancements to VxVM:

FastResync optimizes mirror resynchronization by keeping track of updates to stored data that have been missed by a mirror. (A mirror may be unavailable because it has been detached from its volume, either automatically by VxVM as the result of an error, or directly by an administrator using a utility such as vxplex or vxassist. A returning mirror is a mirror that was previously detached and is in the process of being re-attached to its original volume as the result of the vxrecover or vxplex att operation.) When a mirror returns to service, only the updates that it has missed need to be re-applied to resynchronize it. This requires much less effort than the traditional method of copying all the stored data to the returning mirror.
Once FastResync has been enabled on a volume, it does not alter how you administer mirrors. The only visible effect is that repair operations conclude more quickly.
FastResync allows you to refresh and re-use snapshots rather than discard them. You can quickly re-associate (snapback) a snapshot volume with its original volume. This reduces the system overhead required to perform cyclical operations such as backups that rely on the snapshot functionality of VxVM.

Non-Persistent FastResync

Non-Persistent FastResync, introduced in VxVM 3.1, allocates its change maps in memory. If Non-Persistent FastResync is enabled, a separate FastResync map is kept for the original volume and for each snapshot volume. Unlike a dirty region log (DRL), they do not reside on disk nor in persistent store. This has the advantage that updates to the FastResync map have little impact on I/O performance, as no disk updates needed to be performed. However, if a system is rebooted, the information in the map is lost, so a full resynchronization is required on snapback. This limitation can be overcome for volumes in cluster-shareable disk groups, provided that at least one of the nodes in the cluster remained running to preserve the FastResync map in its memory. However, a node crash in a High Availability (HA) environment requires the full resynchronization of a mirror when it is reattached to its parent volume.

Persistent FastResync

In VxVM 3.2, Non-Persistent FastResync has been augmented by the introduction of Persistent FastResync. Unlike Non-Persistent FastResync, Persistent FastResync keeps the FastResync maps on disk so that they can survive both system reboots, system crashes and cluster crashes. If Persistent FastResync is enabled on a volume or on a snapshot volume, a data change object (DCO) and a DCO log volume are associated with the volume.

The DCO object manages information about the FastResync maps. These maps track writes to the original volume and to each of up to 32 snapshot volumes since the last snapshot operation. The DCO log volume on disk holds the 33 maps, each of which is 4 blocks in size by default.

Persistent FastResync can also track the association between volumes and their snapshot volumes after they are moved into different disk groups. When the disk groups are rejoined, this allows the snapshot plexes to be quickly resynchronized. This ability is not supported by Non-Persistent FastResync. See “Reorganizing the Contents of Disk Groups” for details.

How Non-Persistent FastResync Works with Snapshots

The snapshot feature of VxVM takes advantage of FastResync change tracking to record updates to the original volume after a shapshot plex is created. After a snapshot is taken, the snapback option is used to reattach the snapshot plex. Provided that FastResync is enabled on a volume before the snapshot is taken, and that it is not disabled at any time before the snapshot is reattached, the changes that FastResync records are used to resynchronize the volume during the snapback. This considerably reduces the time needed to resynchronize the volume.

Non-Persistent FastResync uses a map in memory to implement change tracking. Each bit in the map represents a contiguous number of blocks in a volume's address space. The default size of the map is 4 blocks. The kernel tunable vol_fmr_logsz can be used to limit the maximum size in blocks of the map as described on “vol_fmr_logsz” page 240.

How Persistent FastResync Works with Snapshots

Persistent FastResync uses a map in a DCO log volume on disk to implement change tracking. As for Non-Persistent FastResync, each bit in the map represents a contiguous number of blocks in a volume's address space. The default size of the map is 1 block. This can be increased by specifying the dcologlen attribute when the volume is created. The default value of dcologlen is 132 blocks (the plex contains 33 maps, each of length 4 blocks). To use a larger map size, multiply the desired map size by 33 to calculate the value of dcologlen that you should specify. For example, to use an 8-block map, you would specify dcologlen=264. The maximum possible map size is 64 blocks, which corresponds to a dcologlen value of 2112 blocks.

Figure 1-32 “Mirrored Volume with Persistent FastResync Enabled” shows an example of a mirrored volume with two plexes on which Persistent FastResync is enabled. Associated with the volume are a DCO object and a DCO log volume with two plexes.

Figure 1-32 Mirrored Volume with Persistent FastResync Enabled

When the snapstart operation is performed on the volume, this sets up a snapshot plex in the volume and associates a disabled DCO log plex with it, as shown in Figure 1-33 “Mirrored Volume After Completion of a snapstart Operation”

Figure 1-33 Mirrored Volume After Completion of a snapstart Operation

Multiple snapshot plexes and associated DCO log plexes may be created in the volume by re-running the snapstart operation. You can create up to a total of 32 plexes (data and log) in a volume.

A snapshot volume can now be created from a snapshot plex by running the snapshot operation on the volume. As illustrated in Figure 1-34 “Mirrored Volume and Snapshot Volume After Completion of a snapshot Operation”, this also sets up a DCO object and a DCO log volume for the snapshot volume. The new DCO log volume contains the single DCO log plex that was associated with the snapshot plex. If two snapshot plexes were taken to form the snapshot volume, the DCO log volume would contain two plexes.

Associated with both the original volume and the snapshot volume are snap objects. The snap object for the original volume points to the snapshot volume, and the snap object for the snapshot volume points to the original volume. This allows VxVM to track the relationship between volumes and their snapshots even if they are moved into different disk groups.

The snap objects in the original volume and snapshot volume are automatically deleted in either of the following circumstances:

The snapback operation is run to return all of the plexes of the snapshot volume to the original volume.
The snapclear operation is run on a volume to break the association between the original volume and the snapshot volume. If the volumes are in different disk groups, snapclear must be run separately on each volume.

See “Merging a Snapshot Volume (snapback)”, “Dissociating a Snapshot Volume (snapclear)”, and the vxassist(1M) manual page for more information.

Figure 1-34 Mirrored Volume and Snapshot Volume After Completion of a snapshot Operation

Additional Snapshot Features

To make it easier to create snapshots of several volumes at the same time, the snapshot option accepts more than one volume name as its argument. By default, each replica volume is named SNAPnumber-volume, where number is a unique serial number, and volume is the name of the volume being snapshotted. This default can be overridden by using the option -o name=pattern, as described on the vxassist(1M) manual page.

To snapshot all the volumes in a single disk group, specify the option -o allvols to vxassist. However, this fails if any of the volumes in the disk group do not have a complete snapshot plex.

It is also possible to take several snapshots of the same volume. A new FastResync change map is produced for each snapshot taken to minimize the resynchronization time for each snapshot.

By default, the snapshot plex is resynchronized from the data in the original volume during a snapback operation. Alternatively, you can choose the snapshot plex as the preferred copy of the data when performing a snapback as illustrated in Figure 1-35 “Resynchronizing the Original Volume from the Snapshot”. Specifying the option -o resyncfromreplica to vxassist resynchronizes the original volume from the data in the snapshot.

Figure 1-35 Resynchronizing the Original Volume from the Snapshot




	NOTE: The original volume must not be in use during a `snapback` operation in which it is resynchronized from a snapshot. Stop any application, such as a database, and unmount any file systems that are configured to use the volume.

It is possible to grow the replica volume, or the original volume, and still use FastResync. Growing the volume extends the map that FastResync uses to track changes to the original volume. This can change either the size of the map, or the size of the region represented by each bit in the map. In either case, the part of the map that corresponds to the grown area of the volume is marked as "dirty" so that this area is resynchronized. The snapback operation fails if it attempts to create an incomplete snapshot plex. In such cases, you must grow the replica volume, or the original volume, before invoking snapback. Growing the two volumes separately can lead to a snapshot that shares physical disks with another mirror in the volume. To prevent this, grow the volume after the snapback command is complete.

FastResync Limitations

The following limitations apply to FastResync:

FastResync is not supported for RAID-5 volumes.
Neither Non-Persistent nor Persistent FastResync can be used to resynchronize mirrors after a system crash. Dirty region logging (DRL), which can coexist with FastResync, should be used for this purpose.
When a subdisk is relocated, the entire plex is marked "dirty" and a full resynchronization becomes necessary.
If a snapshot volume is split off into another disk group, Non-Persistent FastResync cannot be used to resynchronize the snapshot plexes with the original volume when the disk group is rejoined with the original volume's disk group. Persistent FastResync must be used for this purpose.
If you move or split an original volume (on which Persistent FastResync is enabled) into another disk group, and then move or join it to a snapshot volume's disk group, you cannot use vxassist snapback to resynchronize snapshot plexes with the original volume. This restriction arises because a snapshot volume references the original volume by its record ID at the time that the snapshot volume was created. Moving the original volume to a different disk group changes the volume's record ID, and so breaks the association. However, in such a case, you can use the vxplex snapback command with the -f (force) option to perform the snapback.
Enabling FastResync on a layered volume enables only the parent volume (the top volume). FastResync must be enabled on each individual subvolume in a layered volume.
Any operation that changes the layout of a replica volume can mark the FastResync change map for that snapshot "dirty" and require a full resynchronization during snapback. Operations that cause this include subdisk split, subdisk move, and online relayout of the replica. It is safe to perform these operations after the snapshot is completed. For more information, see the vxvol (1M), vxassist (1M), and vxplex (1M) manual pages.

FastResync

Technical documentation

» Table of Contents

» Glossary