This type of failure is known as Sick But Not Dead (SBND). Even if only a single cluster is in this condition, the failure can affect the performance or functionality (or both) of other clusters that are present in the same grid. 

Grid-wide consequences for this type of problem can be prevented by a “fence” mechanism that is available when all machines in the grid are running microcode R4.1.2 or later. The mechanism isolates the failing box while repair actions are performed. The other clusters in the grid continue to run without being affected by the unscheduled peer-outage. 

In this scenario, the failing box cannot recognize that a problem exists and keeps trying to take part in grid transactions. The other clusters in the grid recognize this condition and run a “remote fence” action against the sick clus ter. The following new LIBRARY REQUEST FENCE options are available to customize the fencing process, depending on business requirements: 

GLYPH<SM590000> Remote Cluster Fence Function enablement: 

LI REQ, <composite-library>, FENCE, <ENABLE|DISABLE> 

The default is DISABLE . This control allows or disallows the execution of remote cluster fence actions from the perspective of the entire grid. 

GLYPH<SM590000> Primary fence action: 

LI REQ, <distributed-library>, FENCE, ACTION, PRI, <NONE|ALERT|OFFLINE|REBOOT|REBOFF> 

This command features the following options: 

– NONE: This setting is the default. Other clusters in the grid cannot perform remote fencing against the specified cluster (“distri buted-library”) if it fails. Therefore, the system depends on the ability of the local cluster to detect that something is wrong with itself. 

– ALERT: Other clusters surface an alert only when they believe that the specified grid member is having problems. 

– OFFLINE: Other clusters trigger a force-offline sequence against the specified cluster to exclude it from grid transactions if the corresponding thresholds are met. If this primary action fails, a secondary fence action (if enabled) is started. 

– REBOOT: Other clusters trigger a reboot sequence against the specified cluster if the corresponding thresholds are met. In some cases, rebooting the SBND cluster is the best solution and faster recovery. The cluster automatically starts online processing after the reboot. If the machine fails to be rebooted, a secondary fence action (if enabled) is started. 

– REBOFF (reboot and stay offline): Other clusters trigger a reboot sequence against the specified cluster when the corresponding thresholds are met. This sequence prevents it from attempting to come online again. Use this option if health checks and troubleshooting are required the system comes back online again. If the machine fails to be rebooted, a secondary fence action (if enabled) is started. 

GLYPH<SM590000> Secondary fence action: 

LI REQ, <distributed-library>, FENCE, ACTION, SEC, <ENABLE|DISABLE> 

Referred to as “Isolate from grid network”, this secondary action is triggered against the specified cluster only if it was previously enabled (default is DISABLE), and the primary fence action failed. In this case, other clusters automatically run a reconfiguration of the intercluster network to logically exclude the failing cluster from any grid transactions. This action is available for offline or reboot primary actions only. 

