GAPTHEGURU

Geek with special skills

Failover Cluster Step-by-Step Guide: Configuring the Quorum in a Failover Cluster

A failover cluster is a group of independent computers that  work
together to increase the availability of applications and services.  The
clustered servers (called nodes) are connected by physical cables  and
by software. If one of the cluster nodes fails, another node begins  to
provide service (a process known as failover). Users experience a
minimum of disruptions in service.
This guide describes the new quorum options in failover  clusters in
Windows Server® 2008 and provides steps for configuring the  quorum in a
failover cluster. By following the configuration steps in  this guide,
you can learn about failover clusters and familiarize  yourself with
quorum modes in failover clustering.
In Windows Server 2008, the improvements to failover clusters
(formerly known as server clusters) are aimed at simplifying clusters,
making them more secure, and enhancing cluster stability. Cluster setup
and management are easier. Security and networking in clusters have
been  improved, as has the way a failover cluster communicates with
storage.  For more information about improvements to failover clusters,
see http://go.microsoft.com/fwlink/?LinkId=62368.
In this guide
Overview of quorum in a failover cluster
Requirements and recommendations for quorum configurations
Steps for viewing the quorum configuration of a failover cluster
Steps for changing the quorum configuration in a failover cluster
Troubleshooting: how to force a cluster to start without quorum
Additional references
For additional background information, also see Appendix A: Details of How Quorum Works in a Failover Cluster and Appendix B: Additional Information About Quorum Modes.

Overview of quorum in a failover cluster

In simple terms, the quorum for a cluster is the number of  elements
that must be online for that cluster to continue running. In  effect,
each element can cast one “vote” to determine whether the  cluster
continues running. The voting elements are nodes or, in some  cases, a
disk witness or file share witness. Each voting element (with  the
exception of a file share witness) contains a copy of the cluster
configuration, and the Cluster service works to keep all copies
synchronized at all times.
It is essential that the cluster stops running if too many  failures
occur or if there is a problem with communication between the  cluster
nodes. For a more detailed explanation, see the next section, Why quorum is necessary.
Note that the full function of a cluster depends not just on  quorum,
but on the capacity of each node to support the services and
applications that fail over to that node. For example, a cluster that
has five nodes could still have quorum after two nodes fail, but each
remaining cluster node would continue serving clients only if it had
enough capacity to support the services and applications that failed
over to it.

Why quorum is necessary

When network problems occur, they can interfere with  communication
between cluster nodes. A small set of nodes might be able  to
communicate together across a functioning part of a network, but  might
not be able to communicate with a different set of nodes in  another
part of the network. This can cause serious issues. In this  “split”
situation, at least one of the sets of nodes must stop running  as a
cluster.
To prevent the issues that are caused by a split in the  cluster, the
cluster software requires that any set of nodes running as a  cluster
must use a voting algorithm to determine whether, at a given  time, that
set has quorum. Because a given cluster has a specific set of  nodes
and a specific quorum configuration, the cluster will know how  many
“votes” constitutes a majority (that is, a quorum). If the number  drops
below the majority, the cluster stops running. Nodes will still  listen
for the presence of other nodes, in case another node appears  again on
the network, but the nodes will not begin to function as a  cluster
until the quorum exists again.
For example, in a five node cluster that is using a node  majority,
consider what happens if nodes 1, 2, and 3 can communicate  with each
other but not with nodes 4 and 5. Nodes 1, 2, and 3 constitute  a
majority, and they continue running as a cluster. Nodes 4 and 5 are a
minority and stop running as a cluster, which prevents the problems of a
“split” situation. If node 3 loses communication with other nodes, all
nodes stop running as a cluster. However, all functioning nodes will
continue to listen for communication, so that when the network begins
working again, the cluster can form and begin to run.
For more information about how quorum works, see Appendix A: Details of How Quorum Works in a Failover Cluster.

Overview of the quorum modes

There have been significant improvements to the quorum  model in
Windows Server 2008.  In Windows Server 2003, almost all server
clusters used a disk in cluster storage (the “quorum resource”) as the
quorum. If a node could communicate with the specified disk, the node
could function as a part of a cluster, and otherwise it could not. This
made the quorum resource a potential single point of failure. In
Windows  Server 2008, a majority of ‘votes’ is what determines whether a
cluster  achieves quorum. Nodes can vote, and where appropriate, either
a disk  in cluster storage (called a “disk witness”) or a file share
(called a  “file share witness”) can vote. There is also a quorum mode
called No Majority: Disk Only which functions like the
disk-based quorum in Windows Server 2003.  Aside from that mode, there
is no single point of failure with the  quorum modes, since what matters
is the number of votes, not whether a  particular element is available
to vote.
This new quorum model is flexible and you can choose the mode best suited to your cluster.

ImportantImportant
In most situations, it is best to use the quorum  mode selected by
the cluster software. If you run the quorum  configuration wizard, the
quorum mode that the wizard lists as  “recommended” is the quorum mode
chosen by the cluster software. We only  recommend changing the quorum
configuration if you have determined that  the change is appropriate for
your cluster.

There are four quorum modes:

    • Node Majority: Each node that is  available and in
      communication can vote. The cluster functions only with  a majority of
      the votes, that is, more than half.
    • Node and Disk Majority: Each node plus a
      designated disk in the cluster storage (the “disk witness”) can vote,
      whenever they are available and in communication. The cluster functions
      only with a majority of the votes, that is, more than half.
    • Node and File Share Majority: Each node  plus a
      designated file share created by the administrator (the “file  share
      witness”) can vote, whenever they are available and in  communication.
      The cluster functions only with a majority of the votes,  that is, more
      than half.
    • No Majority: Disk Only: The cluster has  quorum if
      one node is available and in communication with a specific  disk in the
      cluster storage. Only the nodes that are also in  communication with
      that disk can join the cluster.

Choosing the quorum mode for a particular cluster

The following table describes clusters based on the number  of nodes
and other cluster characteristics, and lists the quorum mode  that is
recommended in most cases.
A “multi-site” cluster is a cluster in which an investment  has been
made to place sets of nodes and storage in physically separate
locations, providing a disaster recovery solution. An “Exchange Cluster
Continuous Replication (CCR)” cluster is a failover cluster that
includes Exchange Server 2007 with Cluster Continuous Replication, a
high-availability feature that combines asynchronous log shipping and
replay technology.

Description of cluster Quorum recommendation
Odd number of nodes Node Majority
Even number of nodes (but not a multi-site cluster) Node and Disk Majority
Even number of nodes, multi-site cluster Node and File Share Majority
Even number of nodes, no shared storage Node and File Share Majority
Exchange CCR cluster (two nodes) Node and File Share Majority

Diagrams of quorum modes

The following diagrams show how each of the quorum modes affects whether a cluster can or cannot achieve quorum.

Node Majority

The following diagram shows Node Majority used (as recommended) for a cluster with an odd number of nodes.
Node Majority quorum mode

Node and Disk Majority

The following diagram shows Node and Disk Majority used  (as
recommended) for a cluster with an even number of nodes. Each node  can
vote, as can the disk witness.
Node and Disk Majority quorum modeThe
following diagram shows how the disk witness also  contains a replica
of the cluster configuration database in a cluster  that uses Node and
Disk Majority.
Node and Disk Majority: replicas

Node and File Share Majority

The following diagram shows Node and File Share Majority  used (as
recommended) for a cluster with an even number of nodes and a  situation
where having a file share witness works better than having a  disk
witness. Each node can vote, as can the file share witness.
Node and File Share Majority quorum modeThe
following diagram shows how the file share witness  can vote, but does
not contain a replica of the cluster configuration  database. Note that
the file share witness does contain information  about which version of
the cluster configuration database is the most  recent.
Node and File Share Majority: replicas

No Majority: Disk Only

The following illustration shows how a cluster that uses  the disk as
the only determiner of quorum can run even if only one node  is
available and in communication with the quorum disk. It also shows  how
the cluster cannot run if the quorum disk is not available (single
point of failure). For this cluster, which has an odd number of nodes,
Node Majority is the recommended quorum mode.
Disk Only quorum mode

Additional information about quorum modes

For more information about quorum modes, see Appendix B: Additional Information About Quorum Modes.

Requirements and recommendations for quorum configurations

Before configuring the quorum for a failover cluster you must  of
course meet the requirements for the cluster itself. For information
about cluster requirements, see http://go.microsoft.com/fwlink/?LinkId=114536. For information about cluster validation, see  http://go.microsoft.com/fwlink/?LinkId=114537 and http://go.microsoft.com/fwlink/?LinkId=114538.
For a cluster using the Node Majority quorum mode (which  includes
almost all clusters with an odd number of nodes), there are no
additional requirements for the quorum. The following sections provide
guidelines for clusters using the Node and Disk Majority quorum mode and
the Node and File Share Majority quorum mode. (The requirements and
recommendations for the Node and Disk Majority mode also apply to the No
Majority: Disk Only mode.)

Requirements and recommendations for clusters using Node and Disk Majority

When using the Node and Disk Majority mode, review the following requirements and recommendations for the disk witness.

noteNote
These requirements and recommendations also apply to the quorum disk for the No Majority: Disk Only mode.
    • Use a small Logical Unit Number (LUN) that is at least 512 MB in size.
    • Choose a basic disk with a single volume.
    • Make sure that the LUN is dedicated to the disk witness.  It must not contain any other user or application data.
    • Choose whether to assign a drive  letter to the LUN based on the
      needs of your cluster. The LUN does not  have to have a drive letter (to
      conserve drive letters for  applications).
    • Make sure that the LUN has been verified with the Validate a Configuration Wizard.
    • We recommend that you configure the LUN with hardware RAID for fault tolerance.
    • In most situations, do not back up the  disk witness or the data on
      it. Backing up the disk witness can add to  the input/output (I/O)
      activity on the disk and decrease its  performance, which could
      potentially cause it to fail.
    • We recommend that you avoid all antivirus scanning on the disk witness.
    • Format the LUN with the NTFS file system.

If there is a disk witness configured, but bringing that  disk online
will not achieve quorum, then it remains offline.  If  bringing that
disk online will achieve quorum, then it is brought online  by the
cluster software.

Requirements and recommendations for clusters using Node and File Share Majority

When using the Node and File Share Majority mode, review the following recommendations for the file share witness.

    • Use a Server Message Block (SMB) share on a Windows Server 2003 or Windows Server 2008 file server.
    • Make sure that the file share has a minimum of 5 MB of free space.
    • Make sure that the file share is  dedicated to the cluster and is
      not used in other ways (including  storage of user or application data).
    • Do not place the share on a node that is a member of this cluster or will become a member of this cluster in the future.
    • You can place the share on a file  server that has multiple file
      shares servicing different purposes.  This  may include multiple file
      share witnesses, each one a dedicated share.  You can even place the
      share on a clustered file server (in a different  cluster), which would
      typically be a clustered file server containing  multiple file shares
      servicing different purposes.
    • For a multi-site cluster, you can  co-locate the external file share
      at one of the sites where a node or  nodes are located.  However, we
      recommend that you configure the  external share in a separate third
      site.
    • Place the file share on a server that is a member of a domain, in the same forest as the cluster nodes.
    • For the folder that the file share uses, make sure that the administrator has Full Control share and NTFS permissions.
    • Do not use a file share that is part of a Distributed File System (DFS) Namespace.
noteNote
After the Quorum Configuration Wizard has been run,  the computer
object for the Cluster Name will automatically be granted  read and
write permissions to the file share.

If there is a file share witness configured, but bringing  that file
share online will not achieve quorum, then it remains offline.   If
bringing that file share online will achieve quorum, then it is  brought
online by cluster software.
For more information about file share witness recommendations, see:

Steps for viewing the quorum configuration of a failover cluster

When you install a failover cluster, the cluster software
automatically chooses an appropriate quorum configuration for that
cluster, based mainly on the number of nodes (even or odd). You can
easily view the quorum configuration of an existing cluster using either
the Failover Cluster Management snap-in or the command line.

To view the quorum configuration of an existing cluster using the Failover Cluster Management snap-in

    1. To open the failover cluster snap-in, click Start, click Administrative Tools, and then click Failover Cluster Management. (If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Continue.)
    1. In the console tree, if the cluster that you want to view is not displayed, right-click Failover Cluster Management, click Manage a Cluster, and then select the cluster you want to view.
    1. In the center pane, find Quorum Configuration, and view the description.
      In the following example, the quorum mode is Node and Disk Majority and the disk witness is Cluster Disk 2.
      Failover Cluster Management snap-in

To view the quorum configuration of an existing cluster using the Command Prompt window

    1. To open a Command Prompt window, on a cluster node, click Start, right-click Command Prompt, and then either click Run as administrator or click Open.
    1. If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Continue.
    1. Review the configuration of the quorum by typing:
      cluster /quorum

Steps for changing the quorum configuration in a failover cluster

You must complete the following steps to change the quorum configuration in a failover cluster.

ImportantImportant
Unless you have changed the number of nodes in your  cluster, it is
usually best to use the quorum configuration recommended  by the quorum
configuration wizard. We only recommend changing the  quorum
configuration if you have determined that the change is  appropriate for
your cluster.

Membership in the local Administrators group  on
each clustered server, or equivalent, is the minimum permissions
required to complete this procedure.  Also, the account you use must be a
domain user account.  Review details about using the appropriate
accounts and group memberships at http://go.microsoft.com/fwlink/?LinkId=83477.

To change the quorum configuration in a failover cluster

    1. To open the failover cluster snap-in, click Start, click Administrative Tools, and then click Failover Cluster Management. (If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Continue.)
    1. In the Failover Cluster Management snap-in, if the  cluster you want
      to configure is not displayed, in the console tree,  right-click Failover Cluster Management, click Manage a Cluster, and select or specify the cluster you want.
    1. With  the cluster selected, under Actions, click More Actions, and then click Configure Cluster Quorum Settings.
      Cluster quorum wizard, first page
    1. Click Next. The following illustration  shows the
      wizard page that displays for a cluster with an even number  of nodes.
      Note that the text on this page varies, depending on whether  the
      cluster has an even number or odd number of nodes. To view more
      information about the selections on this page, at the bottom of the
      page, click More about quorum configurations.
      Cluster quorum wizard, second page
    1. Select a quorum mode from the list. For more information, see Choosing the quorum mode for a particular cluster, earlier in this guide.
    1. Click Next and then go to the appropriate step in this procedure:
        • If you chose Node Majority, go to the last step in this procedure.
        • If you chose Node and Disk Majority or No Majority, go to the next step in this procedure.
        • If you chose Node and File Share Majority, skip to step 8 in this procedure.
    1. If you chose Node and Disk Majority or No Majority, a wizard page similar to the following appears. (For No Majority, the title of the page is Select Storage Resource.) Select the storage volume that you want to use for the disk witness (or if you chose No Majority, for the quorum resource), and then skip to step 9. For information about the requirements for the disk witness, see Requirements and recommendations for clusters using Node and Disk Majority.
      If you change disk assignments on this page, the former storage volume is no longer assigned to the core Cluster Group and instead goes back to Available Storage.
      Cluster quorum wizard, witness disk page
    1. If you chose Node and File Share Majority, the following wizard page appears. Specify the file share you want to use, or click the Browse
      button and use the standard browsing interface to select the file
      share. For information about the requirements for the file share, see Requirements and recommendations for clusters using Node and File Share Majority.
      Cluster quorum wizard, witness file share page
    1. Click Next. Use the confirmation page to confirm your selections, and then click Next.
    1. After the wizard runs and the Summary page appears, if you want to view a report of the tasks that the wizard performed, click View Report.
      noteNote
      The most recent report will remain in the systemroot\Cluster\Reports folder with the name QuorumConfiguration.mht.

Troubleshooting: how to force a cluster to start without quorum

When troubleshooting, you might be in a situation where the  cluster
is offline because it does not have quorum, but you want to  bring it
online. The first thing to understand is your quorum mode and  why you
no longer have quorum. This may provide some insight into how  the
cluster can achieve quorum and come online automatically. If you  need
to force the Cluster service to start, you can make all nodes which  can
communicate with each other begin working together as a cluster by
running the net start clussvc command with an option
for forcing quorum. The cluster will use the copy of the cluster
configuration that is on the node on which you run the command, and
replicate it to all other nodes. To force the cluster to start, on a
node that contains a copy of the cluster configuration that you want to
use, type the following command:
net start clussvc /fq
The command can also be typed as net start clussvc /forcequorum. In Windows Server 2008,  the net start clussvc command no longer includes the /resetquorumlog or /fixquorum options.
Forcing a cluster to start that does not have quorum may be
especially useful in an unbalanced multi-site cluster. If you have a
five-node multi-site cluster and three nodes at Site A fail, then the
two nodes at Site B will go offline since they no longer have quorum. If
there is a genuine disaster at Site A, then it may take a significant
amount of time for the site to come online, and so you would likely
want  to force Site B to come online, even though it does not have
quorum.
When a cluster is forced to start without quorum it  continually
looks to add nodes to the cluster and is in a special  “forced” state.
Once it has majority, the cluster moves out of the  forced state and
behaves normally, which means it is not necessary to  rerun the cluster
command without a startup switch. If the cluster then  loses a node and
drops below quorum, it will go offline again because it  is no longer in
the forced state. At that point, to bring it online  again while it
does not have quorum would require running net start clussvc /fq again.

Additional references

For more information about disk witness recommendations, see:
http://go.microsoft.com/fwlink/?LinkId=115004 (Note that this is a Windows Server 2003 article, but the disk witness recommendations remain unchanged.)
For more information about file share witness recommendations, see:

For a list of technical documentation for failover clusters on the TechNet Web site, see:
http://go.microsoft.com/fwlink/?LinkId=68633
For more information on Cluster Continuous Replication (CCR) in Microsoft Exchange Server 2007:
http://go.microsoft.com/fwlink/?LinkId=114542
For information about choosing and validating hardware for a failover cluster, see the TechNet Web site at:
http://go.microsoft.com/fwlink/?LinkId=115087

Advertisements

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: