You must also consider the bandwidth consequences of a complete fabric outage. Although a complete fabric outage is a fairly rare event, insufficient bandwidth can turn a single-SAN outage into a total access loss event. 

Take the bandwidth of the links into account. It is common to have ISLs run faster than host ports, which reduces the number of required ISLs. 

## Distance extensions options 

To implement remote mirroring over a distance by using the Fibre Channel, you have the following choices: 

GLYPH<SM590000> Optical multiplexors, such as dense wavelength division multiplexing (DWDM) or coarse wavelength division multiplexing (CWDM) devices 

GLYPH<SM590000> Long-distance Small Form-factor Pluggable (SFP) transceivers and XFPs 

GLYPH<SM590000> Fibre Channel-to-IP conversion boxes 

Of these options, the optical distance extension is the preferred method. IP distance extension introduces more complexity, is less reliable, and has performance limitations. However, optical distance extension can be impractical in many cases because of cost or unavailability. 

Check the list of supported SAN routers and FC extenders on the online documentation at: IBM FlashSystem 9100 family . 

## Optical multiplexors 

Optical multiplexors can extend a SAN up to hundreds of kilometers (or miles) at high speeds. For this reason, they are the preferred method for long-distance expansion. If you use multiplexor-based distance extension, closely monitor your physical link error counts in your switches. Optical communication devices are high-precision units. When they shift out of calibration, you will start to see errors in your frames. 

## Long-distance SFPs and XFPs 

Long-distance optical transceivers have the advantage of extreme simplicity. You do not need any expensive equipment, and you have only a few configuration steps to perform. However, ensure that you only use transceivers that are designed for your particular SAN switch. 

## Fibre Channel over IP 

Fibre Channel over IP (FCIP) is by far the most common and least expensive form of distance extension. It is also complicated to configure. Relatively subtle errors can have severe performance implications. 

With IP-based distance extension, you must dedicate bandwidth to your FCIP traffic if the link is shared with other IP traffic. Do not assume that because the link between two sites has low traffic or is used only for email, this type of traffic is always the case. FC is far more sensitive to congestion than most IP applications. 

Also, when you are communicating with the networking architects for your organization, make sure to distinguish between megabytes per second  as opposed to megabits per second.  In the storage world, bandwidth often is specified in megabytes per second (MBps), and network engineers specify bandwidth in megabits per second (Mbps). 

