StorNext File System Tuning
The Distributed LAN (Disk Proxy) Networks
StorNext File System Tuning Guide 20
Network Configuration
and Topology 0
A common source of difficult-to-diagnose issues with SNFS is improper
IP network configuration. Many incorrect IP configurations might appear
to work when tested with particular applications or particular kinds of
hosts, but fail when used with SNFS or when a different kind of host is
added to the cluster.
SNFS Distributed LAN should be run over one or more dedicated IP
subnetworks, using dedicated NICs for each subnetwork. These
subnetworks are typically configured using site-local addresses, such as
10.a.b.c or 192.168.x.y.
For example, if SNFS Distributed LAN is set up to use two Gigabit
Ethernet NICs (for a total bandwidth of 2 Gbits/s for Distributed LAN
I/O), each Distributed LAN Server and Distributed LAN Client should
have at least 4 NICs: 2 dedicated NICs for Distributed LAN, 1 dedicated
NIC for SNFS Metadata traffic, and 1 NIC for administrative access and
other traffic. Each of the NICs must be configured for a separate IP
subnet. So for example, if the subnets chosen for Distributed LAN are
192.168.1.0/24 and 192.168.2.0/24, each Distributed LAN Server and
each Distributed LAN Client must have one (and only one) NIC with an
IP address of 192.168.1.x, plus one (and only one) NIC with an IP
address of 192.168.2.y.
It is best practice to have all of the NICs for the same Distributed LAN
subnetwork directly connected to the same network switch. A router
between a Distributed LAN client and server could be easily
overwhelmed by the data rates required.
By contrast, Quantum recommends having the NICs for different
Distributed LAN subnetworks connected to different network switches, to
avoid overwhelming any one network switch.
All Distributed LAN subnetworks must be completely connected. This
typically means all Distributed LAN clients and all Distributed LAN
servers have dedicated NICs for each of the Distributed LAN
subnetworks. If a Distributed LAN client does not have a dedicated NIC
for a particular Distributed LAN subnetwork, it must be configured with
static routes to give it connectivity to each of the IP addresses advertised
by the Distributed LAN servers. Note that if a Distributed LAN client is
unable to connect to a particular server’s NIC, it will continue forever
trying to connect to it, potentially causing slow file system mounts,
repeated messages in error logs, and other problems.