Network Bonding: What standards/capabilities does the network switch need to support if bonding multiple interfaces?
Bonding (also known as trunking or teaming) is the joining of multiple network interfaces to form a single logical network interface which runs at a higher rate and provides redundancy if one of the links fails.
Wavestore default bonding is "round-robin" or "Static" mode which shares the load over all the operational links in both directions. This is widely used and the network links should be configured at both ends as bonded (trunked). This is sometimes known as Static trunking or Static bonding.
Most bonding issues occur because the ports are not bonded correctly at both ends. All network links (cables) between the bonded Wavestore and the Network Switch should be configured as bonded (trunked) at both ends, and normally all links in a bonded group should go between one Wavestore and one Switch.
When setting the bonding up: only connect one cable between the Wavestore and the Switch while setting bonding at both ends. A single link will work with or without bonding enabled and will work even if bonding is enabled at one end but not the other. Once bonding is set up at both ends, connect the other cables.
If it's desired to split some of the bonded links between multiple network switches then there are techniques such as LACP with MLAG/SMLT which can be used. LACP must be configured at both ends. This is an advanced topic which would need expert advise to configure it correctly and not all network switches support this.
LACP should only be selected on the network switch if LACP mode (see below) is selected on the Wavestore. In the default Wavestore bonding configuration, use Static bonding instead.
Advanced setup: Bonding modes
The bonding mode can be changed by editing the Network configuration section for the bond device in the Advanced Configuration editor and adding a parameter BONDING_OPTS="mode=XX" for mode XX. These bonding modes are provided by the Linux kernel and they are listed below.
For example, if you have an LACP (IEEE 802.3ad) compliant switch you can add BONDING_OPTS="mode=4" to the relevant [Network_n] section in the Configuration editor.
It is recommended that only mode 0 (Static) or mode 4 (LACP) are used on a Wavestore.
Here's some documentation for the various modes supported:
mode=0 (Static) (balance-rr)
Round-robin policy: Transmit packets in sequential order from the first available slave through the last. This mode provides load balancing and fault tolerance. This is the Wavestore default. Static Trunking should be set on the network switch for the connected ports.
mode=1 (active-backup)
Active-backup policy: Only one slave in the bond is active. A different slave becomes active if, and only if, the active slave fails. The bond's MAC address is externally visible on only one port (network adaptor) to avoid confusing the switch. This mode provides fault tolerance. The primary option affects the behaviour of this mode.
mode=2 (balance-xor)
XOR policy: Transmit based on [(source MAC address XOR'd with destination MAC address) modulo slave count]. This selects the same slave for each destination MAC address. This mode provides load balancing and fault tolerance.
mode=3 (broadcast)
Broadcast policy: transmits everything on all slave interfaces. This mode provides fault tolerance.
mode=4 (LACP) (802.3ad)
LACP IEEE 802.3ad Dynamic link aggregation. Creates aggregation groups that share the same speed and duplex settings. Utilizes all slaves in the active aggregator according to the 802.3ad specification. The switch will need to support this mode and LACP trunking should be set on the network switch for the connected ports.
mode=5 (balance-tlb)
Adaptive transmit load balancing: channel bonding that does not require any special switch support. The outgoing traffic is distributed according to the current load (computed relative to the speed) on each slave. Incoming traffic is received by the current slave. If the receiving slave fails, another slave takes over the MAC address of the failed receiving slave. This mode should not be used on a Wavestore server, because the changes to the MAC address can have undesirable effects.
mode=6 (balance-alb)
Adaptive load balancing: includes balance-tlb plus receive load balancing (rlb) for IPV4 traffic, and does not require any special switch support. The receive load balancing is achieved by ARP negotiation. The bonding driver intercepts the ARP Replies sent by the local system on their way out and overwrites the source hardware address with the unique hardware address of one of the slaves in the bond such that different peers use different hardware addresses for the server. This mode should not be used on a Wavestore server.