Etherchannels with LACP
Last Updated: [last-modified] (UTC)
To follow on from the Etherchannel/LAG article, have a look at how to configure etherchannel…
LACP is a protocol that can be used to negotiate the parameters of the channel. This is used to bring the channel up initially, and then to monitor the channel, and add/remove links as required to respond to faults and repairs. LACP does not affect how traffic is load balanced across the channel.
LAGs were originally defined in IEEE standard 802.3ad, and later revised in 802.1ax. This defines link aggregation as a whole, which includes the LACP protocol, which sits in the data-link layer, above MAC addressing. Notice that this is a definition for Link Aggregation (LAGs), not etherchannel. Etherchannel, while very similar to LAGs is technically a Cisco proprietary technology (after all, it includes support for PAgP). However, as mentioned previously, these are compatible technologies.
LACP is still used when configuring vPC’s. This is because the connecting device still uses a normal LAG. For more information, see the Advanced vPC article.
Using LACP to Manage an Etherchannel
There are several advantages to using LACP to manage the channel. One advantage is that LACP is non-proprietary, so etherchannels/LAGs can be build between devices of different vendors. An example of this is building an etherchannel between a Cisco switch and a NetScaler appliance. A second advantage is that LACP will not allow the channel to form if there are problems between the two devices. This gives the administrator an opportunity to troubleshoot the issue before production traffic is added to the channel.
In some cases, there may be more links in the channel than the platform will allow to be active. For example, the switch may allow 8 active links, but there may be 16 links in the channel. LACP will negotiate between the two endpoints do decide which links will become active, and which will be standby. If an active link goes down, LACP will automatically cause a standby link to become active.
There is another unusual scenario where LACP is very useful. Imagine two switches are connected together, and a link fails. Normally, both ends will know that the link has failed, and the link will go down, and the link-status light will go off. Now imagine that the two switches are connected via another device, such as a media converter or some type of network bridge. In this case, if the device between the switches fail, the switches may not be able to detect the link loss themselves. LACP however, is able to detect this fault, and is able to dynamically remove this link from the group, and promote a standby link to active (if available).
The first party in an LACP conversation is called the Actor, and the second party is called the Partner. The actor and partner send LACP frames called LACPDU’s to each other to share state information. No ‘commands’ are sent between the end devices, just the information in the LACPDUs, which the receiving LACP engine uses to make its own decisions. LACPDUs are sent to multicast MAC address 01-80-C2-00-00-02.
LACPDU keep alives are sent on a regular basis. If these LACPDU’s stop being received, the LACP engine knows that the link is down, and can take appropriate action. In addition the regular keep-alives, LACPDU’s can be sent as required to notify the other party about state changes (such as link up/down events) or when one endpoint does not know enough information about the other.
LACP enabled channels can operate in either active or passive mode. In active mode, the device will proactively start sending LACPDU’s. In passive mode, the device will not send LACPDU’s unless it receives LACPDU’s from another device first. This means that if either or both devices are in active mode, a channel will be able to form. However, if both are in passive mode, the channel will be unable to form, as LACPDU’s are not exchanged.
Each link in an LACP channel is assigned a link number. This is used when setting port-priorities, and depending on implementation, may match the hashing values that are used for traffic distribution.
Each port can have a priority assigned. The priority is combined with the port number to get the Port Identifier. This value is used when deciding on which ports need to be active and which need to be standby. If the values on both ends match, the System ID / System Priority is considered.
Each LACP device can be assigned a System Priority, which (on a Cisco switch) is globally configured. The system priority is combined with the MAC address of the channel to create the System ID. The system ID is used to break a tie if port priorities are the same on both endpoints (lower values have a higher preference).
Each channel has an Administrative Key. This is an LACP locally significant value assigned to each LAG. On a Cisco switch, this is the ID of the port-channel (port-channels are discussed further in the configuration section). This is used to make sure that the ports on the remote device are in the same channel. For example, if a channel on a local switch receives different keys from the remote switch, then the ports on the remote switch are not all in the same channel.
A static etherchannel simply creates a channel without negotiating any parameters with the peer device. The upside to this is that the channel comes up instantly, as it doesn’t spend any time in negotiation. However, it does come with some significant downsides too.
The physical ports in a channel should all be configured the same way, in terms of link-speed, duplex, flow control, and so on. A dynamic channel will check that these values match before the channel comes up, and generate syslog messages if there is an error, to notify the admin. A static channel however, does not do this checking. This means that a simple misconfiguration on a port can cause problems for the whole channel.
Another concern is that the channel does not adapt well to dead links. If a link in a dynamic channel fails but the port remains up, the switch (or device) detects this, and stops trying to send traffic down that link. This could happen if there is a passive device somewhere in the link, such as an old fashioned hub or a media converter. In the case of a static channel however, the switch/device will not detect the problem, and continue to push traffic down the dead link, resulting in dropped frames.
So, with all the down sides to static etherchannels, when would it be appropriate to use them? In some cases it may be suitable, such as within a highly-controlled network core. The reason this can be suitable is that the core does not change frequently. This is different to the access layer, where devices are added and removed daily. While the previous concerns are still relevant, they do not pose as big a risk in the core, as once it’s working correctly it isn’t likely to change, and there’s not likely to be any passive devices in place. In general though, a dynamic channel is still recommended.
There are also cases where dynamic channels are not supported at all. One case is VMWare vSphere distributed switches (before version 5.1) or ESXi standard vSwitches. These are cases where the channel must be static, as these switches will simply ignore LACP frames. There are likely to be other devices out there which also do not support LACP.
An ASA cluster running IOS 9.0 is a case where LACP is supported, but the official recommendation is to statically configure the etherchannel. In newer versions (v9.6 is available at the time of writing) the recommendations have changed to recommend LACP. There is a lot of information on Cisco Live’s on-demand library in regard to this.
Traffic DistributionIt is commonly believed that LACP provides better traffic distribution than a static etherchannel. This is not true, as LACP is just the protocol used for negotiation the channel between the two endpoints. The load-balancing method used on the two endpoints decides how well the traffic is distributed.
Cisco – Configuring Etherchannels
WAHLNetwork – Demystifying LACP vs Static Etherchannel For vSphere
VMWare – Configuring LACP on an Uplink Port Group using the vSphere Web Client (2034277)
Cisco – Configuring a Cluster of ASAs
Cisco Support Forum – How do etherchannel HASH algorithm works and LOAD DISTRIBUTION happen?
Packet Pushers – Understand Etherchannel Load Balancing
PacketLife – EtherChannel considerations
Wikipedia – Link aggregation
IEEE – IEEE Standard for Link Aggregation
Cisco – Link Aggregation Control Protocol (LACP) (802.3ad) for Gigabit Interfaces
How Fantastic – Using LACP to create a backup link