Our website is made possible by displaying online advertisements to our visitors.
Please consider supporting us by disabling your ad blocker.
Loading...
 

Advertisement

The Basics of FabricPath

Thread  

Introduction

Spanning-tree has played a vital role in layer-2 networks for many years. It allows redundant loop-free topologies, and quick recovery when links fail. Unfortunately, it has a critical downside. While allowing many uplinks, loop the prevention algorithm will disable some of them. This limits the bandwidth and scalability of the network. Additionally, spanning-tree implements a single Root Bridge. This often leads to sub-optimal paths.

In the example below, one of the links is in a blocking state. The switches at the bottom need to send traffic toward the root bridge to communicate. This results in a sub-optimal path.


Fabricpath STP  

 

One way to address this problem is with etherchannels. This allows many active links between two devices. The problem is that it won't allow uplinks to more than one device at a time.

An extension to EtherChannels is virtual port-channels. vPC's allow uplinks to two different switches to be active at the same time. This is great for a host, such as a server. It's not great for many switches, as it only allows connections to two upstream switches per vPC. It is not scalable for this purpose.

Cisco's Fabricpath is scalable and allows for many active uplinks. It does this by creating a layer-2 fabric or mesh of active links between switches. It is able to make all the links active by taking layer-3 concepts, and applying them at layer-2. FabricPath uses the IS-IS routing protocol. It supports multi-area design, and is not dependent on IP.

 

Fabricpath is a Cisco proprietary protocol, found on some of the Nexus switch range, such as the N5K and N7K. Fabricpath is not supported on the N3K or N9K series. These switches support VxLAN based fabric. This is an alternate way to put non-blocking uplinks in place.



 


Operation

Topology

The classic three-tier network topology uses core, aggregation, and access layers. These use a mixture of routing (layer-3) and spanning-tree or vPC (layer-2). This segments the network into blocks. This can be an advantage, especially in networks where north-south traffic is common. In the data centre, this may not be desirable.


Fabricpath Classic Hierarchy  

 

In the data centre, east-west traffic is more common. This is a lot of traffic is between servers residing at the same layer. Breaking the network into blocks could mean that traffic may first travel north into the core. Then it travels south into another block. These blocks may be in racks sitting side-by-side. You can see how this is inefficient.

Fabricpath uses the spine/leaf topology. Each switch is classified as a spine or leaf. As shown below, leaves connect to hosts and other devices. Spines only direct traffic within the fabric and do not have hosts connected. There is a full mesh of links between spines and leaves. Every leaf has a connection to every spine, and every connection is non-blocking.​


 

Fabricpath Spine  
 

This topology favours east-west traffic patterns. All the uplinks in the fabric are layer-2, but they behave like layer-3 ECMP paths.


Enable FabricPath by installing the Fabricpath feature-set, then enabling the feature-set.

 

install feature-set fabricpath
feature-set fabricpath

 


 

VLANs

In Fabricpath, VLANs can be in either Classical Ethernet (CE) mode or Fabricpath mode. The VLAN has the mode applied in VLAN config, as shown below.

 

vlan 100
  mode fabricpath

 

 

VLAN mode is locally significant to the switch or VDC. This means that there is nothing passed from switch to switch to identify a VLAN as CE or FabricPath.



 

Ports

Needle
Fabricpath defines two types of ports; Edge and Core.

Edge ports connect to regular Classical Ethernet devices. These ports behave as normal CE ports. They use traditional MAC learning, and don't encapsulate traffic with Fabricpath headers.

Edge ports may use either CE or Fabricpath VLANs. If they need to carry traffic from CE devices to the Fabricpath domain, the VLAN must be in Fabricpath mode. The port can be access or trunk.

 

Core ports are Fabricpath aware ports that connect to other core ports. These connections form Fabricpath adjancencies. All traffic passed between Fabricpath adjacancies are encapsulated with a Fabricpath header.

Core ports only carry VLANs in Fabricpath mode

Fabricpath Ports  


To configure a Fabricpath port:

 

interface eth 1/1
  switchport mode fabricpath


 

Roles

A Fabricpath switch can be an ingress switch, a core switch or an egress switch. Ingress and egress switches are also called Layer-2 Gateways.

An ingress switch contains an edge port and receives a Classical Ethernet frame. This is a leaf switch, as shown earlier. The switch does a lookup to determine which Fabricpath interface to forward the frame to.

A core switch contains a core port. This is a spine switch. When a core port receives a Fabricpath frame, it determines which interface to send the frame out of. To do this, it looks at the Fabricpath header on the frame. The egress port will also be a core port.

An egress switch receives a Fabricpath frame on a core port. It then determines which interface to send the frame from. The egress port is an edge port, so it strips the Fabricpath header from the frame. The frame is then sent as a CE frame. Traditional MAC learning only happens on an egress edge port. Core ports do not need MAC addresses, as they use the SID in the Fabricpath header.


 

 


Control and Data Plane

Control Plane

The Fabricpath Control Plane disseminates reachability information. This is like a routing protocol, that builds a view of subnets in the routing table.

Each switch in the Fabricpath domain has a 12-bit switch identifier. An administrator may assign this identifier. Or, Dynamic Resource Allocation Protocol (DRAP) could assign it. The Switch ID is used frequently in the Fabricpath header.

Each switch will also build adjacencies, which are based on IS-IS adjacencies. These form when an interface connects to another Fabricpath switch.

When the adjacencies have formed, they exchange layer-2 routing information. Routes aren't really used at layer-2, but applying IS-IS concepts here make it possible. The Unicast Layer-2 Routing Information Base (U2RIB) is then populated with the routing information.

An SPF algorithm is then run (just like IS-IS) to get the path costs. The lowest path cost is the best path. FabricPath can use as many as 16 ECMP's (Equal Cost Multi-Paths). This is what makes Fabricpath so scalable.

Fabricpath can also handle BUM (Broadcast, Unknown unicast, Multicast) traffic. This traffic requires special handling. Without FabricPath this type of traffic would use Classical Ethernet techniques such as ARP. These techniques just aren't used in pure Fabricpath. This is done with multi-destination trees, which is a bit like a multicast tree. A single switch is elected as the root of the tree. The switch priority, system ID and switch ID influence the election.


 

The Fabricpath Header

The CE frame contains a MAC address section in it's header. This includes the Source AddressDestination Address, and the Type. When a frame enters a FabricPath domain, a new Fabricpath header is added to the front of the frame. The CE MAC address section is still there, but it's now called the iMAC, or Inside MAC. The Fabricpath header added to the front is called the OMAC, or Outside MAC.

The OMAC is made upof three fields. The Outer Destination Address (ODA), the Outer Source Address (OSA), and the FTag. The Ftag contains the ethertype and TTL fields.

The ODA and OSA are the Fabricpath destination and source addresses. These are the switch ID's within the Fabricpath domain.


 

 

Data Plane

The data plane encapsulates and decapsulates CE frames, and forwards of Fabricpath frames. Fabricpath uses a MAC-in-MAC style encapsulation. It prepends a 16-byte Fabricpath header to each frame. It removes the original CRC field and generates a new one.

The Fabricpath header includes a TTL field. Just like the TTL in the IP header, this prevents loops and storms. This is part of what makes it possible to have many active links. This is in contrast to Classical Ethernet, which has no built-in TTL. CE relies on external mechanisms such as spanning-tree for loop prevention.

A FabricPath frame is larger than a CE frame, due to the extra header. FabricPath switches should use jumbo frames, or have the MTU increased.


 

 


Monitoring

Check if Fabricpath is enabled

Switch# show feature-set
Feature Set Name      ID        State
--------------------  --------  --------
fabricpath             2          enabled
fabric                 7          uninstalled
virtualization         5          uninstalled

 

See Fabricpath VLANs

Switch# show fabricpath topology vlan
Topo-Description                 Topo-ID   Configured VLAN List
-------------------------------- --------- -------------------------------------
0                                0         0-4095

 

Show the Fabricpath switch ID

Switch# show fabricpath switch-id
                        FABRICPATH SWITCH-ID TABLE
Legend: '*' - this system
        '[E]' - local Emulated Switch-id
        '[A]' - local Anycast Switch-id
Total Switch-ids: 4
=============================================================================
    SWITCH-ID      SYSTEM-ID       FLAGS         STATE    STATIC  EMULATED/
                                                                  ANYCAST
--------------+----------------+------------+-----------+--------------------
[E] 100         547f.eea7.6501    Primary     Confirmed No      Yes
    100         547f.eeb4.9bfc    Primary     Confirmed No      Yes
*   1357        547f.eea7.6501    Primary     Confirmed No      No
    2995        547f.eeb4.9bfc    Primary     Confirmed No      No

 

Show ISIS routing information

Switch# show fabricpath isis

Fabricpath IS-IS domain : default
  System ID : 547f.eea7.6501  IS-Type : L1 Fabric-Control SVI: Unknown
  SAP : 432  Queue Handle : 11
  Maximum LSP MTU: 1492
  Graceful Restart enabled. State: Inactive
  Last graceful restart status : none
  Graceful Restart holding time:60
  Metric-style : advertise(wide), accept(wide)
  Start-Mode: Complete [Start-type configuration]
  Area address(es) :
    00
  Process is up and running
  CIB ID: 1
  Interfaces supported by Fabricpath IS-IS :
    port-channel1
  Level 1
  Authentication type and keychain not configured
  Authentication check specified
  LSP Lifetime: 1200
  L1 LSP GEN interval- Max:8000 Initial:50      Second:50
  L1 SPF Interval- Max:8000     Initial:50      Second:50
  MT-0 Ref-Bw: 400000
        Max-Path: 16
  Address family Swid unicast :
    Number of interface : 1
    Distance : 115
  L1 Next SPF: Inactive

 

Check if there is a Fabricpath conflict

Switch# Show fabricpath conflict all
No Fabricpath ports in a state of resource conflict.


No Switch id Conflicts

No transitions in progress


 

 

Twitter: @NetwrkDirection

 


Suggested Articles

 


 

 

References

Cisco - Nexus 7000 FabricPath

Cisco - Cisco FabricPath

Cisco - Configuring FabricPath Switching

Cisco - Configuring FabricPath Interfaces

 

Last update 2017-08-29 08:58


Advertisement