Our website is made possible by displaying online advertisements to our visitors.
Please consider supporting us by disabling your ad blocker.
Loading...
 

Advertisement

Network Direction

Dynamic Routing and FEX

Wednesday August 30, 2017

Dynamic Routing and Fabric Extenders

The Problem

A few weeks ago I was working on a customer's network when I found an OSPF problem. For some reason, an ASA wouldn't peer with a Nexus switch. To make it a bit weirder, the problem only happened on the default VRF, and only with OSPFv3. On the Nexus side, I could see the ASA neighbour, but it was stuck in INIT. On the ASA side, I couldn't see the neighbour at all.

 

After hours of troubleshooting, I got the TAC involved. But before I tell you what they found, let me show you the topology I was working with.

ASA+FEX  

 

There is a pair of ASA's, running Threat Defence. They are an active-standby failover pair.

The ASAs connect to a pair of Nexus Fabric Extenders (FEX). The FEX's connect to Nexus 9000 series parent switches in straight through mode.

The ASA's to the switches with vPC. The layer3 peer-router command makes dynamic routing over vPC possible. Neighbours use SVI's for peering.

 

ND Logo
Virtual Port Channels

Take a look at how vPC's work

Virtual Port ChannelsRouting over vPC

 

 

 

The Cause

Speed Limit   I had fallen victim to missing a footnote in one of Cisco's guides. Have a look at the article Cisco Nexus 2000 Series NX-OS Fabric Extender Configuration Guide for Cisco Nexus 9000 Series Switches. Under the Guidelines and Limitations section, the article points out that when a FEX is connected to a 9000 series parent, "the queuing capability on the FEX host interface is limited."

So what does this mean? It means that if we connect a router to a layer-2 FEX port and use SVI's, we cannot reliably run a routing protocol. The port will not prioritise control plane traffic over other types of traffic. This results in dynamic routing traffic getting dropped.

This is the only time I've seen a problem in a topology. Most of the time it works fine. But, the bottom line is that it's an unsupported design.

 

 

 

The Solution

Rubik Solved   There are three solutions that come to mind. The first is to run static routes. This issue does not affect static routing, as it does not use any control plane messages.

The second option is to bypass the FEX and connect straight to the parent switch. Depending on your port requirements, this may need extra SFPs.

The third possibility is to use routed ports on the FEX. This won't be possible in all cases, but is generally preferred to peering over SVI's anyway. Be aware though, that not all FEX models support routed ports.

 

Have you had this problem? Any other ideas to solve it? Share your thoughts in the comments below.

 


Advertisement