vPC and Routing
Last Updated: [last-modified] (UTC)
There has been some confusion around the question; does vPC support routing? We’ll try to clear that up a bit here.
There are two possibilities that this questions covers. Firstly, can the vPC switches and links carry Layer-3 traffic? And secondly, can dynamic routing be used with vPC?
Rather watch the movie than read the book?
It is helpful to understand vPC best practices first
[maxbutton id=”4″ text=”Best Practices” url=”https://networkdirection.net/vPC+Design+and+Best+Practices”]
Routing over a vPC or Peer-Link
We often hear that vPC is a layer-2 technology. But Layer-3 packets can still travel over these vPC links. Consider the example topology below. One host is connected with vPC. The other host is connected to only one of the switches (an orphan port). These hosts still send packets to each other, just like they would over any normal switch.
The vPC switches can even run HSRP or VRRP, and be the default gateway. In this particular example, traffic may follow either vPC uplink. Incidentally, the duplicate frame prevention rule does not apply in this case, one of the hosts uses an orphan port.
Dynamic Routing over the Peer-Link
So, with that cleared up, what about running a dynamic routing protocol such as OSFP? There are several possible scenarios where this may be desirable.
OSPF is used here as an example. This also applies to other dynamic routing protocols like EIGRP. Peering in this context refers to forming neighbour relationships between two devices.
Between the vPC Pair
The first scenario, shown below, is peering between the two vPC switches.
This is supported. Peering happens across the peer-link.
Between a Router and one vPC Nexus
The next scenario is when a router, not vPC connected, needs to peer with the vPC pair. As it’s connected to one of the switches, peering traffic needs to pass through one switch to get to the second.
This is supported, with some guidelines.
If the vPC switches are Nexus 3500’s, 5000’s, or 6000’s, the topology above is fine. Peering traffic will traverse the vPC peer-link. In this topology, Cisco recommends using the peer-gateway command.
Other platforms do not support the exact topology above. This is because of the peer-link. A 9000 series switch, for example, does not support OSPF peering across the peer-link.
That’s not the end of the story though. There is another topology which is supported, as shown below. This uses a separate layer-2 link for peering. This link should have a VLAN assigned for peering traffic. Prune this VLAN from the peer-link. This forces the peering traffic over the layer-2 link.
Any platform that supports the first topology does not support the second. That is, you must choose the correct topology for your Nexus model.
For more information on how the peer-gateway command works, see the Traffic Flow section of the Advanced vPC article
[maxbutton id=”4″ text=”Peer Gateway” url=”https://networkdirection.net/Advanced+vPC#Traffic_Flow”]
Dynamic Routing over vPC
vPC Router to Orphan Router
In another scenario, a router connected to an orphan port needs to peer with a router connected by vPC.
This is supported.
This also applies to peering between two vPC connected routers.
vPC Router to vPC Pair
Now the tricky one. A vPC connected router needs to peer with the Nexus switches.
This may be supported.
But first, a word on why this can be a problem. Let’s assume that OSPF is used as an example. OSPF sends regular hello packets, with a TTL of 1. The TTL is set to 1 becauseOSPF neighbours need to be adjacent. If a router receives a hello packet, and tries to route it, the TTL will decrement to zero, and drop the hello.
Now imagine that a vPC connected router sends a hello packet. The packet may pass either over either link in the vPC. This means that the wrong switch (acting as a router) gets the hello. It decrements the TTL, and drops the packet.
The result is that the neighbourships will be unstable. They will often get stuck in INIT, TWO-WAY, or EXSTART.
Routing over vPC is something that many have wanted for some time. Cisco are finally starting to let us have what we want.
From NXOS 7.2, this is supported on the N7K. The caveat is that they need to be using F-Series modules. From 7.0(3)I5(1), the N9K also adds support. This has been supported on the N9K in ACI mode for some time.
A workaround to this would be to run extra Layer-2 links from the router to the switches. These would be orphan ports, and they should use a special VLAN for peering. Prune this VLAN from the peer-link.
There is a small amount of configuration on the Nexus. First, use peer-gateway. This enable either switch to accept packets on behalf of it’s peer.
The second command is layer3 peer-router. This is the command that tells the Nexus to not decrement the TTL.
If you’re mixing dynamic routing and vPC, work out your topology during the design phase. Don’t wait until something goes wrong, because a topology redesign may be costly. If you’re not sure about your topology, contact the TAC. They won’t recommend topologies or help design the network. They can advise whether they will support your proposed topology.
If there are a few SVI’s used in the Nexus peer switches, put them into passive mode by default. Choose a specific SVI for peering, and disable passive mode. This prevents the switches from forming unnecessary neighbour relationships with every SVI. Also, disable ip redirects on the SVI’s. This prevents unnecessary redirections to a switch’s peer.
Routing over vPC is better used in the aggregation layer. Uplinks to the core are usually routed interfaces.
Cisco – Supported Topologies for Routing over Virtual Port Channel on Nexus Platforms
Brad Hedlund – Routing over Nexus 7000 vPC peer-link? Yes and No
Cisco – Cisco Nexus 7000 Series NX-OS Release Notes, Release 7.2
Cisco Live (about 1:02:00) – BRKDCT-2378 – VPC Best Practices and Design on NX OS
6 thoughts on “vPC and Routing Protocols”
Hanadel 2017-11-25 02:16
Thank you, I just watched the video and it is helpful.
Luke Robertson 2017-11-25 14:48
I’m glad you found it helpful. Thanks for the feedback!
I have a scenario like your first topology in “Between a Router and one vPC Nexus” and I am trying to establish an OSPF adj via peer link to second Nexus only. My devices are nexus 9396px. The ospf adj is stucked at EXSTART state. I have the layer3 peer-router and the peer-gateway feature enabled. I also cannot ping the SVI in the second Nexus via PeerLink. If I disable the peer-gateway I can ping the SVI and the adj goes in Full state. Can you provide any insight on this?
This has confused me more. What are the ospf configurations on the nexus, is the neighbor formed via vlans or port-channels, is there a seperate link between the vpc peers for layer 3. Which vlan interface has the IP address. Does the vpc domain form a neighbour with the downstream port-channel or another SVI…………NONE of this has been cleared up. All i want is for one switch to form a layer 3 ospf neighbor with the 2 x N9K boxes through a port-channel.
yes you can do this, but you need the extra commands shown in this article
i cant believe my comment has still not been addressed ?