Pages

6.17.2013

Cisco Nexus - Part 4.1 - Virtual Port-Channel (vPC) Design

What!?! You thought we would jump straight into configuring a vPC? Well, sorry for yah. Before we can do any configuration we need to go over what makes vPC tick and how you can set it up in your Nexus environment.

Besides, any monkey at the zoo can be trained how to bang on a keyboard and configure a vPC, but a real engineer needs to know how the technology works under the hood, then the configuration comes easy.

So lets get started


Links:

Cisco NX-OS Virtual PortChannel: Fundamental Design Concepts with NX-OS 5.0
Spanning Tree Design Guidelines for Cisco NX-OS Software and vPC
vPC Best Practices Checklist and Designing VPC and Routing by Pete Welcher

Actually, anything by Mr. Welcher over at the Netcraftsmen blog I suggest reading. 


vPC Types


There a several different ways a vPC can be setup and they all have design considerations. Here is the topology we are going to use in this post to help visualize each type of vPC. For those who have not had a lot of exposure to Nexus vPC there are a few unfamiliar items. Most obvious are the dots. vPC ports are usually designated with colored dots to represent vPC membership.

Straght-through (Single-homed)
  • Each Fabric Extender (FEX) is single-homed to one N5k or N7k
  • Server Port-Channel can only form when each uplink connects to a separate N2k
  • Unable to form if both FEXs uplink to the same N5k

In the diagram above Server1 (uplinking to N2k101 and N2k202) is setup in a straght-through design. This design is prefered for servers with multiple NICs and can provide their own redundancy.
   
Dual-homed (Active/Active)
  • FEX is dual-homed to multiple N5ks with vPCs
  • Server Port-Channel cannot be split between N2ks
  • Can dual-home with Active/Active or Active/Standby using TLB (Transmit Load Balancing) if enhanced vPC is not supported
You can dual-home but technically that is called enhanced vPC and only supported in newer versions of NX-OS.

In the diagram above any server with a bond connected to either N2k102 or N2k201 would be considered a dual-homed setup. This design is preferred for servers with only a single NIC. Redundancy can at least be built into the FEX but the server is S.O.L if the NIC or uplink to the FEX goes down.
   
Enhanced vPC 
  • Cisco NX-OS 5.1(3)N1(1) and up supports Enhanced vPC on the Cisco Nexus 5500 series
  • No support for the older Nexus 5010 and 5020 devices
  • Different models of FEXs can be deployed together in an Enhanced vPC topology
In the diagram above Server2 (vPC D) is setup in an enhanced vPC design. Im on the fence if I would use this design so input from anyone would be great.

vPC Components and Terms


vPC Domain - Group of two N5k, N6k or N7k devices (Peers) configured to host vPCs. Once a domain is formed an election is held to determine the primary and secondary role.
  • Lowest priority wins
  • No preemption
  • Only primary sends BPDUs unless using the command peer-switch
    • With peer-switch both primary and secondary send BPDUs with the same BridgeID

Peers - Each Nexus switch in a domain is considered a peer. It is highly recommended to align the STP and vPC domain primary to the same switch.

Peer Link - Dedicated link between vPC peers used to communicate and share vPC information. Even though the traffic utilization will not be high it is highly advised to setup your peer link in a port-channel for redundancy.
        Used to:
    • Simulate a single control plan between two vPC peers
    • Synchronize MAC addresses between peers
    • Transport for multicast
    • Communication of orphaned ports
    • Carries HSRP frames

Peer-Keepalive Link - Layer 3 routed link used to keep track of vPC peer status and availability. Mostly used to insure a split-brain scenario does not ruin your day when the peer-link fails.

Cisco recommends running the peer-link over a management VRF and on the N5k using the mgmt0 interface. Cisco also recommends to not connect the peer-keepalive link interfaces in a point-to-point fasion, but to use a switched network.

These recommendations kinda forces you to either dedicate your sole management port for simple keepalives or have a separate switched network to handle the keep-alive traffic. To work around its been  suggested to configure a dedicated vlan across the peer-link port-channel.
   
vPC Ports - Any port assigned to a vPC channel group. The colored dots in the diagram above all represent vPC ports.

vPC VLANS – Any VLAN configured on the peer link
        VLAN must exists on both peers
        VLAN must exist on the peer link
        VLAN must appear in the allowed list of the trunk

Orphaned Ports - Ports connecting devices in a non-vPC mode to a vPC topology.

In the above diagram Server3 is connected to an orphaned port. It is highley recommended that all orphaned ports reside on the vPC primary due to loss of communication during a peer-link failure.

Cisco Fabric Services (CFS) - Service used by vPC for peer scychronization of foward-plane information.
  • Travels over the peer-link
  • Consistency checks are used by peers (similaur to traditional Port-channels) to verify vPC member ports can form a port-channel. 
    • Types of Inconsistencies:
      • Type-1 - Error will cause the vPC to suspend
        • Global - Affects all vPC member ports but not non-vPC ports
        • Interface-Specific - Affects only the interface
      • Type-2 - Warning
        Check the status with show vpc consistency-parameter 

OK, now that we have the theory out of the way and everyone has a good understanding of vPC we will next dig into vPC configuration and look at the various failure scenarios and the impact of each.