Pages

5.31.2013

Cisco Nexus - Part 3 - Connecting Cisco FEX to Nexus 5000

OK, now that we have the basic theory down we can start configuring some Nexus gear.

First up we will connect a couple fabric extenders (FEX) to a single Nexus 5548 (N5k).

Lets get started!

Configuration


Unlike IOS, NX-OS has most all services disabled by defualt so first we need to enable the FEX service.



Once the FEXs are connected to the N5K and the N5k will show the interfaces as "Fabric Extenders" with a status of "sfpInvalid".



By default all N5k ports are setup as access ports. The port status shows invalid because these SFPs are only capable of working on fabric ports. 

I have two 2248s connected to this N5K. One to ports Eth1/29 and Eth1/30 and the second to ports Eth1/31 and Eth1/32.

Its a good idea to always setup your fabric uplinks into a port-channel. Just keep in mind you can only have a single port-channel per FEX. I will be using Po101 and Po102 respectively.



Thats it!

Now the FEXs are attached and configured on your N5K. Once connected and configured, the N5k will verify the software version loaded on the FEX and upload the version running on the N5k if needed.

This process can take a bit of time and the show fex command will display "Image Download" in the FEX State section. Show fex detail will also give you more information about the FEX and the download status.



Now would be a good time to set some optional setting for each FEX such as description and max-pinning.

Max-pinning allows you to "pin" a set number of host ports to each fabric port. Cisco describes two types of pinning
  • Static pinning - Each port is pinned to a spicifc fabric port. If that fabric port goes down all host ports pinned to that fabric port go down. 
  • PortChannel - Uses standard portchannel logic to distribute traffic dynamically.
Since we have the FEX ports configured in a port-channel the N5K will automatically use the second option.

By default max-pinning is always set to 1. I have no reason why Im configuring it below, since we are using portchannels, outside of showing you the commands.

For more information on static pinning and port assignments when using static pinning check out Cisco's design guide here.



Under the FEX configuration mode you also have the options to enable a beacon LED on the FEX so you can identify the unit in the rack. Very helpful when racking or troubleshooting a large setup.



Verification


After the image download completes we can make sure the interfaces are up and FEXs are online.




Now all FEX interfaces will be attached to the N5K.

The interface numbering is assigned just like line cards in a chassis: Eth fex /slot / port
For this example FEX ports are Eth101/1/1-48 and Eth102/1/1-48

Also note all ethernet interfaces start with Eth. No more Fa1/0/1, Gi1/0/1 Ten1/0/1 or for you veterans E1/0/1. Its all Eth1/0/1



or



Host Interface Configuration


FEX host interfaces are configured just like normal interfaces.



Now that we have our FEXs connected we can get to some more advanced configurations.

Up next Virtual PortChannels!