I’m a big an of the Cisco Fabric Extenders when it comes to getting more ports in a data center topology, I like the easy of management and simple layout for getting connections onto the FEX. However, after speaking with a few coworkers and friends, I came to the conclusion that the supported FEX topologies are still somewhat confusing between the Nexus line, and what is actually supported from a connectivity standpoint on the FEX’s.
Most of the information below is straight from Cisco’s documentation and Support Community.
The Nexus Line differs between models on support for the FEX connectivity. The 5K differs from the 7/9K switches in the topologies that are supported by Cisco. The 2000 FEX line is considered a ‘line card’ from the Nexus switches, and therefore has limitations on support for a variety of installations.
The FEX does provide a very simplified addition to data center connectivity for EOR and TOR designs, by providing the configuration from the parent switch rather than another management point, however the FEX design does impose some limitations:
- No East/West Switching: All traffic on a FEX is required to go North/South, as the FEX does not provide any local switching intelligence
- FEX Cannot have downstream switches connected: The FEX’s purpose is to be an edge device, and not aggregate connections from other switches in the data center or from access switches such as the Catalyst.
- No POE: Currently, the FEX line does not support POE, however future versions on the Cisco Roadmap will include POE.
If you’re planning a topology with FEX, ensure that any downstream switch connections will be connected to the parent switch, not the FEX.
FEX Models:
Currently 2 Lines of FEX switches in the wild, the 2200 and 2300 series. The 2200 and 2300 should be used especially when there is a need for 100MB connections in the data center, as the Nexus Core line of switches does not provide 100MB connectivity.
Nexus 5K:
The Nexus 5K switches are the only Nexus switches that support dual homing the FEX for fully redundant connectivity.
Single Homed Fabric Extender vPC Topology:
Each FEX is connected directly in Line to the parent 5K switch. In this topology, servers would be dual homed to multiple FEX.
Dual Homed Fabric Extender vPC Topology:
Each FEX is has multiple links to a pair of 5K switches in the same vPC Domain, and a single vPC (port-channel) is configured down to the FEX. In this topology, a server would be connected to a single FEX, but could be multi-homed to a 2nd fex.
Nexus 7K:
The Nexus 7K switches (7000 and 7700 series) do not allow for a multi-homed FEX topology, and only allow for a single homed FEX. The Cisco roadmap for the 7K does include a multi-homed FEX topology, however even as of NX-OS version 7.x this is still not supported.
Nexus 9K:
Much like the 7K, the 9K switches also do not support a multi-homed FEX topology. Much like the 7K, the Cisco roadmap does include a multi-homed FEX topology, however this is still TBD.
Official Cisco Support Topologies:
- Cisco Nexus 2300 platform single-connected to one upstream Cisco Nexus 5500, 5600, or 7700 platform or 6000 or 7000 Series Switch: In this deployment model, access-layer redundancy is achieved through redundant server connections to two upstream distributed modular systems using vPC (Cisco Nexus 5500, 5600, or 7700 platform or 6000 or 7000 Series) or server NIC teaming to two Cisco Nexus 2300 platform fabric extenders.
- Cisco Nexus 2300 platform dual-connected to two upstream Cisco Nexus 5500, 5600, or 7700 platform or 6000 or 7000 Series Switches (vPC): In this deployment model, access-layer redundancy is achieved with a combination of Cisco Nexus 2300 platform fabric extenders dual-connected to an upstream parent switch and server NIC teaming.
- Enhanced vPC (EvPC): In this deployment model, access-layer redundancy is achieved in two ways: through redundant connections between the Cisco Nexus 2300 platform and the Cisco Nexus parent switches using vPC; and through redundant server connections to two fabric extenders using vPC and active-active server NIC teaming. This scenario is supported only with the Cisco Nexus 5500 or 5600 platform or 6000 Series used as upstream switches. The Cisco Nexus 7000 Series and 7700 platform currently are not supported as upstream switches in this scenario.
- vPC+: In this deployment model, access-layer redundancy is achieved through server vPC, fabric extender vPC, and EvPC. In addition, a vPC+ domain allows the Cisco Nexus parent switch and the fabric extenders to be viewed as a single virtual switch in a Cisco FabricPath network.
Happy Reading, hope this Helps!