Onto the Logical Router….
In Part 1 of my Deploying NSX series, we covered the prep of NSX in the environment, including deploying the NSX Manager appliance, deploying NSX Controllers and vSphere host preparation. In Part 2 this part of the series, we covered the creation of Logical Switches and our NSX Edge, which consist of our Edge Services Gateway (Providing DHCP, Firewall, VPN, NAT, Routing and Load Balancing capabilities). In our 3rd part in the series, we’ll cover the deployment of the Logical Router, which provides our routing and bridging for the existing networks, as well as configuring routing to get traffic into and out of our new NSX environment.
Logical Router Functions
The Logical Router provides us 2 functions, which we’ll dive a little deeper into below.
- Layer 2 Bridging to the Physical Network
- Layer 3 Routing between Logical Switches
Layer 2 Bridging
The concept of Layer 2 Bridging with NSX is in the simplest terms a trunk/access port from switch to switch. The Logical Router’s Layer 2 Bridge will allow us to easily migrate workloads from traditional VLAN based VM’s into NSX, with no IP changes. We’re just extending that Layer 2 Broadcast domain up into the NSX environment. The Gateway will still exist elsewhere.
Layer 3 Routing
The Logical Router can provide the East/West routing of traffic between Logical Switches. This is your typical Layer 3 switch, but the beauty of the Logical Router is that you’re just doing this on-host, you’re doing it across the entire NSX environment.
Deploying the Logical Router
Deploying the Logical Router is extremely similar to what we did with the Edge Gateway. You can deploy the Logical router appliances in HA mode or not, depending on the requirements. We’ll be deploying in HA mode for this series.
We’ll set the Name and Description, Appliance Settings, such as password and SSH, and location of appliance (cluster, datastore, folder, etc).
Once we get to Configure Interfaces, much like with the Edge deployment we need to think/plan how we’re connecting this Logical Router. We know 2 things from past parts of series: First, the Logical Router will connect to the Edge Gateway for North/South traffic based on our diagram below, and second we will have a number of LIFS (Logical Interfaces).
The VM facing LIFS will be considered Internal, and the Logical Router —> Edge Gateway LIF, well that’s considered an uplink. When we deployed the Edge Gateway we used an Uplink interface on a /30 network to tie us out to the Physical Network via a DvS Port Group. With the Logical Router, one thing that is different is the Management Interface. At first glance it’s a odd interface – you don’t manage the Logical Router via this interface, but VMware has produced a very good Kbase detailing the Management interface. So for this deployment, we’ll just choose a DvS Port Group that exists already.
For connectivity between the Logical Router and the Edge Gateway, we could go one of 2 paths for the uplink: First, we could use an existing Port Group on the DvS to map the interfaces to, or we could create a new Logical Switch specific to this connectivity.
My personal preference is to create a new Logical Switch, and tie that to the Uplink LIF for the Edge Gateway and Logical Router.
For the Uplink interface between the Logical Router and Edge Gateway, we’ll create that Transit interface, but a slight departure from your typical Layer 3 Transit interface of a /30 network. We’ll touch on it a bit later why we need this with OSPF, but for this purpose we’re using a /29 network to provide us more than 2 usable IP’s.
In addition to the Uplink between the Logical Router and Edge appliances, I want to create an interface on the Logical Router and tie that to one of my Logical Switches – now we’re starting to build out our NSX VM network. For the Server VM’s that will exist inside the NSX environment, I’ve created a LIF and a subnet – really this is no different than typing out int vlan 10 and assigning an ip address onto a switch.
So, we’re just about done deploying out Logical Router. Let’s review what we’ve done so far:
- Deployed a new Logical Router in HA mode
- Assigned the Resource Pools for the appliances
- Created a Logical Switch to connect the Edge Gateway and Logical Router – We could have done this when we created our Logical Switches
- Created an LIF to link the Logical Router and Edge Gateway
- Created an LIF for Server VMs
We can see in the Web Client we have both our NSX Edge and Logical Router deployed. Double clicking on either of those items will take you directly into the configuration of the specific appliance and features.
We still have 2 more action items to complete before we can move VM’s into the NSX environment, and pass traffic.
- Create interface on Edge Gateway to Logical Router (reverse of what we did during Logical Router deployment)
- Configure Routing for traffic into and out of NSX environment – either thru Static Routing or Dynamic Routing
Configure Edge Gateway Interface for Logical Router
Adding interfaces to the Edge Gateway and Logical Routers are very simple, and since the leg work has been done for us to add an interface for the Edge Gateway to Logical Router. Since we have already defined vNIC0 on the Edge Gateway as our Uplink to the Physical network, we’ll use vNIC1 as the Edge —> Router Layer 3 link, as shown below. Remember we needed to use a /29 network instead of /30, which will become apparent in the next section.
On our Edge Gateway we now have 2 active interfaces:
And on our Logical Router, we have 2 active interfaces:
So, from the network can we ping the addresses of 10.105.255.9/10 or 10.10.224.1 from our physical network? Unless you’ve jumped ahead, no we can’t. Why can’t we? Well, the physical network doesn’t know about these networks. We proved in our last part in the series that we could ping 10.105.255.2, but that’s because it’s on a vlan that our physical switches know about, being VLAN 103.
From our core switch, if we do a sh ip route command on our addresses, only the Edge Uplink interface is accessible
C3750#sh ip route 10.105.255.2
Routing entry for 10.105.255.0/30
Known via “connected”, distance 0, metric 0 (connected, via interface)
Routing Descriptor Blocks:
* directly connected, via Vlan103
Route metric is 0, traffic share count is 1
C3750#sh ip route 10.105.255.9
% Subnet not in table
C3750#sh ip route 10.100.224.1
% Subnet not in table
C3750#
NSX Traffic Routing
NSX comes with a variety of supported routing functions, both static and dynamic. Simplest to start but difficult to maintain is static routing, and we’d do that just like we would on a standard network.
For Dynamic routing protocols, NSX will support OSPF, BGP and IS-IS. If you’re a Cisco only shop running EIGRP, well you’re out of luck – almost. You can setup OSPF in NSX, and configure a limited OSPF configuration on the Cisco side, and redistribute OSPF into EIGRP on the physical side. While I prefer EIGRP, in my lab I am running OSPF for the sole purpose of NSX. My underlying infrastructure has OSPF running on the default area 0, so nothing fancy there.
To start to get network visibility into NSX, we’ll start by enabling OSPF on the Edge Gateway. Enabling OSPF on the Edge gateway is a simple 3 step process. Always remember to click Publish Changes, or else it won’t be saved!!!
- We need to enable the Router ID for OSPF – This is done under Global Configuration, and will default to our Edge Uplink interface
- Next, enable OSPF. I’m not using the Default Originate at this time.
- Finally, we need to map our OSPF Area to an interface. We’ll start by mapping Area 0 to our uplink interface, which is towards our physical network.
If we look at our OSPF neighbors on the Cisco switch, we can see that we’ve now formed an adjacency with the NSX Edge appliance. We can also verify our OSPF neighbors from the Edge Gateway.
Cisco Switch:
C3750#sh ip ospf neighbor
Neighbor ID Pri State Dead Time Address Interface
10.105.255.2 128 FULL/BDR 00:00:37 10.105.255.2 Vlan103
10.105.254.1 1 FULL/BDR 00:00:39 10.105.254.1 GigabitEthernet1/0/48
Edge Gateway:
nsxedge.etherbacon.net-0> sh ip ospf neighbor
Neighbor ID Priority Address Dead Time State Interface
10.105.254.2 1 10.105.255.1 30 Full/DR vNic_0
This is great, but we’re still not fully there yet. We’ve formed adjacencies with our Edge to our physical network, but now we need to get adjacencies formed between our Edge and Logical Router, and get the subnets the Logical Router owns out onto the network.
We have one more step on the Edge Gateway, but let’s hop over to the Logical Router and enable OSPF. Enabling OSPF on the Logical Router is almost identical to the Edge Gateway, with a few minor differences. Let’s go thru the steps on the Logical Router:
- Configure the Router ID for the Logical Router, which is done under Global Configuration.
- Enable OSPF and input a Protocol and Forwarding Address. This is the key for using a /29 network rather than a /30 in the Subnet for the Edge to Logical Router connection. The protocol address is similar to the management interface on the Logical Router, in that is remains local to the VM, but is still needed on the same subnet as the Forwarding address.
- Next we’ll create a new OSPF area. While we could use 0, it’s not recommend. We’ll create a new Area with the value of 10, and then map Area 10 to our Uplink interface.
Almost there…..
Finally, we need to match up the area we created on the Logical Router on the Edge Gateway. We need to create a new area (10) on the Edge Gateway, and map that to our Edge-Router-Uplink interface.
Now let’s see what we’ve got. If we run the command sh ip route 10.10.224.0 on our Cisco switches we’ll now see that we’re learning about the Server-LIF network on the Logical Router!
C3750#sh ip route 10.10.224.0
Routing entry for 10.10.224.0/24
Known via “ospf 1”, distance 110, metric 1, type extern 2, forward metric 2
Last update from 10.105.255.2 on Vlan103, 00:30:41 ago
Routing Descriptor Blocks:
* 10.105.255.2, from 10.105.255.10, 00:30:41 ago, via Vlan103
Route metric is 1, traffic share count is 1
On the Edge Gateway, we can check the routes we’re learning by running the command sh ip route. Notice we have routes that we’ve learned from 10.105.255.1 (Cisco Switch) and 10.105.255.10 (Logical Router).
nsxedge.etherbacon.net-0> sh ip route
Codes: O – OSPF derived, i – IS-IS derived, B – BGP derived,
C – connected, S – static, L1 – IS-IS level-1, L2 – IS-IS level-2,
IA – OSPF inter area, E1 – OSPF external type 1, E2 – OSPF external type 2,
N1 – OSPF NSSA external type 1, N2 – OSPF NSSA external type 2
Total number of routes: 14
S 0.0.0.0/0 [0/0] via 10.105.255.1
C 1.1.1.0/30 [0/0] via 1.1.1.1
O 10.10.200.0/24 [30/2] via 10.105.255.1
O 10.10.201.0/24 [30/2] via 10.105.255.1
O 10.10.202.0/24 [30/2] via 10.105.255.1
O 10.10.203.0/24 [30/2] via 10.105.255.1
O 10.10.204.0/29 [30/2] via 10.105.255.1
O 10.10.205.0/24 [30/2] via 10.105.255.1
O E2 10.10.224.0/24 [110/1] via 10.105.255.10
O 10.105.254.0/30 [30/2] via 10.105.255.1
C 10.105.255.0/30 [0/0] via 10.105.255.2
C 10.105.255.8/29 [0/0] via 10.105.255.9
O 10.255.250.0/24 [30/2] via 10.105.255.1
O 10.255.251.0/24 [30/2] via 10.105.255.1
And to wrap up Part 3 in the NSX series, we can test connectivity from a Cisco switch to a VM that we’ve migrated into our NSX environment. I’ve moved a VM to use the address 10.10.224.7 on the Server-LIF network. Doing a trace route to that VM shows the following:
C3750#traceroute 10.10.224.7
Type escape sequence to abort.
Tracing the route to 10.10.224.7
1 10.105.255.2 8 msec 0 msec 0 sec —> Our First Hop is our Edge Gateway
2 10.105.255.10 0 msec 9 msec 0 sec—> Our Second Hop is our Logical Router
3 10.10.224.7 0 msec * 0 msec
C3750#
Thanks for reading, I hope you found it informative, and might give you some ideas about deploying NSX in your own environment!