This article describes what the consequences are when configuring an NSX-T Tier 1 Gateway with an Edge Cluster. In my lab environment, I have attached virtual machine sa-web-02 to the Web-Segment, this segment is attached to Gateway-A. The sa-app-01 VM is attached to the App-Segment and this segment is attached to Gateway-B. Both gateways are attached to the BGP-T0-GW-01, a tier 0 gateway.
In the Network Topology overview, there’s also a service active on both gateways. This is the gateway firewall, but since no edges are configured on the Tier 1 gateways. The firewall isn’t active. There is no edge available to host the SRs of these gateway firewalls.
Let’s take a look at the Traceflow troubleshooting tool, available in the Manager user interface.
The packets flow east west through the DR of Gateway-A, the DR of the BGP-T0-GW-01 tier 0 gateway and the DR of Gateway-B. A DR is a distributed router hosted in the kernel of an ESXi host. This all happens on sa-esxi-05 because routing is performed on the host where the packet originates. The source and destination virtual machines are hosted on different transport nodes, that’s why the packets travel through the overlay network between TEPs 172.20.11.152 and 172.20.11.151 after all the routing has been completed.
An edge cluster is needed for hosting service routers for the Tier 1 gateways. Services like NAT, gateway firewall, and DHCP are not distributed. When you want to offer these services to a Tier 1 gateway, you need to configure an edge cluster. Let’s configure the gateways with an edge cluster and see what happens.
After performing a Re-Trace with the Traceflow utility, we can clearly see that the logical path is the same, but the physical path now contains two edges. This is because both gateways are automatically configured with a north-south firewall. The SR component of these firewalls is hosted on the edges, so the traffic must travel through these edges for the egress and ingress gateway firewall.
Both tier 1 gateways are assigned to a different edge instance in the cluster, so traffic jumps from sa-nsxedge-02 to sa-nsxedge-01. Every time traffic jumps from a transport node to another transport node, the traffic needs to be encapsulated and decapsulated to enter and leave the overlay network. The default firewall rule on a gateway is any-any-allow.
If you want to avoid hair pinning, don’t configure an edge cluster on a Tier 1 gateway. The gateway firewall is always enabled. When you want to offer NAT to segment attached to the Tier 1 gateway, and you must configure an edge cluster, the firewall can be bypassed.
The sequential steps in Traceflow show traffic entering sa-nsx-edge-01 for NAT only, the firewall is not in the data path.
