Number of Uplink VLAN required for multiple VRF

[ImAdlan05]

Hello,

I would like to build an NSX environment with an A/A T0 Edge VM Cluster link to an A/P Firewall for N/S traffic.

Im thinking of using BGP/ECMP. Edge Node with two TEP by Edge Node linked to two 10Gb/s dedicated NIC.

Because of our multi-tenant environment I would also take advantages of VRF-Lite implementation on T0.

Let say with 10 VRF, I am a bit confused of how many uplink VLAN I need to configure.

Can I use identical 2 VLAN for all 10 VRF and T0 Uplink ? (Total 1 uplink VLAN)

Can I use only one VLAN by VRF and T0 Uplink ? (Total 11 uplink VLAN)

Or should I should I configure 22 differents VLAN (2 x 10 VRF + 2 x 1 T0) for my design ? (Total 22 uplink VLAN)

My goal is only to reach 20 Gb/s N/S Traffic to my single active FW.

Can you please help me with on this point with detailled design justification ?

Best regards.

[Sreec]

You can configure one VLAN per VRF to peer with a Firewall if you are aiming for a multitenant environment

My goal is only to reach 20 Gb/s N/S Traffic to my single active FW 

This is dependent on the uplink connections and vSphere design, and you may need to adjust a number of performance factors to get the desired result.

https://blogs.vmware.com/networkvirtualization/2023/12/optimizing-nsx-performance-based-on-workload.... 

[ImAdlan05]

Hello @Sreec and thank you for your answer.

Your article has definitly change my mind for design of Edge Node cluster with 4 pnic.

Can you please also give me more detail about differences choosing a 1 VLAN / VRF or 2 VLAN / VRF design ?

Also on a 2 VLAN / VRF design should each edge node tag only one separate VLAN or 2 VLAN must be reach on both edge node ?

Best regards.

Adlan.

[Sreec]

Using several VLANs and peering with firewall devices is appropriate if your firewall design is multi-context, however, if my tenants find a single session acceptable, I will always fine-tune and leak at the firewall.

[DanielKrieger]

if you peer directly over 2 vlans with a firewall, you have to consider that traffic may be blocked due to anti spoofing.

@Sreec 

Why don't you use both pNics for your TEP traffic in your design or is the graphic just misleading? What do you think about MultiTep? So 2 two TEP IP addresses?

In some designs I prefer 4 pNics - two for TEP and two for BGP traffic. This is of course not a design where I don't have many Edge VMs on an ESX server.

[ImAdlan05]

Hello,

I want for sure using VRF for multi-tenant design, and several VLAN for peering with FW.

But here are 5 scenario where im confused for design choice. Edge Cluster is A/A.

Of course I will repeat each of these scenario for each VRF with different VLAN but I would like to understand design difference between these choices.

Best regards.

[Sreec]

100% right my friend. That is why I explicitly mentioned the multi-context scenario as the stateful nature of the firewall will drop such flows. Single/Dual TEP with single BGP peering is optimal for this design.

[ImAdlan05]

" Single/Dual TEP with single BGP peering is optimal for this design."

Ok with this, but from my previous schema which VLAN design it represent  ? 

Scenario 4 ? Scenario 2 ?

Best regards.

[DanielKrieger]

I would prefer scenario 2, but I have already implemented scenario 5. In this case, anti-spoofing must be deactivated on the firewall for the interfacees. Scenario 5 is useful if you need maximum N/S performance and the firewall supports ECMP and disabling anti spoofing is possible.

[ImAdlan05]

Thank you for your answer I am a bit confused because you said earlier :

"if you peer directly over 2 vlans with a firewall, you have to consider that traffic may be blocked due to anti spoofing."

So why anti spoofing is only with Scenario 5 (4 VLAN) and not Scenario 2 (2 VLAN) ?

Why don't you choose Scenario 4 ? (1 VLAN) over Scenario 2 ?

These uplink scenario are really confusing me a lot.. I really appreciate you still help me to understand.

I just want 20 Gb/s N/S bandwith with one 10 Gb/s NIC traffic on each edge node dedicated to this. (2 Edge Node)

ECMP should give me 10+10 and there must a be specific best practice design for this scenario isnt it ? 

Best regards.

Best regards.

[DanielKrieger · Accepted solution]

I understood you to mean that you use 1 uplink VLan for each VRF in Design 2, so anti-spoofing would not be an issue. Maybe I misinterpreted your graphics, if that is the case, then I would clearly go for option 4. Sorry for the confusion.

Design 5 will probably give you the best performance, but it depends on your firewall and whether you want to disable anti-spoofing or not. This may be a security issue. Otherwise you would have to work with AS-PATH prepend and local preference to avoid asyncronous routing. This means that effectively only one VLAN is used and only provides fast failover capacity; you would have 4 routes, but only 2 would be preferred.

BGP load balancing is always source dependent, if NSX uses ECMP, then the firewall must also use ECMP, otherwise only your outgoing traffic will do reasonable load balancing. Not every firewall actually uses ECMP.

Most of the time I don't peer directly with the firewall, but with the ToR switches and also use VRFs there if I need to. My firewall is then usually connected to my ToR via LACP and only uses one VLAN per VRF. But it all depends on your overall environment, your firewall and other decisions.

I have also built a direct peering with NSX and Checkpoint and then used 2 uplink VLANs for this. Anti-spoofing was deactivated on the downlink interfaces of the checkpoint. In addition, the downlink interfaces were still in an LACP bond. So I had 4x25 Gb/s at the checkpoint distributed over 2 VLANs. You have to explicitly switch on ECMP at the checkpoint.

[ImAdlan05]

Hello @DanielKrieger,

Thank you for this detailled answer.

I now got a better of what is possible and where I want to go.

Special thank for the LACP FW tips to use only one VLAN ! This one should perfectly fit in my design as i want to use less streched VLAN as possible.

Best regards,

Adlan.