NSX Installation - VTEPs not working

[jchilton]

I'm trying to set up and install NSX (latest version 4.1.2.1.0) in a vSphere 8 2 host dev environment. I deployed the NSX Manager successfully and created a cluster but am having difficulty configuring the Transport Hosts. Despite reading the vmware pubs and watchhing lots of You tube videos most of which talk about NSX-T i am really struggling.

The latest issue i have of VTEP not tunneling. I don't know if its a bug with the software or i'm doing something wrong. I've created 2 transport zones (overlay and VLAN) and added by hosts ot it. The VDS has been configured with 9000MTU and even modified the security aspects of the Port Groups associated with VTEP and external VLANS. I must have unistalled and installed NSX several times to no avail.  I now can't seem to delete Transport Zones that were created. The hosts recieve a valid I{P from the VTEP pool

Help someone pse.  Can someone send me a link to a good fault finding guide pse. This is so frustrating and I am starting to question using this product for a project. It has to be simpler to set up - surely. The product name change doesn't help the situation.

Oh and to top it off i can't seem to deploy a 2nd Edge Node VM - the first one seems to deploy ok but the 2nd one just times out and the error messages are a joke. When I get back in teh lab i can add photos etc

[chandrakm · Accepted solution]

Adding some screenshots should help to narrow down.

Couple of things to check:

- VTEP VLAN is configured correctly on physical switches/VDS and accessible to all ESX hosts?

- Can you do a normal vmk ping from TEP IP of a esx host to TEP network gateway?

- Can you do a normal vmk ping between TEP IPs of different ESX hosts?

- Can you do 8000 MTU vmk ping between TEP IPs of different ESX hosts?

- Do you have test VM's on NSX segment on two ESX hosts have you tried to ping between these test VM's?

- Have you used traceflow between these test VMs?

- In case if edges and ESX hosts are using same TEP VLAN then you should create NSX trunk VLAN segment and connect EDGEs to NSX trunk VLAN Segment in stead of VDS trunk port group. Otherwise tep tunnels wont come up between esx hosts to edge nodes.

[jchilton]

Chandrakm, thanks for reply.  Ref your points below I will try later today.  Of note, I am trying to use the same VLAN vDS Port group to connect my Edge VM  as I am the ESXi hosts - doh!  I'll create an NSX segment (of overlay type?).  My VTEP VLAN (103) is a port group so do i just create another nsx segment of overlay type and give it a VLAN tag 103 or make it a trunk port??

[chandrakm]

You need to create NSX VLAN Segment(not overlay segment). and connect Edge VM uplinks to NSX VLAN Segment. You can allow multiple vlans(TEP, BGP peering) throught this NSX VLAN segment.

[jchilton]

I'm now having trouble getting the vtep to work between hosts. I've noticed the vTEP port group on my VDS is not showing on the host network VDS.  I'm going to reboot the rig to see if the VTEP Port Group on the VDS comes back up. I seem to be going in circles.

I want to start again to get the host vTEP working before starting on Edge Node VM. I did have some success as when I did the changes you recommend above I got a tunnel up between the host the Edge VM is sat on with the Edge itself ie tunnel came up.

Do you know why I can't seem to deploy a 2nd Edge Node. It just sits and times out?? Is this a bug with this version??

[chandrakm]

Send screenshots

[jchilton]

Chandra, i will tomorrow.  I'm trying to follow the Easy Adoption Guide with no success.Are you aware of the guide here NSX Easy Adoption Design Guide | VMware.

Can you help me with a few questions:

1. Page 67 - what is a trunk segment? Is this an NSX segment? Or a VDS port group trunking port?

2. Page 70  Fig 42 - i've only seen the Edge node with 2 NVDS switches in documents, which make sense when mapping fast path interfaces; how do i map FP1 as a TEP and its trunk segment?

3. same page, how do you had mulitple TEPS to a host??

I've raised a SR but your network engineer is recommending i create separate clusters for mngt and workloads (then configure nsx on the workload cluster that houses the Edge Nodes) - not convinced. I want to implement this design for a use case with 2 powerful servers and a SAN.

Is there an issue with an Edge Node on the same cluster? Why cant my Tunnels come alive....argghh!!!!!

[chandrakm]

1. NSX have two types of segments Overlay Segments & VLAN backed segments. Trunk segment is VLAN segments with multiple VLAN's allowed under it(where you can mentioned multiple VLAN's under this VLAN segment like TEP VLAN, BGP Peering VLAN etc)

2. In the recent NSX releases there is no need to have multiple NVDS switches for Edge. One NVDS would be sufficient for most of the usecases. You should connect edge uplinks to these Trunk VLAN segments. TEPs are automatically created based on number of active uplinks on NVDS.

3. Number of TEPS Edge node or ESX Transport node is determined by and equal to number of uplinks selected NVDS or VDS has.

Separating Mgmt and Workload cluster in production environments is a recommended practice. But even with collapsed cluster architectures everything should work with right best practices are followed. Edge nodes can be on NSX prepared clusters connected to NSX Trunk VLAN segments(not VDS trunk portgroups) and TEP tunnels should come up without any issues.

[jchilton]

Chandrakm, thanks its becoming clearer.

I've included some photos below.  Do you think you could support a webex and i can share screens???

Few questions in meantime:

1. Do the Transport Nodes Tunnels between hosts come up automatically? Or does an Edge node need to be deployed for the host tunnels to come up?

2. I deployed an Edge node 01 and its hasn't acquired a vTEP address from the Pool. Does this signify that its hasn't worked?

3. I've just tried to deploy a 2nd Edge Node on host 2 and it just sits there in deployment mode - then will time out after many hours

[chandrakm]

1. VTEP Tunnels between hosts are automatic and doesn't need Edges. try connecting test VM's on two different ESX hosts connecting to a same segment and try to ping between each other. considering healthy MTU in the backend fabric and NSX configurations tunnels should automatically come up. if not try traceflow tool to see what's happening.

2. Check your configurations, profiles and pool, if its local pool and have enough free IP's, it shouldn't be a problem to receive IP vTEP IP form a pool. Make sure VLAN trunk is configured on physical and virtual environments properly as well. Configure Transport VLAN on profile.

3. Review for error messages on NSX as well as vCenter. hope you have got enough resources and configurations are correct.

[jchilton]

Chandrakm,

Good news the vTEP tunnels have come up for the Transport nodes now set I spun up a T1 GW as distributed only and have Test VMs on overlay segments.

I'm having issues deploying the Edge Node now using the NSX Manager. See pic attached. Error occurred during vmdk transfer. Failed to connect to the dest URI. Pse check NSX connectivity to <host>.  I can ping NSX to esxi hosts and port 443 is open. Any ideas.

I guess I could try deploying the Edge OVA manually using the vSphere web client. I'll try that in meantime.

Thanks for your help.

[DanielKrieger · Accepted solution]

Is your DNS name resolution working? Is your NTP working? How do you do the IP assignment for the edge nodes?

[chandrakm · Accepted solution]

Is this production or lab environment? what is the vCenter, ESX and NSX versions running? Make sure you are able to connect to using fqdn names of vCenter, ESX and NSX and DNS is configured with forward and reverse lookups properly. Also ensure all services in NSX are healthy and running as expected especially "install-upgrade" service on NSX.

[jchilton]

DNS wasn’t set on the NSX. Strange cos I deployed it with the settings during ova deployment. Used set command within the Photon OS. 

[jchilton]

Thank you. It was my DNS not set up

[jchilton]

Your last point is key. Pse amend your Quick Start guide to emphasis this fact as I’m sure I won’t be the only one who has wasted 3 days head scratching.