Its been a while since I last wrote a blog post so I thought I should put up a quick one.
In NSX deployments and designs I have worked on, common questions I get when using a teaming method on NSX-v of Loadbalance-SRCID/MAC is:
1. How can I determine what physical interface VXLAN encapsulated traffic will egress/ingress via on an ESXi host?
2. What will be the SRC IP address and SRC MAC address of the VXLAN traffic?
Please note this does not include traffic traversing a DLR, I will cover that in a later post.
So to determine this you can follow the below process:
If you run the command esxtop from an ESXi hosts cli, then type n (for network) you will get a list of VM's DVPorts and VMKernel interfaces (VTEPs) and more importantly what DVUplink/vmnic on the host they are mapped to as below shows. In this example I'm going to be using VM04 as an example.
So in the below example from esxtop VM04 is using vmnic1 (DVUplink1) on my DVSwitch.
Output of esxtop for VM's:
67108876 39870:VM04.eth0 vmnic1 DvsPortset-1 0.00 0.00 1.94 0.00 0.00 0.00
67108877 49943:wah1.eth3 vmnic1 DvsPortset-1 0.00 0.00 0.00 0.00 0.00 0.00
67108878 49943:wah1.eth2 vmnic2 DvsPortset-1 0.00 0.00 0.00 0.00 0.00 0.00
67108879 49943:wah1.eth1 vmnic2 DvsPortset-1 0.00 0.00 0.00 0.00 0.00 0.00
67108880 49943:wah1.eth0 vmnic1 DvsPortset-1 0.00 0.00 0.00 0.00 0.00 0.00
67108881 50475:wah2.eth3 vmnic1 DvsPortset-1 0.00 0.00 0.00 0.00 0.00 0.00
67108882 50475:wah2.eth2 vmnic2 DvsPortset-1 0.00 0.00 0.00 0.00 0.00 0.00
67108883 50475:wah2.eth1 vmnic2 DvsPortset-1 0.00 0.00 0.00 0.00 0.00 0.00
67108884 50475:wah2.eth0 vmnic1 DvsPortset-1 0.00 0.00 0.00 0.00 0.00 0.00
67108885 51009:wah3.eth0 vmnic1 DvsPortset-1 0.00 0.00 0.00 0.00 0.00 0.00
If you then run the below esxcli command to list what VTEP a DVPort is using, we can see that VM04 on Switch port 67108876is using a VTEP with an ID of 0.
/var/log # esxcli network vswitch dvs vmware vxlan network port list --vds-name=DSwitch-Cluster2 --vxlan-id=5000
Switch Port ID VDS Port ID VMKNIC ID
-------------- ----------- ---------
67108870 vdrPort 1
67108876 142 0 <<< VM04
67108877 141 0
67108878 143 1
67108879 144 1
67108880 145 0
67108883 139 1
67108884 140 0
67108885 138 0
If we then use the below esxcli command to check what vmknic VTEP ID 0 maps to, this will provide the IP address and MAC address we will SRC traffic from when VM04 traffic is encapsulated in VXLAN from this host.
~ # esxcli network vswitch dvs vmware vxlan vmknic list --vds-name=DSwitch-Cluster2
Vmknic Name Switch Port ID VDS Port ID Endpoint ID VLAN ID IP Netmask IP Acquire Timeout Multicast Group Count Segment ID
----------- -------------- ----------- ----------- ------- ---------- ------------- ------------------ --------------------- ----------
vmk1 67108871 130 0 0 172.16.1.5 255.255.255.0 0 0 172.16.1.0
vmk2 67108872 137 1 0 172.16.1.6 255.255.255.0 0 0 172.16.1.0
~ # esxcfg-vmknic -l <trunkated output>
Interface Port Group/DVPort IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type
vmk1 130 IPv4 172.16.1.5 255.255.255.0 172.16.1.255 00:50:56:68:93:ed 1600 65535 true STATIC
We can then check again in esxtop what vmnic the VTEP vmk1 also mapped to and see if they matches what VM04 was using (vmnic1)
Output of esxtop for vmknic's:
~ #
67108871 vmk1 vmnic1 DvsPortset-1 0.19 0.00 0.77 0.00 0.00 0.00
67108872 vmk2 vmnic2 DvsPortset-1 0.00 0.00 0.97 0.00 0.00 0.00
So from the above VM04 is using VTEP ID 0 which is vmk1. Both vmk1 and VM04 are using vmnic1
So in conclusion when VM04 sends traffic to be encapsulated in VXLAN:
1. Outer SRC IP address is 172.16.1.5 (vmknic1)
2. Outer SRC MAC address is 00:50:56:68:93:ed (vmknic1)
3. Egress host physical nic - vmnic1 (DVUplink1)
Thanks for reading
Kev Barrass