Quantcast
Channel: VMware Communities : Blog List - All Communities
Viewing all articles
Browse latest Browse all 3135

Log Analysis: Scenario when PCI slot id is changed for the same physical nic leading to vmnic enumeration

$
0
0

Symptoms


- vmnic enumeration changed due to change in PCI Slot id.

 

- Duplicate mac address noticed for two vmnics in esx.conf file.

 

Cause

 

The vmnic label changes due to change in PCI Slot ID and many other factors.

 

Resolution


- Open the /etc/vmware/esx.conf file and check if there are duplicate mac address for multiple nics.

vmnic10 - vmnic27 -  00:25:b5:00:00:2c

vmnic11 - vmnic28 -  00:25:b5:00:00:5c

vmnic12 - vmnic29 -  00:25:b5:00:00:4c

vmnic13 - vmnic30 -  00:25:b5:00:00:7c

vmnic14 - vmnic31 -  00:25:b5:00:00:6c

vmnic16 - vmnic33 -  00:25:b5:00:00:8c

vmnic15 - vmnic32 -  00:25:b5:00:00:9c

vmnic17 - vmnic34 -  00:25:b5:00:00:ad

 

 

- So here vmnic10 label has changed to vmnic27 after the server migration activity since both the labels are pointing to the same physical mac address 00:25:b5:00:00:2c.

 

- Open the lspci.txt located under commands directory in scripts folder and check for the current mapping of vmnic to mac address.
         

000:005:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic0]

000:006:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic8]

000:007:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic16]

000:008:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic20]

000:009:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic21]

000:010:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic22]

000:011:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic23]

000:012:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic24]

000:013:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic25]

000:014:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic26]

000:015:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic27]

000:016:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic28]

000:017:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic29]

000:018:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic30]

000:019:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic31]

000:020:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic32]

000:021:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic33]

000:022:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic34]

000:023:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic35]

000:024:00.0 Network controller: Cisco Systems Inc Cisco VIC Ethernet NIC [vmnic36]


- Here vmnic27 is mapped to the concerned mac address after ESXi booted up. So the vmnic10 label actually does not exist.

 

- Goto var/run/log and grep for vmnic10 (old nic label) in old vmkernel logs before the migration happened.

      

cat /var/run/log/vmkernel.7 | grep -i vmnic10

 

2013-04-13T12:54:31.211Z cpu22:4771)PCI: 3938: 000:006:00.2 named 'vmnic10' (was '')

2013-04-13T12:54:35.995Z cpu22:4771)VMK_PCI: 684: Device 000:006:00.2 name: vmnic10

2013-04-13T12:54:35.995Z cpu22:4771)Uplink: 5262: Device vmnic10 not yet opened

2013-04-13T12:54:36.057Z cpu10:4771)Uplink: 6912: Opening device vmnic10

2013-04-13T12:54:36.057Z cpu23:4155)DMA: 524: DMA Engine 'vmnic10' created.

2013-04-13T12:54:36.057Z cpu23:4155)IDT: 991: 0xaa< vmnic10-rx-0> exclusive (entropy source), flags 0x10

2013-04-13T12:54:36.057Z cpu23:4155)IDT: 991: 0xb2< vmnic10-tx-0> exclusive (entropy source), flags 0x10

2013-04-13T12:54:36.057Z cpu23:4155)IDT: 991: 0xba< vmnic10-err> exclusive (entropy source), flags 0x10

2013-04-13T12:54:36.057Z cpu23:4155)IDT: 991: 0xc2< vmnic10-notify> exclusive (entropy source), flags 0x10

2013-04-13T12:54:36.913Z cpu22:4118)<6>enic 0000:06:00.2: vmnic10: Link UP


- The older entry of vmkernel logs indicate vmnic10 was slotted on PCI ID: 000:006:00.2.

 

- grep for vmnic27 (new label) in latest vmkernel logs after the migration completed.
         
                  cat var/log/vmkernel.0 | grep -i vmnic27 

 

 

2013-04-28T02:44:47.299Z cpu17:4771)PCI: 3938: 000:015:00.0 named 'vmnic27' (was '')

 

2013-04-28T02:44:55.282Z cpu17:4771)VMK_PCI: 684: Device 000:015:00.0 name: vmnic27

 

2013-04-28T02:44:55.282Z cpu17:4771)Uplink: 5262: Device vmnic27 not yet opened

 

2013-04-28T02:44:55.396Z cpu21:4771)Uplink: 6912: Opening device vmnic27

 

2013-04-28T02:44:55.396Z cpu19:4155)DMA: 524: DMA Engine 'vmnic27' created.

 

2013-04-28T02:44:55.396Z cpu19:4155)IDT: 991: 0xaa< vmnic27-rx-0> exclusive (entropy source), flags 0x10

 

2013-04-28T02:44:55.396Z cpu19:4155)IDT: 991: 0xb2< vmnic27-tx-0> exclusive (entropy source), flags 0x10

 

2013-04-28T02:44:55.396Z cpu19:4155)IDT: 991: 0xba< vmnic27-err> exclusive (entropy source), flags 0x10

 

2013-04-28T02:44:55.396Z cpu19:4155)IDT: 991: 0xc2< vmnic27-notify> exclusive (entropy source), flags 0x10

 

2013-04-28T02:44:56.446Z cpu17:4113)<6>enic 0000:0f:00.0: vmnic27: Link UP

  - The vmkernel logs indicate vmnic27 is slotted on PCI ID: 000:015:00.0.

- Based on the above log snippets, we can confirm that the PCI ID of physical nic with MAC address 00:25:b5:00:00:2c was changed from 000:006:00.2 to 000:015:00.0.

- That resulted in vmnic label to change from 10 to 27.


Viewing all articles
Browse latest Browse all 3135

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>