Friday, June 20, 2014

Vcenter Inventory Service Failed to start in Vcenter appliance or accessing vsphere webclient shows the error " Client is not authenticated to Vmware inventory service"

Issue : Vcenter appliance inventory service failed to start or accessing the vsphere webclient shows that the client is not able to authenticated to the VMware inventory services.

Where to check the logs for this :  

1.Connect the vcenter appliance with putty (ssh session) or through the console

2.Navigate to the path /var/log/vmware/vpx/inventoryservice  and check the ds log file by running the below command.
                       cd  /var/log/vmware/vpx/inventoryservice
                      cat ds.log | more

3. check to see if the database is showing as corrupt , if it is then follow the solution. This steps is for version esxi 5.5 , you can refer this vmware  kb article 2037952 for rest of the version

Solution :

1.Connect the vcenter appliance with putty (ssh session) or through the console

2.Stop the inventory service by running the command
                   Service vmware-inventoryservice stop

3.remove the data
                   rm -rf /storage/db/inventoryservice/data

4. Navigate to the path /usr/lib/vmware-vpx/inventoryservice/scripts/
                   cd /usr/lib/vmware-vpx/inventoryservice/scripts

5. run the ds-invoke.sh with the default password
                     ./ds-invoke.sh -Dvim.logdir=/var/log/vmware/vpx/inventoryservice com.vmware.vim.dataservices.CreateDb /storage/db/inventoryservice/data changeme default changeme
6.start the inventory service
                     Service vmware-inventoryservice start

7. register the vcenter appliance
                      vcenterhost=vcneterservername.yourdomaname.com

8.navigate to the directory /usr/lib/vmware-vpx/inventoryservice-registration
                      cd /usr/lib/vmware-vpx/inventoryservice-registration

9. run the vcregtool.sh
                   ./vcregtool.sh -Dvim.logdir=/var/log/vmware/vpx/inventoryservice-registration com.vmware.vim.dataservices.vcregtool.RegisterVC -action register -lookupserviceurl https://$vcenterhost:7444/lookupservice/sdk -isurl https://$vcenterhost:10443 -vcurl https://$vcenterhost/sdk/vimService -vccert /etc/vmware-vpx/ssl/rui.crt -vcprivkey /etc/vmware-vpx/ssl/rui.key -vcinstancecfg /etc/vmware-vpx/instance.cfg -vcendpointsdir /usr/lib/vmware-vpx/endpoints -vcextensionsdir /usr/lib/vmware-vpx/extensions

10. restart the vmware-vpxd service
                   service vmware-vpxd restart



ESXI host 5.5 takes long time to boot and stays at VMW_Vaaip_Cx loaded successfully

Issue :  ESXI 5.5 boot take long time and stays at VMW_VAAIP_CX  loaded successfully , this may take an hour or so to bring the ESXI 5.5 completely.

Cause : This will be happening on the environment which have Microsoft cluster vm and RDM disk is presented to that Microsoft cluster vm . Especially the RDM lun is not showing as Perennially reserved =true. You can check whether the lun is perennially reserved or not by running the below command

esxcli storage core device list 

Solution: Will explain how to fix this on ESXi 5.5 version , for rest of the version please check the vmware kb article number 1016106.

1.We need to know the LUN identifier ( naa.id ) for all the rdm disk and  this identifier varies for different storage vendor.
2. There are multiple ways to get the identifier , simple way is to click edit settings on the vm and click the rdm lun and on the right hand side click manage path , note down the lun number and identifier.
3. We can also get the identifier from Configuration---storage adapter and click one of the hba .
4.ssh into the esxi server
5.Run the command for each RDM lun

esxcli storage core device setconfig -d naa.id --perennially-reserved=true

Do this for all the esxi host which is part of the same cluster.