Vcenter Inventory Service Failed to start in Vcenter appliance or accessing vsphere webclient shows the error " Client is not authenticated to Vmware inventory service"

Issue : Vcenter appliance inventory service failed to start or accessing the vsphere webclient shows that the client is not able to authenticated to the VMware inventory services.

Where to check the logs for this :  

1.Connect the vcenter appliance with putty (ssh session) or through the console

2.Navigate to the path /var/log/vmware/vpx/inventoryservice  and check the ds log file by running the below command.
                       cd  /var/log/vmware/vpx/inventoryservice
                      cat ds.log | more

3. check to see if the database is showing as corrupt , if it is then follow the solution. This steps is for version esxi 5.5 , you can refer this vmware  kb article 2037952 for rest of the version

Solution :

1.Connect the vcenter appliance with putty (ssh session) or through the console

2.Stop the inventory service by running the command
                   Service vmware-inventoryservice stop

3.remove the data
                   rm -rf /storage/db/inventoryservice/data

4. Navigate to the path /usr/lib/vmware-vpx/inventoryservice/scripts/

                   cd /usr/lib/vmware-vpx/inventoryservice/scripts

5. run the ds-invoke.sh with the default password
                     ./ds-invoke.sh -Dvim.logdir=/var/log/vmware/vpx/inventoryservice com.vmware.vim.dataservices.CreateDb /storage/db/inventoryservice/data changeme default changeme
6.start the inventory service
                     Service vmware-inventoryservice start

7. register the vcenter appliance
                      vcenterhost=vcneterservername.yourdomaname.com

8.navigate to the directory /usr/lib/vmware-vpx/inventoryservice-registration
                      cd /usr/lib/vmware-vpx/inventoryservice-registration

9. run the vcregtool.sh
                   ./vcregtool.sh -Dvim.logdir=/var/log/vmware/vpx/inventoryservice-registration com.vmware.vim.dataservices.vcregtool.RegisterVC -action register -lookupserviceurl https://$vcenterhost:7444/lookupservice/sdk -isurl https://$vcenterhost:10443 -vcurl https://$vcenterhost/sdk/vimService -vccert /etc/vmware-vpx/ssl/rui.crt -vcprivkey /etc/vmware-vpx/ssl/rui.key -vcinstancecfg /etc/vmware-vpx/instance.cfg -vcendpointsdir /usr/lib/vmware-vpx/endpoints -vcextensionsdir /usr/lib/vmware-vpx/extensions

10. restart the vmware-vpxd service
                   service vmware-vpxd restart

ESXI host 5.5 takes long time to boot and stays at VMW_Vaaip_Cx loaded successfully

Issue :  ESXI 5.5 boot take long time and stays at VMW_VAAIP_CX  loaded successfully , this may take an hour or so to bring the ESXI 5.5 completely.

Cause : This will be happening on the environment which have Microsoft cluster vm and RDM disk is presented to that Microsoft cluster vm . Especially the RDM lun is not showing as Perennially reserved =true. You can check whether the lun is perennially reserved or not by running the below command

esxcli storage core device list 

Solution: Will explain how to fix this on ESXi 5.5 version , for rest of the version please check the vmware kb article number 1016106.

1.We need to know the LUN identifier ( naa.id ) for all the rdm disk and  this identifier varies for different storage vendor.
2. There are multiple ways to get the identifier , simple way is to click edit settings on the vm and click the rdm lun and on the right hand side click manage path , note down the lun number and identifier.
3. We can also get the identifier from Configuration---storage adapter and click one of the hba .
4.ssh into the esxi server
5.Run the command for each RDM lun

esxcli storage core device setconfig -d naa.id --perennially-reserved=true

Do this for all the esxi host which is part of the same cluster.

How to add the spare disk on netapp filer

Here is the steps to add the disk as spare disk on netapp

1. SSH into the controller
2. turn off the auto assign option by running the command options disk.auto_assign off
3. Put the new hard drive on one of the empty slots on disk shelf
4. run the command disk show -n  , this will show the newly added disk as not owned by any controller
5. run this command disk assign oa.00.8  to assign the controller where 0a is the controller identification ,00 is the disk shelf number and 8 is the disk number
6. Turn on the auto assign option by running the command options disk.auto_assign on
7. run disk show , you will be seeing the new disk as spare
8. for some reason you want to remove that hard drive you can unassign the disk by running the command disk assign 0a.00.8 -s unowned -f


 

windows File server move using robocopy

Often I used to get request to move the file share server to a different server , There are several option available to finish this job, the easiest one would be the robocopy one.

Robocopy will take care of ntfs permission on the folder but will not copy the share permission. If robocopy is used for the file share move then we need to export the share permission from exisitng windows server then import that to the new fileshare server.

Here is the high level steps

1. copy the below command in a text file and save it as filename.bat

@ECHO OFF
SETLOCAL

SET _source=D:\
SET _dest=E:\
SET _what=/COPYALL /SEC /MIR /Z
 :: /COPYALL :: COPY ALL file info
 :: /B :: copy files in Backup mode.
 :: /SEC :: copy files with SECurity
 :: /MIR :: MIRror a directory tree
SET _options=/R:0 /W:0 /LOG:Logfile.txt /NFL /NDL
 :: /R:n :: number of Retries
 :: /W:n :: Wait time between retries
 :: /LOG :: Output log file
 :: /NFL :: No file logging
 :: /NDL :: No dir logging

ROBOCOPY %_source% %_dest% %_what% %_options%

Note : please replace d:\ with the source path and also I:\ with the destination path. Also keep in mind that this will excatly mirror the source to destination , if destination has any extra files or folders that will be deleted.

2.download robocopy and save that exe and filename.bat on same location

3. run the filename.bat on a administrative command prompt.

4. The log file will be generated on the same location which includes start time ,end time , how much size is copied or ignored or failed . If any files copy is failed it will be reported on the log file

5.From the HKEY_LOCAL_MACHINES\SYSTEM\CurrentControlSet\Services\LanmanServer\Shares

6. export the above key from source server and import on the destination server

Note: this will work if you are moving entire file share server .if you are moving only portion of file share server then after import delete the unwanted share key.

 

sim.vmdk was not found error while powering on netapp simulator on esxi host

Today I tried to deploy the netapp simulator 8.2 on my lab . When I tried to power on the simulator it failed with the error  " Mydataontap-sim.vmdk was not found "

Solution :

ESXI server is not loading the vmkernel multiextent module  , we have to load this manually.

Here is the steps to  load that manually

1. login to the ESXi SSH session
2. run the command  vmkload_mod multiextent

How to change the web session limits in UCS manager

Here is the steps to limit the  maximum number of sessions per user and total maximum number of sessions for the UCS manager

Login to  UCS manager SSH session and type the below command

scope system

scope services

show web-session-limits

scope web-session-limits

set total 100 ( This limit to 100)

set per-user 10 (This limit to 10)

commit-buffer

How to change the maintenance policy for a service profile template

Problem : Default maintenance policy is set to "reboot immediate" for the service profile . If we do any changes on the template and if  that change requires server reboot , then we end up in rebooting all the server immediately because of this maintenance policy.

How to confirm the default maintenance policy for a template :

Navigate to UCS manager > Servers > Service profile templates > root > service profile template name 

On the Right pane  under properties  click Maintenance policy and see what is exactly configured .

How to change the Maintenance policy :

Navigate to UCS manager > Servers > Policies > root > Maintenance Policies > default

on the right hand side , select the maintenance policy as User Ack  and click save changes ( this option will ask for the confirmation to the user before rebooting the server)

Note : if this policy is attached to the service profile template and the service profile template is in updating mode then this will update on all service profile .


If you don't want to touch the default maintenance policy and you want to create  a new policies for the each service profile template , you will be able to create that under service profile template console itself

Navigate to UCS manager > Servers > Service profile templates > root > service profile template name 

click change maintenance policy on the right pane

click create maintenance policy

Provide the service policy name and description and select the reboot policy and click ok.

Drop down and select the new maintenance policy and click ok

Note: if you using updating template then all your service profile will get updated  for this immediately