Fixing Time Errors on VMware vSphere and ESX Hosts

Tuesday, July 19, 2011
Time synchronization across a Windows domain is very important.  If a member server's clock varies more than 5 minutes from other domain servers, Kerberos tickets will fail.  This causes random authentication errors for users and/or applications which are sometimes difficult to troubleshoot.

Normally, time is synchronized in a Windows domain using the domain hierarchy.  The domain controller holding the PDC Emulator FSMO role is normally configured to get time from an authoritative NTP time source, and syncs time with all the other DCs in the domain.  The domain clients in each site sync time from the DCs in their local site, maintaining a relatively close synchronization of time across the domain.

Virtual machines are no different than physical computers and normally sync time using the same domain hierarchy.  Lately, however, I've seen VMs running on VMware vSphere boot up with random time differences from the domain.  I've seen this problem with three different clients lately, so I figured this might be a pervasive enough issue to blog about.

The trouble happens when the VMware vSphere, ESX or ESXi host does not have an accurate source of time, or time "drifts" due to an inaccurate system clock module.  vSphere and ESX hosts run a proprietary operating system and are not domain member servers, therefore they do not participate in domain hierarchy time synchronization. 

Most companies that use VMware hosts use vCenter to manage these hosts and their VMs.  Often, the servers that run vCenter are domain member computers and administrators think that since the vCenter syncs time with the domain, the hosts and VMs do, too.  Not true.  You need to configure the vSphere or ESX hosts to sync time from an accurate time source, otherwise the VM guests may start up with the wrong time - this can happen even if time synchronization between the virtual machine and the ESX server in VMware Tools is not enabled.


Here's how to configure your vSphere or ESX hosts to get time from an authoritative source.
  • Logon to vCenter and select your vSphere or ESX host.
  • Click the Configuration tab and then Time Configuration under the Software heading.  Notice that the time on the vSphere host does not match the domain time shown on the Windows client running vCenter .

  • Click Properties in the top left of the Configuration tab.  This opens the Time Configuration window.

  • Click the Options button and add a new NTP server that is the accurate source of time.  I recommend using the PDC emulator, since it should already be configured as an authoritative time source. 

  •  Select the checkbox to Restart NTP service to apply changes and click OK twice to close the Time Configuration window.  You will see that the vSphere/ESX host now has the correct time and is configured to use dc01.companyabc.com as its time server.

You may need to restart the VM guests running on that VMware host to have them sync time with the domain.  The Windows Time service will not correct the time on the VMs if it varies too much from domain time.  All domain computers sync time when they start up on the domain, regardless of how far out of sync they were.

I have not seen this type of behavior with Hyper-V, only vSphere, ESX and ESXi hosts.

6 comments:

  1. Perhaps for a DC which holds the PDC, you need to do the following: •Select the virtual machine in the VMware Infrastructure Client inventory. On the Summary tab, click Edit Settings, then click the Options tab and select General (under Advanced). Click Configuration Parameters, then click Add Row and add this information:


    tools.syncTime = "0"
    time.synchronize.continue = "0"
    time.synchronize.restore = "0"
    time.synchronize.resume.disk = "0"
    time.synchronize.shrink = "0"
    time.synchronize.tools.startup = "0"

    time.synchronize.resume.host = "0"

    This is because Time is resynchronized when you migrate the virtual machine using vMotion, take a snapshot, restore to a snapshot, shrink the virtual disk, or restart the tools service in the virtual machine (including rebooting the virtual machine).

    Source: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1189

    ReplyDelete
  2. Thanks for the info, but this affects all VMs, not just the PDC emulator. I ran into the exact same problem with another client yesterday. It was taking 30 MINUTES before I could log into them via RDP after a restart. I fixed the clock on the ESX host (it was off 10 minutes) and it fixed the problem.

    ReplyDelete
  3. Yes I have also ran into this issue and even it casued issues with RDP by name but IP was ok- all pointed to ESXi host forcing time change- even though tools configured to not sync.

    ReplyDelete
  4. Hi Jeff,

    this article was very useful to me.
    Thanks for bringing this up. :)

    a VDI-Technician

    ReplyDelete
  5. Hi!

    I have one question, not related with this topic (which I found though google), how can you have the VMs shown in the host inventory (lateral left panel) in a hierarchy view, downing from the Host in which they are, just as you probably have?

    I looked documentation and google but I cannot find answer, and my boss asked me to do this (yep, I am a internship: sometimes my time is for non really important things, but practical)

    Thank you very much and best regads

    ReplyDelete
  6. hi,

    this post was very helpful, the really crazy thing was, my bios was set to the right time the virtual server was set to the right time, the only thing that was set to the wrong time was the time configuration setting on my Host server, even tho the host bios was showing the correct time.

    Thanks again and keep up the good work!

    ReplyDelete

Thank you for your comment! It is my hope that you find the information here useful. Let others know if this post helped you out, or if you have a comment or further information.