CentOS 5.5 losing time synchronisation on Hyper-V R2
Recently I deployed a CentOS 5.5 x64 guest on a Server 2008 R2 Hyper-V host for running a basic mail server. However I quickly noticed that the guest was losing time sync very rapidly, in the order of several positive minutes per hour. This was a surprise as I had already installed the Hyper-V Linux Integration components from Microsoft which I had assumed would just take care of everything for me. Not so, apparently.
I might also add that Dovecot (an IMAP/POP3 server) kept crashing due to the periodic NTP sync being so far out that it resulted in the guest’s clock actually going BACK in time! This appears to be an acknowledged bug in Dovecot but it is understandable why they don’t feel a pressing need to fix it. Though I understand the 2.0 release has. For the record, I was running the 188.8.131.52.el5 release.
Anyway, it turned out that the solution was very simple.
Modify the /boot/grub/grub.conf as follows:
Essentially you need to modify the lines that start with the word “kernel” and add two extra options onto the end:
This sets the clock source to use the Programmable Interrupt Timer (PIT). This is a fairly low level way for the kernel to track time and it works best with Hyper-V and Linux.
This is included more as belt-and-braces than anything. Because setting the PIT clock source (above) should already imply this setting really. But I include it for pure expressiveness🙂
This adjusts the PIT frequency resolution to be accurate to 10 milliseconds (which is perfectly sufficient for most applications). This isn’t strictly required but it will reduce some CPU load caused by the VM. If the VM will be running time sensitive calculations a lot (such as say a VoIP server or gaming server) then you probably shouldn’t include this option.
Once you’ve done this, save the file and reboot the box.
The time should now be synchronized precisely with the Hyper-V host!
PS: I’ve not tested whether this solution will work without the Hyper-V Linux Integration Components installed, but I believe that it will. As it operates independently of Hyper-V’s clock synchronisation mechanism and relies purely on the virtualised Programmable Interrupt Timer that is exposed by Hyper-V. And the PIT clock source will remain unchanged whether you have the Linux Integration Components installed or not.
PPS: The VMware knowledge base has a great article on this subject: http://kb.vmware.com/kb/1006427. Although take it with a slight pinch of salt (because the subject is Hyper-V) but it certainly gives several more options and ideas to try for different Linux distributions and 32-bit vs 64-bit etc environments.