Solved: Hyper-V’s VMConnect tool sporadically losing connection to VM
Last night we performed a big switch over in our data centre. We moved everything onto a new managed switch and Sonicwall firewall, re-pointed and re-addressed lots and lots of servers, and well basically just done a bunch of stuff we should have done yonks ago! Everything seemed to go really well except for one thing: our Hyper-V hosts were now throwing really annoying and random disconnection errors when connected straight into a VM using its “Connect…” menu item, or otherwise known as
VMConnect.exe. The connection would work for at least a couple seconds, sometimes for as long as a minute or two. But then it would barf up and the following error message dialog would be displayed.
The full description of the error was as follows:
The connection to the virtual machine was lost. This can occur when a virtual machine stops unexpectedly, or when network problems occur. Try to connect again. If the problem persists, contact your system administator.
Would you like to try to reconnnect?
This was really annoying because we were connecting to local VMs that were present on the exact same VM host from which we were connecting. So presumably there wouldn’t be any packets hitting the network, and thus ruling out any of the new hardware and network changes we had just made.
After racking my brains on it for a bit (which included firing up Wireshark to perform a sanity check), I loaded up TCPView. This is a really great little tool from Mark Russinovich‘s stable called Windows Sysinternals. With this tool running I then retried the VMConnect, so that I could see what socket activities it was performing.
What this showed is that even when connecting to the local VM host using “
localhost” or “
127.0.0.1” as the address (i.e. IPv4) the VMConnect tool was seemingly transforming this into a IPV6 address and then forming a TCPV6 connection. This was interesting.
I immediately went to check whether IPV6 was actually enabled on the VM host’s network adapters. Low and behold, it was not. Turns out that when we flicked over the Gateway IP to point to the new firewall, we also subconciously turned off the IPV6 protocol on the list! A fairly innocuous thing to do, one would think, especially on an internal LAN!
So there you have it. If you come across this problem with Hyper-V, I would recommend you immediately check to ensure that you have not inadvertently disabled the IPV6 protocol on your virtual network adapter for Hyper-V.
The very moment we re-enabled IPV6, the problem with VMConnect constantly disconnecting every few seconds totally went away!
Not many problems get more obscure than this.