The line that says “Running inside a VM; adjusting spinout timeout to 180 seconds” would suggest that KVM implements enough of our backdoor interface to make it look like we’re running under a VMware hypervisor. When we’re running in this environment, we use the backdoor to get the host TSC frequency. I suspect that KVM doesn’t implement the “GETMHZ” backdoor call, so we are confused about the TSC frequency. The 30ms delay turns into … 30 hours? 30 years?
So they had a source code change for QEMU 1.7.0, however it obviously doesn’t work in 2.x. It was rolled up stream, and then made into a switch to disable with a simple flag to add into the command line.
So with VIRL in hand, the next thing I wanted to do was play with some LACP, and VMWare ESX. Of course the best way to do this is under KVM as you can use UDP to bounce packets around between virtual machines, like the VIRL L2 switch. I went ahead and fired up 5.5 and got this nice purple screen of death.
So naturally I need to force the processor type. Also after reading a few sites, I needed to turn on a nested & ignore_msrs settings:
So it’s basically the same, just no mounted CD-ROM image. Now this is all fun, but what about networking? As I had mentioned before, I bought a VIRL license, which includes a l2 Catalyst image, so why not use that, instad of a ‘traditional’ Linux bridge? Sure! In this example I’m going to connect the 4 ethernet ports from the ESXi into the first 4 ports on the cisco switch, with the last port connecting to a Linux bridge, that I then route to, as I wanted all my lab crap on a seperate network. To start the switch I use this script:
Now as you can see the udp sockets are inverse of eachother, meaning that the ESX listens on 10000 and sends to 127.0.0.1 on port 20000, while the switch listesns on 20000, and sends packets to 10000 for the first ethernet interface pair.
By default VMware only assigns the first NIC into the first virtual switch, so after enabling CDP, we can see we have basic connecitivity:
AMD-kvm#sho run int gig0/1
Current configuration : 99 bytes
no negotiation auto
AMD-kvm#show cdp neigh
Capability Codes: R – Router, T – Trans Bridge, B – Source Route Bridge
S – Switch, H – Host, I – IGMP, r – Repeater, P – Phone,
D – Remote, C – CVTA, M – Two-port Mac Relay
Device ID Local Intrfce Holdtme Capability Platform Port ID
KVMESX-1 Gig 0/0 155 S VMware ES vmnic0
Total cdp entries displayed : 1
And of course the networking actually does work… I created a quick VM, and yep, It’s online!
AMD-kvm#show mac address-table
Mac Address Table
Vlan Mac Address Type Ports
—- ———– ——– —–
1 000c.2962.09e5 DYNAMIC Gi0/0
1 002e.3c92.2600 DYNAMIC Gi0/0
1 76b0.3336.34b3 DYNAMIC Gi2/3
Total Mac Addresses for this criterion: 3
And of course some obliguttory pictures:
With ip forwarding turned on my Ubuntu server, and an ip address assigned to my bridge interface, I can then access the NT 4.0 VM from my laptop directly.
Nex’t time to make the L2 more complicated, and add in some L3 insanity…
So for my email setup I use an OpenBSD firewall behind a hardware firewall (provided by the telecom), and from there I use OpenVPN to connect up to the VPS that in turn forwards email to my Exchange server.
It works great.
Except that the OpenBSD VM just crashed. And to top it off I had no other way of accessing inwards except for some test machine that luckily was still on, and I had SSH enabled, along with port redirection.
So a few seconds with putty and you can redirect a local port on your computer to connect to a port on the remote network. Dangerous as hell but, it certainly can save the day! (Yes you can even SSH to a machine, and then OpenVPN to it….)
Checking VMware KB 1012382 details a list of what ports are needed by which versions of their products to do what.
These are the two ports needed for basic checking in on the status of a standalone ESXi machine. So in this case I can point the VMware fat client to attach to 127.0.0.1, and add in redirects for TCP ports 443 & 902, which let me login, and start a remote console to see how the VMs are doing.
In later versions, you need to use a proper host name. To set this up edit your %windir%\system32\drivers\etc\hosts file, and make sure you have something like this:
127.0.0.1 localhost esxiloop
And then point the client to esxiloop, and it ought to connect.
Today I’m migrating this old SQL 7/Windows 2000 database server from VMWare ESX 3.5 to Proxmox VE. However this server started out on a VMWare ESX 2.5 server. And in the subsequent years had been decommissioned , and never updated but rather just copied onto the 3.5 cluster as we decommissioned the 2.5 cluster. At least I figured disk space was cheap enough we should keep the old VMs that “we will never need again” because.. Eventually someone will panic, and realize they need it again.
In the first step of doing so I needed to remove the old version of VMWare tools. But the catch is, this old version requires you to have the msi package handy to remove it. Well isn’t that a fun little catch. And you’ll find all kinds of ideas on what to do now that you don’t have your original “VMWare Tools.msi”. And more importantly you’ll now realize that you should have not only saved your old ISOs of ESX, but you should have also pulled out the tools ISOs and saved them as well.
Luckily I did save the software keys thought! Although I suspect that is also somewhere on their website, but they make it a chore to find the old stuff.. At any rate with 30 minutes of searching I finally came across the last version of 2.5, ESX Server 2.5.5 Build 57619.
Now it would seem that the VMWare tools are kept in an RPM file. Which is going to be a major pita for me to extract on Windows so I decided to take the more insane route, and install ESX on Qemu!
First I create a 5GB IDE disk to boot VMWare ESX server off, and a 10GB SCSI disk for the vmfs.
qemu-img.exe create -f raw esx25.disk 5G
Formatting ‘esx25.disk’, fmt=raw size=5368709120
And Qemu should easily boot the graphical installer.
So using my configuration, I dedicate one Ethernet card to ESX, another to the guests, and share the SCSI adapter between the console and the guests…
And when it comes to the partitioning, I simply extended the root partition to the rest of the drive, and setup vmfs2 on the SCSI disk. I’m not even thinking about clustering, I’m primarily after the extensions.
Installation takes about 20 minutes. It is just the way it is. The pegasus cimom for linux takes forever, along with the provider-esx package. I have no idea why, it’s probably thousands of little files or something crazy like that. But be patient, it’ll install.
And there we go, a successful installation!
Now VMWare will want to reboot, I just kill Qemu, and then launch it booting off the IDE harddisk (-boot c).
Now we get an ESX and Linux boot menu. I’m feeling brave, so let’s try to boot ESX!
And it panics. Bug 1406:2154 BugNr=34866 “measured cpu and bus speeds conflict”
Oh well. But we can boot into Linux, and scp out the extensions! Which do hide in /usr/lib/vmware/isoimages/windows.iso . So it’s not a total loss. I did notice on VMWare Fusion there was a setting for ESX, perhaps I can run ESX 2.5.5 on my Mac? Perhaps, but I’ll try that for later.
Now with the ISO finally in hand, I put it in my VM, and tell it to uninstall the extensions, I provide it with the VMWare Tools.msi and I get…
The file VMWare Tools.msi is not a valid installation package for the product VMware Tools.
Try to find the installation package ‘VMware Tools.msi’ in a folder from which you can install VMware Tools.
However the ISO did offer a chance to ‘upgrade’ my apparently older 2.5 extensions. So I did that, rebooted, then with a matching level ISO I was able to remove them. Wow was that convoluted! If anything I guess we’ve found out you want to hold onto these extension CD’s not matter what.. You never know if someone comes in with an old VM, or if you had a decommissioned VM that suddenly has to be brought back to life, it’s best to have these handy to get them back into shape. Just because your setup is all ‘complete’ it doesn’t stop people from throwing you curve balls.