Running VMWare ESXi on Raspberry PI

(This is a guest post by Antoni Sawicki aka Tenox)

Just for fun with virtualization I wanted to try out VMWare ESXi for ARM64, most specifically Raspberry PI. ESXi for ARM has been around for a couple of years now. Since PI4 packs 8GB of RAM and has a reasonably fast CPU it can be a worthwhile experience. Also more OSes for Raspberry PI are now available in UEFI boot mode.

Not going to go through exact installation steps as these are all around the web and youtube. Just to summary you will need to download an image from VMWare website as well as bunch of UEFI firmware files from github and combine it all together on to a SD card. When you boot it you will go through an install process which is straightforward. You can overwrite install media and use it as the target so no need for multiple SD cards. Once it boots you will see familiar ESXi boot screen:

ESXi booting on Raspberry PI 4

In order to get it going you will obviously need to add some storage. You can use NFS, iSCSI or locally attached USB drive. For the latest you need to disable USB arbitrator.

# /etc/init.d/usbarbitrator stop
# chkconfig usbarbitrator off

What can it run?

ESXi ARM only officially supports only UEFI boot based OSes. Fortunately this is a default option for Ubuntu PI, Free/Net/OpenBSD also work and so does Windows. But what about OSes that use U-Boot? Since ESXi-ARM Fling 1.1 you can boot oses in a “direct” mode with no UEFI! This is a huge step, but unfortunately as of today it doesn’t support UEFI-less VGA, only a serial port. Hopefully this can be fixed in future. I would love to have a RISC OS and/or Plan 9 VM. On the other hand Plan 9 supports EFI boot so an image could be made.

Windows guest install was also much easier than I expected. Thanks to UUP dump you basically roll your own bootable ISO. I think it’s actually easier to get it going on ESXi than natively on RPI hardware or QEMU.

Windows 10 Guest VM on ESXi Fling Raspberry PI

NIC driver obviously did not work by default, but there is a VMXNET3 ARM64 driver in the wild:

VMXNET3 for Windows 10 ARM64 on ESXi Fling on Raspberry PI

What is it good for?

Right now probably just for fun. But I can easily see datacenters filled in with ARM servers running ESXi. Future is bright and free of Intel! Personally I will keep it around for development purposes if I need to make builds for ARM on various OSes.

Interestingly enough you can even run VMWare ESXi ARM on QEMU with nested virtualization!

Also this is the official VMWare ESXi ARM Blog worth checking for future updates.

Web Rendering Proxy (WRP) 4.5.2

(This is a guest post by Antoni Sawicki aka Tenox)

Pleased to announce WRP version 4.5.2. This is just a bug fix release however it also contains two frequently requested features:

UI customization via HTML template file. This has been requested by many users and it makes total sense. To use it download wrp.html from github, place in the same directory as wrp binary and edit to your liking. WRP will load built-in version if file is not present.

This should enable easy development of more modern UI for never browsers. Potentially with JS and CSS. Please send PR if you make something!

Second most frequently asked feature – re-capture (retake?) of a screenshot without page reload. For example if the page did not capture correctly or if something is changing on the page.

I have also updated Docker Hub and gcr.io repos.

Sun IPX using WRP at VCF West

As usual please test and report bugs!

The next update will focus on issues with page size, viewport and rendering full length pages (h=0) which is currently very broken.

Windows Dev VM

(This is a guest post from Antoni Sawicki aka Tenox)

I have been living under a rock for several years now when it comes to Windows development. Recently wanting to do some maintenance on couple of my projects I needed to download Visual Studio and Windows SDK. Poking around the download page I have discovered that Microsoft now provides a fully pre-installed VMs with Visual Studio, SDK etc. for VMware, Hyper-V, VirtualBox and Parallels. That’s actually super cool and handy. Thank you Microsoft!

Looks like this has been available for 3 or 4 years now. Oh well.

Yedit – The missing edit.com replacement for modern Windows

(This is a guest post by Antoni Sawicki aka Tenox)

I previously lamented about lack of a small text editor for Windows text mode console. Something simple to just edit a text, configs, .ini, .reg or .cmd/bat files. Kind of like edit.com in MSDOS and Windows 3.x/9.x. Edit.com could even run under Windows NT up until XP inside NTVDM DOS emulator. I seen plenty of people use it this way. Sadly, instead of porting it to the modern Windows it was just whacked. While there are plenty of 3rd party editors to choose from none of them is perfect. They are either expensive, overly complicated, poorly ported from Unix, VMS or TOPS20, no longer maintained or require a steep learning curve for a casual Windows user.

My cries must have been heard in Redmond circles. Malcolm Smith aka malxau (author of Yori, a super awesome command line replacement for Windows) stepped in and made a brand new, text mode editor called Yedit. By default It comes as part of Yori installer however it’s easy to separate it to a stand alone exe.

Yedit on Windows NT

There really isn’t much to talk about Yori. It has no special features. It doesn’t try to be an IDE, word processor, hex editor or an operating system. It’s just a perfect replacement for edit.com. Thank you Malcolm! I really wish that Microsoft just put it in Windows 10 now.

Yedit on Windows 10

While checking out Yedit you should also give Yori a try. It’s really a fantastic CMD replacement shell.

Reverse-engineering QNX 1.2 to boot from HDD

This is a guest post from Mihai Gaitos, hawk.ro. A winning entry for Virtualization Challenge IV – Act II – QNX 1.2 HDD Boot ($2000 prize).


Since a lot of people (especially Zir Blazer) tried to use the available QNX 1.2 / QNX 2 tools to install a HDD boot loader and load the existing kernel, I decided to take a different approach and build a new loader. At first I was under the impression that maybe a BIOS disk driver was already present in the kernel. After realizing that there was no HDD driver included, I decided to try reverse-engineering the relevant parts of QNX.

Starting from the start (boot sector) helped me extract the kernel from the boot diskette and analyze it just enough to validate that it’s the right thing and the assumed entry point is correct.

In order to make things easier (and because it was a fun project per se) I wrote a somewhat simple QNX filesystem access tool that enabled me to extract files from the diskette and HDD images.

Going for mount

Afterwards, my main activity was centered on mount. As opposed to typical Linux/Unix mount, here it also loads the HDD driver. After finding out the executable file format (ftp.qnx.com/usr/free2/technotes/qnx_load) I wrote another small tool to extract the code and data segments of QNX executables. Analyzing the disassembly, I have determined what operations mount performs in order to install the driver and mount a QNX partition. The main steps are:

  • load the driver file contents into malloc-ed buffer (inside the data segment of mount)
  • send a TA_ALLOC_SEG message to task in order to allocate a separate segment and copy driver there
  • build a DEFINE_DRIVER message using data from driver file and the allocated segment address and send the message to fsys (part of kernel, but separate task)
  • send a SET_ATTR message to fsys that has the side-effect of initializing the driver
  • use the driver to read first HDD sector (partition table)
  • send another SET_ATTR message to adjust disk size and offset to values read from partition

Knowing this gave me an idea to what my loader would need to do beside simply loading the kernel from HDD. However, this still depended on having an already running kernel to send messages to.

Back to kernel

The kernel is split into 5 parts:

  • task (task and memory management)
  • fsys (disk and filesystem)
  • dev (terminal devices)
  • idle (CPU arbiter)
  • shared (int 72 handler, mostly libc and other shared functions)

Description in parenthesis are my assumptions.

The copy protection routine (tries to read the last sector from diskette and if the read succeeds resets the computer) provides a good entry-point into the fsys part of the kernel. I assumed it can be replaced with some code to emulate what mount does. However, trying to allocate a segment (via TA_ALLOC_SEG message) hangs. I think this is causing a deadlock, since fsys initialization is called from task before it finished its initialization. Fortunatelly, while digging into this I noticed the header structure of the kernel, thus enabling me to increase its size in order to fit the xt driver at the end of fsys (it would have been slightly easier to put it at the end of shared, but that didn’t occur to me at the time).

Failing to use syscalls (DEFINE_DRIVER and SET_ATTR) meant I had to determine what those messages actually did. I disassembled fsys separately and proceeded to manually follow the code path in order to determine the effect each of those messages should have in the context of mounting a disk. Eventually it emerged that almost all of the data structures can be prefilled in the kernel image, leaving only the call to driver initializaion function.

I modified the kernel to add the xt driver at the end of fsys (modifying the header by hand), replaced the copy-protection routine with code to call its initialization, and indeed the harddrive was available from the start, without the need to run mount. I was still booting from diskette at this time but I was past the most difficult hurdle.

Finishing touches

Loading the kernel proved somewhat simple (I still have some knowledge about 16-bit assembly and real-mode BIOS) but the kernel “insisted” in trying to run /cmds/sh from floppy. At first I solved this by an ugly hack, modifying the command line string in kernel image from “/cmds/sh” to “3:/xi/sh” and “/config/sys.init” to “3:/xi/sys.init” (3: being the HDD identifier, similar to C: from DOS). The xi was needed in order to keep the same string length, or at least not making it larger since there was some other importand data just past this.

This mostly solved the challenge (there were some other minor mistakes and fixes), except I disliked that hack and went on to analyzing that first start of /cmds/sh, disassembling fopen (in shared) and finally finding the memory location where of the search system variable (somewhat similar to PATH). Modifying that variable eliminated the need for starting the first shell with “3:”.

Room for improvement

At present some parameters are hardcoded and the kernel is just placed at the end of the HDD, outside of QNX parition and its position and size is written in the boot sector (somewhat similar to the original QNX diskette approach). The partition size itself is hardcoded (by hand) in the kernel data structures instead of being read from the partition table on boot. Still, for something that is unlikely to ever run outside an emulator, I deem it good enough (for now).

Thanks

  • to Zir Blazer for putting a lot of effort into his approach and documenting each step
  • to Mitchell Schoenbrun for providing insight into QNX system philosophy
  • to forty for beating the first challenge and identifying the copy-protection routine address
  • and of course, to Tenox, Neozeed and Dan Dodge for the challenge. And for providing me with a great prize for 3 weeks of hard-working fun!

To access files, tools, bootable image and ready to run in your browser PCjs with QNX 1.2 go to Mihai’s site hawk.ro post.

Virtualization Challenge IV – Act II – QNX 1.2 HDD Boot ($2000 prize)

(This is a guest post by Antoni Sawicki aka Tenox)

A couple of months ago we hosted VIrtualization Challenge for QNX v1.2. I expected that the hard part would be to circumvent the copy protection and the rest would be easy. It turned out to be quite the opposite! The copy protection was worked around in no time by several people independently. What turned out to be impossible is to install the OS on a hard disk.

QNX 1.2 does have several drivers for different hdd controllers including BIOS mode. It has fdisk, can create partitions, install MBR, format fs, mount hard disk volumes… but it cannot install boot code. Apparently this functionality has been added only in QNX 2.x. After a long debate we settled for a solution where you boot kernel from a floppy disk and use the rest of the os from a hard disk. This was implemented by Forty who won the challenge which was outlined in this post.

In a rather unexpected turn of events Dan Dodge, co-creator and CEO of QNX Software Systems himself reached out to us and offered to extend the contest to finish the process properly. Dan is offering $2000 prize for making QNX 1.2 boot from hard disk without use of the boot floppy disk. I have confirmed the details in an email exchange.

Rules: As always the winner will be the first person who provides a working image in the comments. Any emulator/hypervisor is allowed. You can use boot loader from QNX 2.x, or write your own or anything else you come up with. There are some tips in Dan’s comment. Ask away for more details. The QNX repository is here. Good luck! 🙂

Update: The challenge has been completed! The winner is Mihai Gaitos and this is the winning entry also a post on this blog. I will work with Mihai to get a more detailed blog post of what has been done and Dan to hand out the $2000 prize. Congratulations!!!

Update: You can try QNX 1.2 straight in your browser with PCjs!

UnixWare 1.0 on 86Box

(This is a guest post by Antoni Sawicki aka Tenox)

I certainly can’t claim to be the first as this has already been done by our friends at OS/2 Museum. However with low vanilla VGA resolution and no networking the results were unsatisfactory. Having so much success with 86Box I decided to try to do a little better.

I bought my UnixWare 1.0 media kit years ago on eBay. Unlike the tape set owned by OS/2 Museum mine had CDROM as install media. Unfortunately despite many many tries with different types of cdrom/bus/ide/scsi card I could never get the OS to see it. The cdrom/iso image is just a typical set of sysv packages. As such I wanted to see if it would be possible to convert it to a set of floppy disk images and install this way. Attached the iso image in UnixWare 7.1.4 VM and did a pkgtrans like so:

pkgtrans -s cdrom1 diskette1

From there I created a bunch of floppy disk images, which I later used for installation. Thanks to Plamen I was also able to get TCP/IP disks which I added to the install set.

Update: thanks to ArtiomWin I also got a BusLogic HBA driver disk, which allowed me to see the cdrom attached over SCSI. As such I decided to remaster the original iso image with added TCP/IP set, Update package and bash+gzip. The iso image is here.

Upon first boot after install from CDROM you get prompted to choose a NIC driver:

Unfortunately none of them really worked in 86Box for some reason. They get detected and you can see the MAC Address but not much after that. 3C503 and NE2K freeze the system, WD works bit better but you can’t really communicate with anything. Maybe it’s just my PCap configuration.

After installation I mounted the cdrom again and added TCP/IP set:

mount -r -F cdfs /dev/cd0 /mnt
pkgadd -d /mnt tcpnfs
pkgadd -d /mnt update

cp bash gzip /bin

One of main issues bugging me was lack of proper resolution. UnixWare 1.0 has a high resolution mode for Tseng ET4000 card which is supported in 86Box. You can change the resolution using /usr/X/adm/setvgamode as root. It worked perfectly, except for fonts, which required some surgery in /usr/X/defaults/Xwinfont (remove everything after 75dpi font path). This is how it looks like fixed up:

UnixWare comes with Merge DOS emulator. It can even run graphical applications in windowed mode for CGA and HGC. VGA is only possible in full screen mode.

All this cool stuff before Linux was even born!

DOS Menu is invoked by Scroll Lock. You can switch consoles between text and X11 by pressing CTRL+ALT+SYSRQ and ‘p’. I have also added bash and gzip binaries.

The ready to run 86Box image is here. Virtual Box OVA here. Install media here. Login with user/user, root/root.

Have fun with Virtualization !

QNX 1.2 Virtualized

(This is a guest post by Antoni Sawicki aka Tenox)

A few week ago we ran a Virtualization Challenge to virtualize QNX 1.2. The difficult part of this one was that the boot disk was copy protected. Thanks to Kryoflux and SuperCard Pro I was able to image the disks and convert them to usable images using HxC software tool HFE.

Technically the competition has been won by Crazyc who was the first to submit disk images with copy protection worked around. He however waived his monetary prize and did not do any further work on making whole system bootable from hard disk.

While the copy protection turned out to be quite easy to circumvent and several people did it independently, installation on a hard disk proved to be quite impossible. You can fdisk, create partitions, lay out file system, mount and copy files to hard disk. However there is no way to install a boot loader and the kernel. QNX 2.x and above provide a way of doing it but unfortunately not version 1.x. Many people including various QNX gurus looked at it and we all gave up at this point.

Probably the only reasonable way of using hard disk with QNX 1.x is to copy all files from all the floppies to the hdd. Then use the boot floppy disk for booting and the rest from hard disk. This is likely why the disk set came with a backup copy of boot disk. This is what Forty eventually did in effect winning the competition. Forty supplied a 86Box ready to run configuration with patched and modified boot floppy to mount and use the hard disk image. I have buffed it up a bit to a faster XT and EGA video for better resolution. This is how it looks like during boot:

You can safely ignore date/time prompt with enter. To login to the system just enter slash ‘/‘ as the user name:

You can find all the binaries in /cmds directory. The system does have some sort of networking facility but I have not figured it out yet. Probably a good candidate to explore in another post.

QNX has a super cool editor which is basically ed on steroids. Documentation for it can be found in 2.x manuals.

Also working C compiler:

Finally QNX has some sort of a DOS emulator or hypervisor called QDOS:

Unfortunately I don’t know how to exit that. There is a little bit information about QDOX in expl inform section about other QNX products:

Congratulations to Forty for winning the competition and gettin $100 via PayPal. Thanks to his time and work you can boot and play the system yourself. 86Box files are here.

You may also be interested in QNX 2.x, QNX Windows and QNX 4.x posts.

Finally QNX 1.2 also works under PCjs emulator and you can try it online here.

Have fun with virtualization!

UPDATE : QNX 1.2 challenge Act II – HDD Boot

UPDATEReverse-engineering QNX 1.2 to boot from HDD

Dell Unix on 86Box

(This is a guest post by Antoni Sawicki aka Tenox)

In a few recent projects such as QNX 1.2 (and demo disk), Interactive Unix (also older post ) and Caldera (and older post) I have tried the 86Box emulator. Unlike others, now I could utilize an emulated video and network cards of wide variety. As everything I did simply worked out of the box I instantly fell in love. Truly awesome 86Box is now my daily drive for running old PC operating systems. As such I have decided to revisit some of previously half assed posts with the new weapon.

I have virtualized Dell Unix back in 2012 using Bochs and QEMU. Even with community support we have struggled to get a decent video resolution and had to use SLIP for networking. Today let me present Dell Unix more properly, with 1024×768, 256 colors video and proper networking using emulated VGA and NIC.

I started with allsoft.img which is Dell Unix and all packages installed from the tape on Bochs. I have disabled a few services in /etc/rc2.d namely mouse daemon (mse), sendmail, uucp, lp, etc.

For X Window I have edited /usr/lib/X11/Xconfig, enabled serial mouse (Microsoft) and 1024×768 mode. I have used Tseng ET4000AX VGA which is detected by Xmach server. This allowed X / xinit to run correctly. However for startx to work you need to edit /usr/lib/X11/xinit/xserverrc as it seems to be using slightly different configuration. For graphical login you can probably add x:3:respawn:/usr/bin/X11/xdm -nodaemon to /etc/inittab. However I have noticed that when ran from init, xdm seem not to pick up the Dell customized config files. Perhaps rc startup script should be created instead.

As a final note on X, the system has virtual consoles, like other SVR4 you access them by pressing SYSRQ and F keys. F1 is a text mode console, F2 is Xserver. This is my Dell Unix hero shot:

Networking was even easier. Dell Unix supports WD8003 and 3C503 NICs. I wanted to first try the WD one. In /etc/conf/pack.d/wdn/space.c you can find the predefined hardware probes. I have picked one of supported modes and the card was detected on subsequent reboot. That’s it. No need for kernel rebuild or any configuration. I have not tried 3C503 but if you want the driver for it is named ie6. For TCP/IP configuration you set your IP address in /etc/hosts and gateway in /etc/inet/rc.inet file.

I was able to quickly compile Mosaic, which curiously had Makefile settings for Dell Unix, and take it for a spin on the web with help of WRP:

One could probably want to compile more recent version of Mosaic with PNG support or maybe some more recent browser all together.

The system comes with a bunch of open source software in /usr/dell however as there was no bash or even gzip I have compiled some essentials. They are available here and as a /usr/local tarball.

For the lazy, as usual you can get a complete os image for 86Box here. Make sure to attach pcap to your local network interface and set IP address / gateway / dns server accordingly.

If you port some cool software or find any interesting gems in Dell Unix please comment!

Have fun with virtualization!

Update: I have been looking at contents of various distribution media for Dell Unix that have surfaced here and there. On a DAT tape I bought on eBay a few years back I found this file:

Whoa! Of course I want to install all of it! This is how FrameMaker 3.0 looks on Dell Unix:

I have updated the disk image for 86Box to have this included. You can run demo mode of FrameMaker by executing /usr/frame/bin/demomaker. I also imagine that this can be installed on pretty much any x86 SVR4 and above, maybe even Linux. If anyone has a license code / serial number please let me know!

Fun with VP/ix under INTERACTIVE UNIX 3.0

(This is a guest post by Antoni Sawicki aka Tenox)

This is a continuation of the vintage DOS/Windows hypervisors and emulators for Unix series. So far I have covered things like Merge, MergePro and Wabi. This time I’m taking a closer look at VP/ix. This early DOS hypervisor was developed by Interactive Systems Incorporated (ISC). Initially released and included with their INTERACTIVE UNIX System V/386 operating system it was also available for SCO Xenix 386, Sun 386i, AT&T WGS as Simul-Task 386. The last two versions were significantly enhanced to allow DOS/Windows graphical apps run in windowed mode, which unfortunately is not the case with IX and Xenix, where graphical apps can only run on the console. VP/ix was released around the same time as Merge in 1987 and it was its main competitor. Both products are early hypervisors, they use Virtual 8086 mode and require 386+ to run on. This is in contrast to SoftPC which is a full x86 emulator that can run on different CPU/architecture hosts.

VP/ix comes with ISC INTERACTIVE UNIX that is covered in my previous article. The product was installed as part of the 50 floppy disk set. You run it with an icon in Looking Glass environment or invoke from terminal or console via “vpix” command.

VP/ix comes with it’s own custom version of MS-DOS 3.30. It allows a variety of cross unix/dos enhancements such as shared disks, automatic dos/unix file format conversion, listing unix attributes from dos as well as running unix commands from dos and vice versa. One of super cool features is that you can pipe output of DOS commands to Unix command, for example:

C:\> dir | wc -l

…will do a DOS dir and pipe it to Unix wc command. You can map Unix paths to DOS drives:

VP/ix has an interactive Menu invoked by SYSRQ + ‘m’ key:

You can load floppy disks, turn sound on/off, restart/quit or run unix shell.

As for running normal text mode apps it’s business as usual:

Multiple instances of DOS can be launched and files shared between them. Also if you are a different user on different terminal or connected remotely. Remote terminal also supports mapping dos line characters to ASCII.

The same however cannot be said about graphical DOS or Windows apps. Under INTERACTIVE UNIX and Xenix you need to run them from the text mode console:

One day I will probably want to look at VP/ix on Sun 386i or AT&T WGS as they solved this problem. Newer versions of Interactive Unix (4.x) and VP/IX also need to be investigated.

According to the documentation, you can run Windows 3.x in real mode using win /r however I did not have patience to install this.

INTERACTIVE UNIX 3.0 with VP/ix preinstalled can be downloaded here for 86Box or VBox OVA, however the later does not have networking and resolution is only 800×600. Login as root/root. When importing OVA in Vbox you may need to disable import as VDI. For 86Box version please read readme on how to circumvent licensing error.

Also VP/ix for SCO Xenix is available here.

Have fun with virtualization!