Reverse-engineering QNX 1.2 to boot from HDD

This is a guest post from Mihai Gaitos, A winning entry for Virtualization Challenge IV – Act II – QNX 1.2 HDD Boot ($2000 prize).

Since a lot of people (especially Zir Blazer) tried to use the available QNX 1.2 / QNX 2 tools to install a HDD boot loader and load the existing kernel, I decided to take a different approach and build a new loader. At first I was under the impression that maybe a BIOS disk driver was already present in the kernel. After realizing that there was no HDD driver included, I decided to try reverse-engineering the relevant parts of QNX.

Starting from the start (boot sector) helped me extract the kernel from the boot diskette and analyze it just enough to validate that it’s the right thing and the assumed entry point is correct.

In order to make things easier (and because it was a fun project per se) I wrote a somewhat simple QNX filesystem access tool that enabled me to extract files from the diskette and HDD images.

Going for mount

Afterwards, my main activity was centered on mount. As opposed to typical Linux/Unix mount, here it also loads the HDD driver. After finding out the executable file format ( I wrote another small tool to extract the code and data segments of QNX executables. Analyzing the disassembly, I have determined what operations mount performs in order to install the driver and mount a QNX partition. The main steps are:

  • load the driver file contents into malloc-ed buffer (inside the data segment of mount)
  • send a TA_ALLOC_SEG message to task in order to allocate a separate segment and copy driver there
  • build a DEFINE_DRIVER message using data from driver file and the allocated segment address and send the message to fsys (part of kernel, but separate task)
  • send a SET_ATTR message to fsys that has the side-effect of initializing the driver
  • use the driver to read first HDD sector (partition table)
  • send another SET_ATTR message to adjust disk size and offset to values read from partition

Knowing this gave me an idea to what my loader would need to do beside simply loading the kernel from HDD. However, this still depended on having an already running kernel to send messages to.

Back to kernel

The kernel is split into 5 parts:

  • task (task and memory management)
  • fsys (disk and filesystem)
  • dev (terminal devices)
  • idle (CPU arbiter)
  • shared (int 72 handler, mostly libc and other shared functions)

Description in parenthesis are my assumptions.

The copy protection routine (tries to read the last sector from diskette and if the read succeeds resets the computer) provides a good entry-point into the fsys part of the kernel. I assumed it can be replaced with some code to emulate what mount does. However, trying to allocate a segment (via TA_ALLOC_SEG message) hangs. I think this is causing a deadlock, since fsys initialization is called from task before it finished its initialization. Fortunatelly, while digging into this I noticed the header structure of the kernel, thus enabling me to increase its size in order to fit the xt driver at the end of fsys (it would have been slightly easier to put it at the end of shared, but that didn’t occur to me at the time).

Failing to use syscalls (DEFINE_DRIVER and SET_ATTR) meant I had to determine what those messages actually did. I disassembled fsys separately and proceeded to manually follow the code path in order to determine the effect each of those messages should have in the context of mounting a disk. Eventually it emerged that almost all of the data structures can be prefilled in the kernel image, leaving only the call to driver initializaion function.

I modified the kernel to add the xt driver at the end of fsys (modifying the header by hand), replaced the copy-protection routine with code to call its initialization, and indeed the harddrive was available from the start, without the need to run mount. I was still booting from diskette at this time but I was past the most difficult hurdle.

Finishing touches

Loading the kernel proved somewhat simple (I still have some knowledge about 16-bit assembly and real-mode BIOS) but the kernel “insisted” in trying to run /cmds/sh from floppy. At first I solved this by an ugly hack, modifying the command line string in kernel image from “/cmds/sh” to “3:/xi/sh” and “/config/sys.init” to “3:/xi/sys.init” (3: being the HDD identifier, similar to C: from DOS). The xi was needed in order to keep the same string length, or at least not making it larger since there was some other importand data just past this.

This mostly solved the challenge (there were some other minor mistakes and fixes), except I disliked that hack and went on to analyzing that first start of /cmds/sh, disassembling fopen (in shared) and finally finding the memory location where of the search system variable (somewhat similar to PATH). Modifying that variable eliminated the need for starting the first shell with “3:”.

Room for improvement

At present some parameters are hardcoded and the kernel is just placed at the end of the HDD, outside of QNX parition and its position and size is written in the boot sector (somewhat similar to the original QNX diskette approach). The partition size itself is hardcoded (by hand) in the kernel data structures instead of being read from the partition table on boot. Still, for something that is unlikely to ever run outside an emulator, I deem it good enough (for now).


  • to Zir Blazer for putting a lot of effort into his approach and documenting each step
  • to Mitchell Schoenbrun for providing insight into QNX system philosophy
  • to forty for beating the first challenge and identifying the copy-protection routine address
  • and of course, to Tenox, Neozeed and Dan Dodge for the challenge. And for providing me with a great prize for 3 weeks of hard-working fun!

To access files, tools, bootable image and ready to run in your browser PCjs with QNX 1.2 go to Mihai’s site post.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.