Thunking for fun & a lack of profit

So, with a renewed interest in OS/2 betas, I’d been getting stuff into the direction of doing some full screen video. I’d copied and pasted stuff before and gotten QuakeWorld running, and I was looking forward to this challenge. The whole thing hinges on the VIO calls in OS/2 like VioScrLock, VioGetPhysBuf, VioScrUnLock etc etc. I found a nifty sample program Q59837 which shows how to map into the MDA card’s text RAM and clear it.

It’s a 16bit program, but first I got it to run on EMX with just a few minor changes, like removing far pointers. Great. But I wanted to build it with my cl386 experiments and that went off the edge. First there are some very slick macros, and Microsoft C just can’t deal with them. Fine I’ll use GCC. Then I had to get emximpl working so I could build an import library for VIO calls. I exported the assembly from GCC, and mangled it enough to where I could link it with the old Microsoft linker, and things were looking good! I could clear the video buffer on OS/2 2.00 GA.

Now why was it working? What is a THUNK? Well it turns out in the early OS/2 2.0 development, they were going to cut loose all the funky text mode video, keyboard & mouse support and go all in on the graphical Presentation Manager.

Presentation Manager from OS/2 6.605

Instead, they were going to leave that old stuff in the past, and 16bit only for keeping some backwards compatibility. And the only way a 32bit program can use those old 16bit API’s for video/keyboard/mouse (etc) is to call from 32bit mode into 16bit mode, then copy that data out of 16bit mode into 32bit mode. This round trip is called thunking, and well this sets up where it all goes wrong.

Then I tried one of the earlier PM looking betas 6.605, and quickly it crashed!

SYS2070:

Well this was weird. Obviously, I wanted to display help

Explanation:

This ended up being a long winded way of saying that there is missing calls from DOSCALL1.DLL. Looking through all the EMX thunking code, I came to the low level assembly, that actually implemented the thunking.

EXTRN   DosFlatToSel:PROC
EXTRN   DosSelToFlat:PROC

After looking at the doscalls import library, sure enough they just don’t exist. I did the most unspeakable thing and looked at the online help for guidance:

No VIO

So it turns out that in the early beta phase, there was no support for any of the 16bit IO from 32bit mode. There was no thunking at all. You were actually expected to use Presentation Manager.

YUCK

For anyone crazy enough to care, I uploaded this onto github Q59837-mono

It did work on the GA however so I guess I’m still on track there.

Announcing EmuWoW Beta 1 (AXP Preview1)

Freecell & Winver

This is a guest post by CaptainWillStarblazer

Hello, everyone. This is a continuation of my previous blogpost on EmuWoW (formerly win32emu) found here, but to summarize, I’m the 18-year-old developer behind a project that allows running applications compiled for the MIPS/Alpha version of Windows NT on standard x86 PCs through emulation, but without requiring a full system emulator. Since that last post, the project has made some substantial strides.

Since then, I’ve adapted the MIPS emulator from MAME, which is both more accurate and faster than my own (writing my own was a fun exercise but to the end of running applications, borrowing an emulator was a better decision). This alone enabled WinMine to function, which can be seen below.

Winmine for MIPS!

Around the same time, another VirtuallyFun member named x86matthew entered the scene. His prior credits include a similar project, for Win16. He was inspired by the initial win32emu blog post, created his own similar project called WoWMIPS, which quickly started running a few simple apps, such as WinMine, Solitaire, and Notepad. Be sure to check out WoWMIPS – MIPS Emulator for Windows, Part 1: Introduction, and parts 2,3,4,5,6!

This had actually been what I was hoping for from the beginning; I always saw win32emu as a proof of concept for a smarter person (either myself in the future or someone else altogether) to come back later and do it better, and this is a massive undertaking for a single individual. What makes x86matthew’s WoWMIPS so incredible is its clean design. For one, there’s no thunk DLLs required. When an application tries to import a DLL, the host x86 DLL is first loaded, and then any attempts to get the address of an exported procedure from it are then redirected to an auto-generated stub of MIPS code which invokes the emulator to call the function. In other words, the thunk DLLs are generated at load time. Additionally, there’s no window procedure thunks required either. Instead, MIPS code pages are marked as non-executable, causing an access violation when Windows attempts to call them. Using an exception handler implemented via Windows XP’s Vectored Exception Handling feature, one can detect this and then invoke the emulator to execute the callback. Storing the CPU state in thread-local-storage allowed proper support for multithreading. I duplicated these design choices into EmuWoW, and the results largely speak for themselves. Unlike WoWMIPS (at time of writing), EmuWoW is still capable of loading MIPS DLLs (and will prefer to do so if possible), however.

Various MIPS Windows NT applications running

To aid the process of fixing faults in the emulator, I hacked up a quick, minimal debugger into EmuWoW. First of all, there’s a “crash screen” which will indicate the type of error and dump registers and the current instruction if there’s a fault, and a running disassembly can be printed as you go, but most crucially, there’s a limited degree of interactive debugging functionality.

The built in debugger to EmuWow

The main capabilities contained herein are dumping registers and memory to the screen, disassembling regions of memory, listing loaded modules, getting import entry points, setting breakpoints, and single-stepping, along with printing functions. It’s no gdb (especially for the lack of PDB symbol support), but it’s something.

MIPS is all well and good, however, but we can already emulate Windows NT for MIPS. MIPS is a fairly clean, simple architecture, often used for teaching, and short of weirdness like delay slots, is dead simple to emulate. And it fits the bill for what I’m trying to do here – a RISC architecture Microsoft abandoned, fixed-length 32-bit instructions, the works. However, adding support for the DEC Alpha AXP will finally get this project to where it was intended to be from the beginning, and I hope for it to eventually be able to run the Visual C++ compiler toolchain under EmuWoW. Being able to compile (and even test!) Alpha apps on readily available modern PCs will be a game-changer, and I intentionally wrote EmuWoW to be CPU-agnostic, so adding rudimentary support for the Alpha wasn’t difficult. Like with MIPS, I lifted the emulator from MAME. This admittedly did pose some challenges, due to MAME’s use of C++ features such as

  • Classes (which I had to substitute for passing pointers to structs)
  • Templates (which I had to substitute for macros)
  • Function-style casts (which I had to change into C-style casts)

There was a lot of weirdness in how Alpha function calls, but it was my misunderstanding of the calling convention, and when to consider values to be 32-bit vs 64-bit posed a modest issue, but I got it to the point where some very simple DEC Alpha programs started to run, though many of them have various issues.

Various Dec Alpha Windows NT applications running

This is the first time this has ever been possible; emulation of AXP NT programs on standard PCs has just become possible now. I’m hoping to see EmuWoW continue to evolve, and I welcome contributions, whether that be improving support for the CPUs I already have, adding support for another (such as PowerPC) or anything else. To take a look at the code, go to github.com/bhty/emuwow and to download EmuWoW and try it out for yourself, click here.

New version of the MS-DOS Player

And it’s a big update on takeda-toshiya.my.coocan.jp!

From cracyc and roytam’s fork, I have incorporated a correction.
These include file access using FCB and fixing exceptions around the FPU of the MAME version of the i386 core.
In addition, the DAA/DAS/AAA/AAS/AAM/AAD instructions of the MAME version of the i386 core have been modified based on the DOSBox implementation.
With the Pentium 4 version, the testi386.exe is the same as the real thing.

The I386 core of NP21/W has been updated to equivalent to ver0.86 rev92 beta2.
Also, fixed the build time warning so that it does not appear.

Improved checking when accessing environment variables, referencing incorrect environment tables.
Recent builds have resolved an issue that prevented testi386.exe from working.
Improved the efficiency of memory access handling.
Basic memory, extended memory, and reserved areas (such as VRAM) can be accessed in that order with a small number of conditional branches.
The processing speed may be slightly increased.

MS-DOS Player for Win32-x64 Mystery WIP Page (coocan.jp)

Takeda has been very busy indeed!

I don’t want to complain or anything, I’m very thankful for the tool. It’s just so amazing.

but on my Windows 10 install I have so many issues relating to the font/screen changes, that I just made an incredibly lame fork, and commented out those changes, msdos-player_. I stumbled onto the issue by accident by redirecting stdout/stderr, and compiling stuff ran fine, but as soon as it started to mess with the console it’d just crash.

No console changes, no crashes.

OK so you can run some basic stuff like compilers, but what about ORACLE?!

Oracle 5!

I did have to subst a drive, as I didn’t feel like dealing with paths and stuff, I had extracted it from oracle-51c-qemu, and modified the autoexec & config.ora and yeah, using the 386 or better emulation it just worked! Sadly there is no network part of the install, although there is a SDK so I guess there ought to be a way to proxy queries.

OK, but how about something even more complicated?! NETWARE!

Netware 3.12 on MS-DOS Player

Obviously there is no ISA MFM/IDE disks in MS-DOS Player, but the server loaded!

Needless to say this update is just GREAT!

I’d say try the one hosted on Takeda’s site! It’ll almost certainly work fine for you. Otherwise I guess try mine. Or not.

Getting Graylog onto the internet

A while back I had made a small post about getting Graylog running on Windows. It was fun, as it’s just JAVA so you know it should be portable, and other than some weird disk access thing it does seem to run fine.

Of course, the next step is to create a dashboard to replace what I used on wp-statistics, as it was crashing taking up 100% of my CPU, and exceeding PHP’s 12GB of RAM per process limit. You know things are messed up when I’m replacing you with not one, but 2 Java apps! (Graylog & Opensearch).

magical dashboard

It’s by no means perfect, but the guide How to se -up graylog geoip configuration, is all around great to have. The rest of it is me learning how to do aggregate searches, and simple lists, to see latest hits, 404’s and count the pages and build a graph.

Again, this is all good, now for the real question, how to get this onto the Internet?!

The firs thing to do is enable cors.. It’s for being on the internet!

http_enable_cors = true

Next enable the external URI name

http_external_uri = https://dash.board.com/

And now the changes I had to make in my haproxy config

frontend http-in
        acl host_graylog hdr(host) -i dash.board.com
        http-request set-uri %[path,regsub(/api/api,/api)] if host_graylog
        use_backend graylog if host_graylog

backend graylog
        option forwardfor
        http-request add-header X-Forwarded-Host %[req.hdr(host)]

        acl bad_ct path_end .js
        http-response set-header Content-Type application/javascript if bad_ct

        http-request set-header X-Graylog-Server-URL http://192.168.33.5:9000
        server graylog 192.168.23.33:9000 maxconn 20 check

I kind of wish I saved the logs while going crazy but YES for some reason it’ll try to reference itself as /api/api. I don’t know why, so I had to do some uri regex to fix that. Neat!

Next for some reason Graylog responds that all .js (javascript) files are actually text. Chrome doesn’t allow that to work, so yes you need to set the content type header to “application/javascript” for Chrome to be happy.

I had wasted over an hour with this and couldn’t get it working. So, I walked away for a few hours, and it suddenly was working. I think Cloudflare was doing some caching against it.

This is probably too terse to be really useful, and I lost all the pages I was reading about setting stuff in haproxy as I was doing that incognito. Oops. I picked this config out of fragments from five other people’s stuff. There is other considerations to host it on a subdirectory of a public site, but I just wanted to K.I.S.S.

Targeting OS/2 with Visual Studio 2003

Sydney’s idea of what Visual C++ for OS/2 should look like

No, it’s not a typo.

This is a long-winded post, but the short version is that I found a working combination to get the C compiler from Visual Studio 2003 targeting OS/2.

Once I’d learned how C compilers are a collection of programs working in concert, I’d always wanted to force Microsoft C to work in that fashion, however it is born to be a compiler that integrates everything but linking. There has been a “/Fa” or output assembly option, but I’ve never gotten it to do anything useful. I’m not that much into assembly but it seemed insurmountable.

But for some reason this time things were different.

This time I used:

Microsoft (R) Macro Assembler Version 6.11

After the great divorce and the rise of Windows NT, Microsoft had shifted from the OMF format to COFF. However somewhere buried in their old tools it still supports it, namely MASM. For example, if I try to run LINK386 (the OS/2 Linker) against output from Visual C++ 2003 I get his

abandon.obj :  fatal error L1101: invalid object module
 Object file offset: 1  Record type: 4c

However if I output to assembly and then have MASM assemble that, and try the linker, I’m bombarded with errors like this:

warp.obj(warp.asm) :  error L2025: __real@4059000000000000 : symbol defined more than once
warp.obj(warp.asm) :  error L2029: '__ftol2' : unresolved external

If I was smart I’d have given up, there is pages and pages of this stuff. But I’m not smart, so instead I decided to something different, and use SED, the stream editor, and try to patch out the errors.

The ftol2 call is for newer CPU’s and any OS/2 library won’t have it. But instead of binary editing symbols we can replace the ftol2 with ftol with this simple line:

 sed -e 's/_ftol2/_ftol/g'

For some reason Visual C++ likes to make all it’s reals “public” meaning there can only be one, but yet there is so many. Why not comment them all out?

sed -e 's/PUBLIC\t__real@/;PUBLIC\t__real@/g'

And there are various other annoying things, but again they can be all patched out. Just as the older Windows 1991 Pre-release compilers also have weird syntax that MASM doesn’t understand.

astro.asm(59): error A2138: invalid data initializer

which goes into how Microsoft C used to initialize floating point constants:

CONST   SEGMENT  DWORD USE32 PUBLIC 'CONST'
$T20000         DQ      0a0ce51293ee845c8r    ; 1.157407407407407E-05
$T20001         DQ      0bffffd3441429ec5r    ; 2440587.499999667
CONST      ENDS

while I found that the C compiler in Xenix 386 initialises them like this:

$T20000         DQ      0a0ce51293ee845c8h    ; 1.157407407407407E-05
$T20001         DQ      0bffffd3441429ec5h    ; 2440587.499999667

This one was a little hard for me as I’m not a sed expert, but I did figure out how to mark the section, and then to replace it

sed -e "s/DQ\t[0-9a-f]r/&XMMMMMMX/g" $.a1 |  sed -e "s/rXMMMMMMX/H/g"

And so on. At the moment my ‘mangle’ script is now this:

.c.obj:
        $(CC) $(INC) $(OPT) $(DEBUG) /c /Fa$*.a $*.c
        wsl sed -e 's/FLAT://g' $*.a > $*.a1
        wsl sed -e "s/DQ\t[0-9a-f]*r/&XMMMMMMX/g" $*.a1 \
        | wsl sed -e "s/rXMMMMMMX/H/g" \
        | wsl sed -e 's/call \t\[/call DWORD PTR\[/g' \
        | wsl sed -e 's/PUBLIC\t__real@/;PUBLIC\t__real@/g' \
        | wsl sed -e 's/_ftol2/_ftol/g' > $*.asm
        ml /c $*.asm
        del $*.a $*.a1 $*.asm

This allows me to plug it into a Makefile, so I only have to edit it in one place.

Not surprisingly, this allows the LINK from Visual C++ 1.0 to link the MASM generated object files and get a native Win32 executable. Even from the oldest compiler I have from the Microsoft OS/2 2.00 Beta 2 SDK from 1989!

But now that we have the C compilers being able to output to something we can edit and force into a Win32, there is a few more things and suddenly:

        C:\cl386-research\bin\13.10.6030\cl386 /u /w /G3 /O  /c /Faphoon.a phoon.c
C:\cl386-research\bin\13.10.6030\CL386.EXE: warning: invoking C:\cl386-research\bin\13.10.6030\CL.EXE
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.6030 for 80x86
Copyright (C) Microsoft Corporation 1984-2002. All rights reserved.

phoon.c
        wsl sed -e 's/FLAT://g' phoon.a > phoon.a1
        wsl sed -e "s/DQ\t[0-9a-f]*r/&XMMMMMMX/g" phoon.a1  | wsl sed -e "s/rXMMMMMMX/H/g"  | wsl sed -e 's/call \t\[/call DWORD PTR\[/g'  | wsl sed -e 's/PUBLIC\t__real@/;PUBLIC\t__real@/g'  | wsl sed -e 's/_ftol2/_ftol/g' > phoon.asm
        ml /c phoon.asm
Microsoft (R) Macro Assembler Version 6.11
Copyright (C) Microsoft Corp 1981-1993.  All rights reserved.

 Assembling: phoon.asm
        del phoon.a phoon.a1 phoon.asm
        msdos286 run286 C:\cl386-research\bin\ddk12\LINK386.EXE @phoon.lnk

Operating System/2 Linear Executable Linker
Version 2.01.012 Nov 02 1993
Copyright (C) IBM Corporation 1988-1993.
Copyright (C) Microsoft Corp. 1988-1993.
 All rights reserved.

Object Modules [.obj]: astro.obj date_p.obj phoon.obj
Run File [astro.exe]: phoon2 /NOE /NOI /NOD:OLDNAMES
List File [nul.map]: nul.map
Libraries [.lib]: ..\..\lib2\libc.lib +
Libraries [.lib]: ..\..\lib2\os2386.lib
Definitions File [nul.def]: nul.def;
LINK386 :  warning L4071: application type not specified; assuming WINDOWCOMPAT

I know it’s a bit of a word salad, but the key thing here is that using Visual C++ 2003’s compiler (version 13.10.6030), and outputting to assembly that we can edit, we can then use MASM to build objects that surprisingly LINK386 version 2.01.012 will link with. I suspect this has to do with device drivers, and probably the majority of the OS/2 operating system.

Anways, we’ve done the incredible, using the same object files, we made both a Win32 application, and an OS/2 application!

phoon-13.10.6030.exe: PE32 executable (console) Intel 80386, for MS Windows
phoon2.exe: MS-DOS executable, LX for OS/2 (console) i80386
Phoon compiled by Visual C++ 2003 on OS/2 2.00

Incidentally Happy CNY!

Obviously, this is VERY cool stuff.

I know the next question is do we have to rely on a 16bit linker? How about Watcom?

C:\cl386-research\proj\trek>wlink @trek.wlk
WATCOM Linker Version 10.0
Copyright by WATCOM International Corp. 1985, 1994. All rights reserved.
WATCOM is a trademark of WATCOM International Corp.
loading object files
searching libraries
Warning(1008): cannot open LIBC.lib : No such file or directory
Warning(1008): cannot open OLDNAMES.lib : No such file or directory
creating an OS/2 32-bit executable

Ignore the warnings and YES we can Link from something much newer & 32bit! In this example I linked the old TREK game, also built with Visual C++ 2003. The response file looks lke:

SYS os2v2
NAME trek2w
FILE abandon.obj,attack.obj,autover.obj,capture.obj,cgetc.obj,checkcond.obj
FILE check_out.obj,compkl.obj,computer.obj,damage.obj,damaged.obj,dcrept.obj
FILE destruct.obj,dock.obj,dumpgame.obj,dumpme.obj,dumpssradio.obj,events.obj
FILE externs.obj,getcodi.obj,getpar.obj,help.obj,impulse.obj,initquad.obj
FILE kill.obj,klmove.obj,lose.obj,lrscan.obj,main.OBJ,move.obj
FILE nova.obj,nullsleep.obj,out.obj,phaser.obj,play.obj,ram.obj
FILE ranf.obj,rest.obj,schedule.obj,score.obj,setup.obj,setwarp.obj
FILE shield.obj,snova.obj,srscan.obj,systemname.obj,torped.obj,utility.obj
FILE visual.obj,warp.obj,win.obj
LIBR ..\..\lib2\LIBC.LIB
LIBR ..\..\lib2\OS2386.LIB

It’s probably needing additional stack space, maybe some other stuff, or resources, maybe how to flag it’s windowing compatible.

TREK built by Visual C++ 2003, and Linked using Watcom C/C++ 10.0

How do I get started, if I dare?! First download and unpack cl386-research-v2. Ideally on the root of your C: drive, because why not?

run the ‘env’ command to set your environment up. Its pretty complicated but in the proj directly there is currently:

*NOTE that I do use SED scripts, I have it set to use Linux in the WSL package.
I tried some Win32 sed but it didn’t work. So you need WSL or a working sed!

you can then go into each directory and run

make os2

and it’ll compile populate a floppy and launch the emulator

Its all good fun.

Read the Makefiles to configure a compiler, how to run it, and if you need to mangle the assembly. The 32bit new stuff needs to be mangled, the older stuff almost always works with just compile.

# Version 6.00.054      1989
# https://archive.org/details/os-2-cd-rom_202401
PLATFORM = ddksrc

In this case it’ll select the platform from the ‘ddksdk’ release. The next is if the compiler is OS/2 based or native win32. Basically 73g / windows 95 & below are native Win32.

In the above example we comment out the dos extended cross

# dos exteded cross
CC =  $(EMU) $(DOSX) $(CL386ROOT)\$(PLATFORM)\cl386
# native CC
# CC =  $(CL386ROOT)\$(PLATFORM)\cl386

Next is the mangle strategy. In this case it’s an ancient OS/2 (like) compile so
try un commenting the ‘just compile’ line

# must include ONLY ONE strategey..
# for OS/2 it must have been assembled my MASM 6.11

include ..\-justcompile.mak
#include ..\-mangleassembly.mak
#include ..\-plainassembly.mak

save the makefile, and run

nmake os2

You can just close the emulator as after each run it’ll unpack a hard disk image, so nothing will be lost. or saved. It’s just for testing. You may need to periodically clean the floppy drive, as that is the only way to transfer stuff in and out of the VM.

What versions of CL386 have I found? Well, it’s quite a few, although I know I’m missing quite a few.

== c386 ============================
Microsoft C 5 386 Compiler

Microsoft C 5.2 286/386 Compiler -- Driver

@(#)C Compiler Apr 19 1990 11:48:30

Copyright (c) Microsoft Corp
1984-1989. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 1.00.075
Quick C Compiler Version 2.00.000
1.00.075

== ddk12 ============================

C 6.00 (Alpha) Aug 24 1990 19:12:31

Copyright (c) Microsoft Corp
1984-1989. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.054
Quick C Compiler Version 2.00.000
6.00.054

== ddk20 ============================

C 6.00 (Alpha) Aug 16 1990 23:04:06

Copyright (c) Microsoft Corp
1984-1989. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.054
Quick C Compiler Version 2.00.000
6.00.054

== ddksrc ============================

C 6.00 (Alpha) Aug 24 1990 19:21:49

Copyright (c) Microsoft Corp
1984-1989. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.054
Quick C Compiler Version 2.00.000
6.00.054

== nt-sep ============================

@(#)C Compiler 6.00 Feb 06 1991 17:15:19
@(#)C Compiler 6.00 May 13 1991 23:54:12
@(#)C Compiler 6.00 Jun 03 1991 15:16:22


Copyright (c) Microsoft Corp
1984-1991. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.077
Quick C Compiler Version 2.00.000
6.00.077

== nt-oct ============================

@(#)C Compiler 6.00 Jun 03 1991 15:16:22
@(#)C Compiler 6.00 Jun 13 1991 22:07:23
@(#)C Compiler 6.00 Oct 10 1991 00:42:24

Copyright (c) Microsoft Corp
1984-1991. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.080
Quick C Compiler Version 2.00.000
6.00.080

== nt-dec ============================
@(#)C Compiler 6.00 Jun 03 1991 15:16:22
@(#)C Compiler 6.00 Jun 13 1991 22:07:23
@(#)C Compiler 6.00 Oct 10 1991 00:42:24

Copyright (c) Microsoft Corp
1984-1991. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.081
Quick C Compiler Version 2.00.000
6.00.081

== 73g  ============================
1984-1993. All rights reserved.
Copyright (c) Microsoft Corp
8.00.3200
32-bit C/C++ Optimizing Compiler Version
Microsoft (R)


== msvc32s ============================

Microsoft 8.00.0000 - Copyright (C) 1986-1993 Microsoft Corp.
Microsoft 8.00.0000 - Copyright (C) 1986-1993 Microsoft Corp.
@(#) Microsoft C/C++ 32 bits x86 Compiler Version 8.00.XXXX

8.00.000

== 13.10.6030 ============================

Microsoft (R) C/C++ Compiler Version 13.10.6030
From my install of Visual Studio 2003 Enterprise

As you can see many of these earlier OS/2 compilers report the same versions but are in fact different builds on the inside. I suspect Microsoft had to support one version, and an Alpha version of version 6 is as good as it got. I would have imagined there were internal 32bit versions of 6 or 7, but I haven’t seen them.

Compiling and running TREK

Hopefully this gives some idea of how I tried to made a probably too modular build system to try all kinds of different compilers. I might have to see if it’s possible to run the tools from the 1992 versions of Windows NT in this setup, perhaps they are interesting as well.

One thing in my porting GCC to OS/2 experience is that the usability of the C compilers from 1991 were dramatically better than what Microsoft had given IBM at the time of the divorce. No doubt the upcoming NTOS/2 project was placing a bigger demand on the tools team.

If anyone has any access to other ‘cl386’ compilers, or early OS/2 2.00 stuff, please let me know! I’d love to do build/tests and see if my idea of distributing objects ‘just works’!

Porting GCC to 32bit OS/2

I know what you are going to think, that it was already done, and it was called EMX. Or was it GCC/2? Well sure but what if you are not running the GA (General availability) version of OS/2. For example, years ago I had managed to get Citrix Multiuser 2.0, and it’s not at the GA level. All that is available is some ancient beta version of Microsoft C 5.2 from 1989?!

A little while back I had worked on getting GCC to build and run on the FPU enabled versions of Windows NT from 1991. I had mentioned that it turns out thanks to the Xenix assembler, that GCC had been basically available the entire time Windows NT had been available, but lamented that since the OS/2 compiler is 16/32bit, the 5.2 compiler couldn’t handle compiling GCC without blowing it’s heap. 16bit issues in a 64bit world.

However after doing some research on all the early cl386 compilers I could get my hands on, including the Windows NT Pre-release ones, I’d noticed that if I built CC1.EXE (the actual compiler) first for Win32, then rebuilt those object files with the December NT Pre-release compiler, that some versions of LINK386 from the OS/2 2.1 DDK would actually link with them. And sure enough it worked!

First life of GCC on OS/2

I have to admit I was pretty amazed, I had managed to ‘cross compile’ GCC using quite the tool chain.

First the compiler from the December NT Pre-release CD-ROM is shipped as a 16bit OS/2 compiler, but I’m using Windows 11. First I use the MS-DOS player with a quick fix from crazyc to allow Phar Lap 286|Dos Extender to run, which provides a basic enough OS/2 emulation to allow the compiler to run under ‘dos’. The linker on the DDK suffers the same fate as far as it also being 16bit. However the combination of MS-DOS player & Phar Lap gets stuff working! The only weird catch is that the 386 emulator causes strange floating point related crashes, while the 286 or 486 emulators work fine.

Now targeting OS/2 or running on OS/2 isn’t all that new, but building it from a Microsoft C compiler is. And now of course you’ll ask yourself, who cares? why is it interesting?

Well, the vast majority of the GCC ports to OS/2 don’t support the OMF object file binary standard, instead they used the much outdated a.out format, and rely on tools to convert the objects if needed. Additionally, they have DLL dependencies, and other startup issues with things needing to be setup. And of course they rely on a binary standard that is ‘GA’. *HOWEVER* by using a Microsoft compiler, I have OMF object files that the OS/2 built in system linker LINK386 can understand. So in plain English I can just relink the compiler and it’ll run on a new ‘version’ of unsupported OS/2.

I made a diskette image with my objects & a linker script and in a few moments I had it running!

GCC running on OS/2 2.00 6.123

The substantial thing here is that the binary format for OS/2 changed twice, and each release introduced changes that broke binary compatibility, in an effort to force people onto the new tools. So there is no way that the old ‘LE’ format would ever work. And you can see it’s running! In addition I could take the same object files, and copy them to my Citrix server, and likewise it was just a matter of linking, and it too now has GCC!

Converting a.out to OMF via emxomf

One annoying thing is that the LINK386 that ships with OS/2 2.00 GA doesn’t like the output of the Xenix assembler, so I built the a.out traditional assembler, and the emxomf tool to convert the a.out to OMF, and that worked well.

I still have much to mess with, including the pre-processor & main ‘gcc’ program. I have not built anything beyond a trivial program, so there is indeed much more work to be done before I can even try anything challenging. Some programs like emxomf have portions in the debug support that require the ‘long long’ type, which obviously Microsoft compilers from 1989-1991 don’t have, so I’ll have to re-build them with GCC.

Ive been putting my ports onto github (cl386-research) as it handles the rapid changes well enough. It’s a bit of a complex setup and it involves using a build system that I’ve put over on archive.org here: cl386-research-v2.

Not that I can imagine anyone wanting to try but I’ve uploaded some disks with the objects. Copy them to a hard drive, and run the ‘build.cmd’ command and it’ll link to a native freestanding executable.

I’ll explain it more with a post later, along with going over all the versions of cl386 I’ve acquired, over the years in more of a part 2: Targeting OS/2 with Visual Studio 2003!

Modernising the National Geographic CD-ROM collection

The fancy collector’s box set

I wanted to get some research into some early space flight, and look into that magical transition when everything went from Cowboys & Indians to moonmen & the race to space. Granted Toy Story covers cultural touchstone pretty well, reading period pieces is fun too. I had a few CD-ROM’s containing the 1980’s National Geographic CD-ROM’s when they were sold up in decade sets, but what always escaped me was the fancy collectors box with the whole thing.

$169!!

I always figured this was going to be one of those weird collectors’ items that probably was under produced, over sold, and lost to the winds of time. Looking on eBay for a 1950’s and 1960’s set .. and thinking about the 1970’s as well, and it was going to get close to £40. Ouch. So for the heck of it, I look for the fancy box set.

wow

I was surprised for as much as I was going to end up paying for one or two sets, I could get the entire thing. In the legendary fancy wooden box. I got mine, shipped for under £20. Much wow!

Okay what is the catch?

Wait a minute, these National Geographic CDs are obsolete!Christopher Elliott

I always was running mine on MacOS using Cockatrice III with the monitor resolution set to the absolutely absurd resolution of 1152×870 provided by the 1991 21″ monitor. I couldn’t imagine why these CD’s wouldn’t work. And of course the first step was to rip the CD-ROM’s. There was 31 National Geographic, and an additional clip art disc in my box. I fired up my XP machine to have it’s hard disk give up and die on me. Luckily since I had removed the mechanical disk from the iMac G5, I had a spare SATA disk handy. I didn’t feel like fighting the XP installer, and I’m impatient, so I made a netboot.xyz bootable flash drive, and installed Debian 10 over the internet. Very nice. Now I could get down to ripping CD’s. Luckily the drive from the DVD-RAM drive disaster reads at 48x, so it took me under 4 hours to rip them all. In case you are wondering:

32 File(s) 17,894,987,776 bytes

The flash drive I used is 32Gb. I got it in a 3 pack from Tesco. I think I paid £10 for it. That means UTZOO + NatGEO all fit one of the drives. I wonder if they’ll ever offer high resolution scans on a USB drive in a fancy wooden box?

For the hell of it, I used 7zip to decompress all the ISO’s and that’s when I noticed that although the files were spread over discs there was a clean decade break between volumes from the 1900’s, and that they had a logic to them.

On the NGS_1956_1959 CD-ROM there is a hierarchy something like this:

E:\IMAGES\256I

The 2 is for the 20th century, and 56 is the year. I is the month; in this case I is September. Since we live in the future, and rendering jpeg’s is quicker than real-time, 256IC01A.JPG can be shown to be the cover. The next pattern is for the adverts, 256IA02A.JPG – 256IA30A.JPG are the into adverts. Now, we get to the interior content 256I0287.JPG – 256I0426.JPG. Next is the closing/trailer advertisements 256IZ01Z.JPG – 256IZ13Z.JPG. And Finally the back of the issue, 256IB14Z.JPG is the rear of the magazine.

So we now have some understanding of the format. Putting this into order could be done with something simple like this:

find ./ -name '2[0-9][0-9]IC[0-9]*.JPG' > interior
find ./ -name '2[0-9][0-9]IA[0-9]*A.JPG' >> interior
find ./ -name '2[0-9][0-9]I[0-9]*.JPG' >> interior
find ./ -name '2[0-9][0-9]IZ[0-9]*Z.JPG' >> interior
find ./ -name '2[0-9][0-9]IB[0-9]*Z.JPG' >> interior

Doing so makes a nice list file of what images should go in which order. I could probably use ffmpeg and painstakingly check the images for ‘pullouts/double wide’ ones, and have it stitch the rest together as a two pager ( ffmpeg -i left.jpg -i right.jpg -filter_complex hstack combined.jpg ), but that still sounds like a lot of work. Also the images are very low quality, It’s a shame they didn’t use black & white on the text, and scan the images separately, but that’d require something like PDF, and no doubt a LOT of time. Although Kodak did sponsor the set, the developer, Mindscape didn’t go with fancy PDF technology of the late 1990s, instead it’s just the blurry jpeg scans we have today.

NG July 1981

That’s when I found out about Tesseract. Running it against this one paragraph reveals:

Voyager is watching two small moons
that-seem to be playing tag as they race
around Saturn in almost the same orbit. The
trailing moon is traveling faster than the
leader, and should catch up with the leader
in January 1982 (pages 20-21). The two pre-
sumably have been playing this game for
billions of years, Through what sleight of
physics do they avoid colliding?

National Geographic, July 1981

I have to admit, that’s pretty good! And how amazing that I have a LOT of files to scan.

188553 File(s) 11,061,111,690 bytes

That is a LOT of files. Okay that’s nice, but can Tesseract read the list that I generated per issue? YES. The only thing that I’d love to see Tesseract do is create PDF’s with the scanned text embedded. OH WAIT, IT ALREADY DOES THAT!

How on earth did I not know this?

I put together a few scripts, and I was able to separate out all the images into years & months, then I created the needed list files with all the images in the correct order. It’s not a fast process I think it may take me a week or so to do this.

My CPU hates me

So far, I’m up to 1903, so I’ll update with some rough idea of when this finished.

So, the applications needed are old and obsolete Win16 or Classic MacOS needed machines, with an optical drive. Yes they still work (with emulation) on modern machines, although you still need to read the physical discs. Thankfully the images are easily mapped into the right order, and you can map them as your own. Neat!

Totally unfair comparison of Microsoft C

Because I hate myself, I tried to get the Microsoft OS/2 Beta 2 SDK’s C compiler building simple stuff for text mode NT. Because, why not?!

Since the object files won’t link, we have to go in with assembly. And that of course doesn’t directly assemble, but it just needs a little hand holding:

Microsoft (R) Program Maintenance Utility   Version 1.40
Copyright (c) Microsoft Corp 1988-93. All rights reserved.

        cl386 /Ih /Ox /Zi /c /Fadhyrst.a dhyrst.c
Microsoft (R) Microsoft 386 C Compiler. Version 1.00.075
Copyright (c) Microsoft Corp 1984-1989. All rights reserved.

dhyrst.c
        wsl sed -e 's/FLAT://g' dhyrst.a > dhyrst.a1
        wsl sed -e "s/DQ\t[0-9a-f]*r/&XMMMMMMX/g" dhyrst.a1  | wsl sed -e "s/rXMMMMMMX/H/g" > dhyrst.asm
        ml /c dhyrst.asm
Microsoft (R) Macro Assembler Version 6.11
Copyright (C) Microsoft Corp 1981-1993.  All rights reserved.

 Assembling: dhyrst.asm
        del dhyrst.a dhyrst.a1 dhyrst.asm
        link -debug:full -out:dhyrst.exe dhyrst.obj libc.lib
Microsoft (R) 32-Bit Executable Linker Version 1.00
Copyright (C) Microsoft Corp 1992-93. All rights reserved.

I use sed to remove the FLAT: directives which makes everything upset. Also there is some weird confusion on how to pad float constants and encode them.

CONST   SEGMENT  DWORD USE32 PUBLIC 'CONST'
$T20001         DQ      0040f51800r    ;        86400.00000000000
CONST      ENDS

MASM 6.11 is very update with this. I just padded it with more zeros, but it just hung. I suspect DQ isn’t the right size? I’m not 386 MASM junkie. I’m at least getting the assembler to shut-up but it doesn’t work right. I’ll have to look more into it.

Xenix 386 also includes an earlier version of Microsoft C / 386, and it formats the float like this:

CONST   SEGMENT  DWORD USE32 PUBLIC 'CONST'
$T20000         DQ      0040f51800H    ;        86400.00000000000
CONST      ENDS

So I had thought maybe if I replace the ‘r’ with a ‘H’ that might be enough? The only annoying thing about the Xenix compiler is that it was K&R so I spent a few minutes porting phoon to K&R, dumped the assembly and came up with this sed string to find the pattern, mark it, and replace it (Im not that good at this stuff)

wsl sed -e "s/DQ\t[0-9a-f]r/&XMMMMMMX/g" $.a1 \
| wsl sed -e "s/rXMMMMMMX/H/g" > $*.asm

While it compiles with no issues, and runs, it just hangs. I tried the transplanted Xenix assembly and it just hangs as well. Clearly there is something to do with how to use floats.

I then looked at whetstone, and after building it noticed this is the output compiling with Visual C++ 8.0

      0       0       0  1.0000e+000 -1.0000e+000 -1.0000e+000 -1.0000e+000
  12000   14000   12000 -1.3190e-001 -1.8218e-001 -4.3145e-001 -4.8173e-001
  14000   12000   12000  2.2103e-002 -2.7271e-002 -3.7914e-002 -8.7290e-002
 345000       1       1  1.0000e+000 -1.0000e+000 -1.0000e+000 -1.0000e+000
 210000       1       2  6.0000e+000  6.0000e+000 -3.7914e-002 -8.7290e-002
  32000       1       2  5.0000e-001  5.0000e-001  5.0000e-001  5.0000e-001
 899000       1       2  1.0000e+000  1.0000e+000  9.9994e-001  9.9994e-001
 616000       1       2  3.0000e+000  2.0000e+000  3.0000e+000 -8.7290e-002
      0       2       3  1.0000e+000 -1.0000e+000 -1.0000e+000 -1.0000e+000
  93000       2       3  7.5000e-001  7.5000e-001  7.5000e-001  7.5000e-001

However this is the output from C/386:

      0       0       0  5.2998e-315  1.5910e-314  1.5910e-314  1.5910e-314
  12000   14000   12000  0.0000e+000  0.0000e+000  0.0000e+000  0.0000e+000
  14000   12000   12000  0.0000e+000  0.0000e+000  0.0000e+000  0.0000e+000
 345000       1       1  5.2998e-315  1.5910e-314  1.5910e-314  1.5910e-314
 210000       1       2  6.0000e+000  6.0000e+000  0.0000e+000  0.0000e+000
  32000       1       2  5.2946e-315  5.2946e-315  5.2946e-315  5.2946e-315
 899000       1       2  5.2998e-315  5.2998e-315  0.0000e+000  0.0000e+000
 616000       1       2  5.3076e-315  5.3050e-315  5.3076e-315  0.0000e+000
      0       2       3  5.2998e-315  1.5910e-314  1.5910e-314  1.5910e-314
  93000       2       3  5.2972e-315  5.2972e-315  5.2972e-315  5.2972e-315

Great they look nothing alike. So something it totally broken. I guess the real question is, does it even work on OS/2?

Since I should post the NMAKE Makefile so I can remember how it can do custom steps so I can edit the intermediary files. Isn’t C fun?!

INC = /Ih
OPT = /Ox
DEBUG = /Zi
CC = cl386

OBJ = dhyrst.obj

.c.obj:
	$(CC) $(INC) $(OPT) $(DEBUG) /c /Fa$*.a $*.c
	wsl sed -e 's/FLAT://g' $*.a > $*.a1
	wsl sed -e "s/DQ\t[0-9a-f]*r/&XMMMMMMX/g" $*.a1 \
	| wsl sed -e "s/rXMMMMMMX/H/g" > $*.asm
	ml /c $*.asm
	del $*.a $*.a1 $*.asm

dhyrst.exe: $(OBJ)
        link -debug:full -out:dhyrst.exe $(OBJ) libc.lib

clean:
        del $(OBJ)
        del dhyrst.exe
        del *.asm *.a *.a1

As you can see, I’m using /Ox or maximum speed! So how does it compare?

Dhrystone(1.1) time for 180000000 passes = 20
This machine benchmarks at 9000000 dhrystones/second

And for the heck of it, how does Visual C++ 1.0’s performance compare?

Dhrystone(1.1) time for 180000000 passes = 7
This machine benchmarks at 25714285 dhrystones/second

That’s right the 1989 compiler is 35% the speed of the 1993 compiler. wow. Also it turns out that MASM 6.11 actually can (mostly) assemble the output of this ancient compiler. It’s nice when something kind of work. I can also add that the Infocom ’87 interpreter works as well.

YAY!

Apparently talking about DOS Extenders is too hot for Twitter: AKA Phar Lap 386

I had a small twitter account, and I tried not to get dragged into anything that would just be basically wasting my time. Just stay focused and on topic. FINE. I just wanted to see if anyone ever saw it, if it was even worth the effort of doing WIP’s as I didn’t want to make it super annoying.

I logged on to post a fun update that I’d finally gotten a Phar Lap 386 version 4.1 app to do something halfway useful, the sairen AGI interpreter up and running in the most basic sense.

Talking about DOS Extenders is spammy and manipulation!

I don’t get what triggered it, but oh well there was a ‘have a review’ and yeah that was fine. Great. So I’m unlocked so I go ahead and post with the forbidden topic, as I’m clearly dumb, and forgetting that Twitter is for hate mobs & posting pictures of food, and cat pictures.

The Sairen AGI interpreter built with Watcom 386/7.0 & Phar Lap 386 4.1

So yes, that was a line too far, and now that’s it.

Now some of you may think, if you buy ‘the plan’ you’ll no doubt be exempt from the heavy hands of Twitter

3 squids a month

But I already was and had been for a while.

Your account is suspended

So that’s the end of that. I guess it’s all too confusing for a boomer like me.

Cancel me, cancel you

So needless to say I cancelled Twitter as well. Kind of sneaky they didn’t auto-cancel taking money.

So yeah, with that out of the way, let’s continue into DOS Extender land. I added just enough 386 magic, onto github: neozeed/sarien286. Yes I see now it really was a poorly named repo. Such is life.

There is 3 main things for porting old programs where they take care of all the logic, it’s going to be File I/O, Screen I/O, and timers. Luckily this time it was easier than I recalled.

Over on usenet (google groups link) Chris Giese shared this great summary on direct memory access from various methods:

/* 32-bit Watcom C with CauseWay DOS extender */
int main(void) {
char *screen = (char *)0xA0000;
initMode13();
*screen = 1;
return 0;
}

/* 32-bit Watcom C with DOS/4GW extender
(*** This code is untested ***) */
int main(void) {
char *screen = (char *)0xA0000;
initMode13();
*screen = 1;
return 0;
}

/* 32-bit Watcom C with PharLap DOS extender
(*** This code is untested ***) */
#include <dos.h> /* MK_FP() */
#define PHARLAP_CONVMEM_SEL 0x34
int main(void) {
char far *screen = (char far *)MK_FP(PHARLAP_CONVMEM_SEL, 0xA0000);
initMode13();
*screen = 1;
return 0;
}

/* 16-bit Watcom C (real mode) */
#include <dos.h> /* MK_FP() */
int main(void) {
char far *screen = (char far *)MK_FP(0xA000, 0);
initMode13();
*screen = 1;
return 0;
}

It is missing the Phar Lap 286 method:

/* Get PM pointer to text screen */
  DosMapRealSeg(0xb800,4000,&rseg);
  textptr=MAKEP(rseg,0);

But it’s very useful to have around as documentation is scarce.

Which brings me to this (again?)

Phar Lap 386|Dos-Extender 4.1

Years ago, I had managed to score a documentation set, and a CD-ROM with a burnt installed copy of the extender. I didn’t know if it was complete, but of course these things are so incredibly rare I jumped on the chance to get it!

2011!

Unfortunately, I didn’t feel right breaking the books apart, and scanning them, then add in some bad life choices on my part, and I ended up losing the books. Fast forward *years* later and Foone uploaded a document set on archive.org. GREAT! As far as I can tell the only difference in what I had is that I’ve got a different serial number. Thankfully I was smart enough to at lest email myself a copy of the CD-ROM contents! And this whole thing did inspire me to gut and upload the Phar Lap TNT 6.0 that I had also managed to acquire.

Although unlocking the video RAM wasn’t too bad, once I knew what to do, the other thing is to hook the clock for a timer. ISR’s are always hell, but at least this is a very simple one:

void (__interrupt __far *prev_int_irq0)();
void __interrupt __far timer_rtn();
int clock_ticks;
#define IRQ0 0x08
void main()
  {
   clock_ticks=0;
   //get prior IRQ routine
   prev_int_irq0 = _dos_getvect( IRQ0 );
   //hook in new protected mode ISR
   _dos_setvect( IRQ0, timer_rtn );

/* do something interesting */
   //restore prior ISR
   _dos_setvect( IRQ0, prev_int_irq0 );
  }

void __interrupt __far timer_rtn()
  {
    ++clock_ticks;
    //call prior ISR
    _chain_intr( prev_int_irq0 );
  }

The methodology is almost always the same, as always, it’s the particular incantation.

So yeah, it’s super simple, but the 8086/80286 calling down to DOS/BIOS from protected mode via the int86 just had to be changed to int386, and some of the register structs being redefined. I’m not sure why but the video/isr code compiled with version 7 of Watcom, but crashes. I think its more drift in the headers, as the findfirst/findnext/assert calls are lacking from Watcom 7, so I just cheated and linked with Watcom 10. This led to another strange thing where the stdio _iob structure was undefined. In Watcom 10 it became __iob, so I just updated the 7 headers, and that actually worked. I had to include some of the findfirst/next structures into the fileglob.c file but it now builds and links fine.

Another thing to do differently when using Watcom 7, is that it doesn’t include a linker, rather you need to use 386LINK. Generating the response file, as there is so many objects didn’t turn out too hard once I realized that by default everything is treated as an object.

Another fun thing is that you can tell the linker to use the program ‘stub386.exe’ so that it will run ‘run386’ on it’s own, making your program feel more standalone. From the documentation:

386 | LINK has the ability to bind the stub loader program, STUB386.EXE, to 
the front of an application .EXP file. The resulting .EXE file can be run by 
typing the file name, just like a real mode DOS program. The stub loader 
program searches the execution PATH for RUN386.EXE (the 

386 | DOS-Extender executable) and loads it; 386 | DOS-Extender then loads 
the application .EXP file following the stub loader in the bound .EXE file. 


To autobind STUB386.EXE to an application .EXP file and create a bound 
executable, specify STUB386.EXE as one of the input object files on the 
command line.

So that means I can just use the following as my linker response file.

agi.obj,agi_v2.obj,agi_v3.obj,checks.obj,cli.obj,console.obj,cycle.obj
daudio.obj,fileglob.obj,font.obj,getopt.obj,getopt1.obj,global.obj
graphics.obj,id.obj,inv.obj,keyboard.obj,logic.obj,lzw.obj,main.obj
menu.obj,motion.obj,pharcga3.obj,objects.obj,op_cmd.obj,op_dbg.obj
op_test.obj,patches.obj,path.obj,picture.obj,rand.obj,savegame.obj
silent.obj,sound.obj,sprite.obj,text.obj,view.obj
words.obj,picview.obj stub386.exe
-exe 386.exe
-lib \wat10\lib386\dos\clib3s.lib \wat10\lib386\math387s.lib
-lib \wat10\lib386\dos\emu387.lib

It really was that simple. I have to say it’s almost shocking how well this went.

So, this brings me back, full circle to where it started, me getting banned for posting this:

32bit!

I thought it was exciting!

For anyone who feels like trying it, I prepped a 5 1/4″ floppy disk image.

running on 86box, 386DX-40 CGA graphics

One interesting observation is that the 386 extender is actually smaller than the 286 one. And being able to compile with full optimisations it is significantly faster.

16bit on the left, 32bit on the right.

I ran both the prior 16bit protected mode version (on the left), and 32bit version (on the right), on the same IBM PS/2 80386DX 16Mhz machine. You can see how the 32bit version is significantly faster!.

I really should profile the code, and have it load all the resources into RAM, it does seem to be loading and unloading stuff, which considering were in protected mode, we should use all ram, or push the VMM386 subsystem to page, and not do direct file swapping, like it’s the 1970s.

Phar Lap’s 286/DOS-Extender: why nobody used it for games

Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.

-Ian Malcolm

Ever since I got my first 286 board way back in the early 90’s (1990? 1991?) I have been intrigued by the whole protected mode of operation. Unfortunately, in the era the required tools were way out of my reach, and of course were not available at retail. But now I live in the future where I have all the parts! Let’s look at the needed parts

I have to once more again thank my patrons, and people tolerating the google ads as it made all the difference in being able to buy all this stuff. And now is as good a time as any other to put it all together.

I stumbled upon this repo on sourceforge, Sarien. It included a Turbo C++ port, which is pretty exciting! So, this became my goal, get Sarien running on Phar Lap 286.

Installing Microsoft C 6.0a

Installing Microsoft C requires you to pick and choose both hosting, targeting environments, along with what the preferred libraries are. In the business we call this foreshadowing as this can be such a giant PITA. At least virtual machines are fast, plentiful, and cheap. In addition I had been using MS-DOS player to host the tools on Windows 11. This of course proved weird later.

The first step was getting it running on MS-DOS using Microsoft C 6.0a. This was actually pretty easy, the hard part was working out the makefile, as some files don’t compile with optimisations. And overall, the project doesn’t seem to work with /Ox at all. I haven’t spent enough time mixing and matching settings to find what actually doesn’t work, but I’m in a hurry, and /Os seems to work just fine.

In no time I had both the CGA & VGA drivers up and running and verified working on my PS/2. Great!

Now comes the fun, getting it ready to run on Pharlap.

The magic!

Phar Lap’s 286|DOS-Extender is pure magic. A DOS extender is a special program that can load a protected mode program into memory on a 286 or better computer and run it. At it’s heart, it can proxy MS-DOS functionality from protected mode to real mode, allowing you to use a lot of methodology and code from traditional real mode code. Phar Lap, goes beyond that by providing a pseudo OS/2 1.2 environment on MS-DOS, including advanced features like DLL’s, and being able to use ALL the RAM in your computer. Of course on the 286 there is a massive caveat:

The 286 has no built-in function for switching from protected mode to real mode. This makes programs that rely a LOT on MS-DOS potentially very slow. You can absolutely feel the difference between the real mode and the protected mode version of Sarien.

Phar Lap does include a test program, swtest which can benchmark the switching methods, so let’s run it and get some scores.

Switch code version = 1.14
BIOS signature: BA66CC86
BIOS date: 02/13/87

Machine ID = 0, A20 method = PS2, Reset method = Standard
Starting test for Switch Mode 3 (SLOW) ... Test complete.
Avg switch time (usecs): To prot = 34, To real = 101, Total = 135
Min switch time (usecs): To prot = 32, To real = 98, Total = 130
Max switch time (usecs): To prot = 35, To real = 103, Total = 138

Machine ID = 0, A20 method = PS2, Reset method = Standard
Starting test for Switch Mode 2 (AT) ... Test complete.
Avg switch time (usecs): To prot = 34, To real = 86, Total = 120
Min switch time (usecs): To prot = 33, To real = 83, Total = 116
Max switch time (usecs): To prot = 36, To real = 88, Total = 124

Machine ID = 0, A20 method = PS2, Reset method = Standard
Starting test for Switch Mode 1 (SURE) ... Test complete.
Avg switch time (usecs): To prot = 34, To real = 70, Total = 104
Min switch time (usecs): To prot = 32, To real = 68, Total = 100
Max switch time (usecs): To prot = 35, To real = 72, Total = 107

For those of you wondering what the timing is like on a 386, here is my 16Mhz PS/2 Model 80 board (now with fully 32bit memory)

Switch code version = 1.14
BIOS signature: 039D2DB4
BIOS date: 03/30/87

Machine ID = 0, A20 method = PS2, Reset method = Standard
Starting test for Switch Mode 5 (386) ... Test complete.
Avg switch time (usecs): To prot = 31, To real = 22, Total = 53
Min switch time (usecs): To prot = 30, To real = 20, Total = 50
Max switch time (usecs): To prot = 32, To real = 23, Total = 55

I’m honestly surprised the 286 switches from protected back to real so quickly! Although as you can see from the 386 timings it’s significantly faster than the 286.

Here is a quick video, real mode on the left, protected mode on the right. Yes I need to get a VGA capture card. Sorry.

Real mode on the left, Protected on the right. Running on the 386

Being a DOS extender it does have built in functions for things like hooking interrupts like this:

    /* install our new timer tick routine */
    DosSetPassToProtVec(IRQ0, (PIHANDLER)new_prot_timer_tick,
        &old_prot_timer_tick, &old_real_timer_tick);

Unlike some kind of OS/2 method which would involve creating a thread and or timers.

Setting video modes via the video BIOS is also supported, using the built in int86 style calls:

    memset(&r,0x0,sizeof(r));
    r.h.ah = 0;
    r.h.al = 3;
    int86(0x10, &r, &r);

Using pointers into things like video ram do require ‘asking for permission’ but it’s not too involved:

    int rseg;
  /* Get PM pointer to text screen */
    DosMapRealSeg(0xb800,4000,&rseg);
    textptr=MAKEP(rseg,0);

with the segment mapped, and a pointer to the segment, and now I can read/write directly into video RAM!

  /* save text screen */
    memcpy(textbuf,textptr,4000);

Just like that!

I’m not sure what I screwed up on the VGA graphics, as it doesn’t work correctly, but oddly enough CGA does work.

And now this is where everything goes off the rails.

Sairen running on Qemu & DOSbox

It ran fine on emulation. So all excited I fired up the PS/2 and….

General protection fault

This lead me to more fun in how on earth to debug this. Of course Phar Lap 286 version 2.5 requires me to have Microsoft C/C++ 7.0. I shamelessly downloaded a disk set from pcjs.org. You actually need to install it, to copy out the files required:

  • 28/05/1991 05:37 pm 47,216 CFIG286.EXE
  • 26/11/1991 11:19 am 13,531 GORUN286.EXE
  • 19/03/1992 04:00 am 42,720 shw0.dll
  • 19/03/1992 04:00 am 105,039 eew0cxx.dll
  • 19/03/1992 04:00 am 410,112 cvw4.exe
  • 19/03/1992 04:00 am 91,118 eew0can.dll
  • 19/03/1992 04:00 am 74,400 emw0w0.dll
  • 03/08/1992 09:34 pm 2,649 INT33.DLL
  • 03/08/1992 09:40 pm 2,718 MSG.DLL
  • 03/08/1992 09:40 pm 1,702 NAMPIPES.DLL
  • 03/08/1992 09:42 pm 2,073 NLS.DLL
  • 03/08/1992 09:43 pm 5,184 PTRACE.DLL
  • 03/08/1992 09:45 pm 2,320 SESMGR.DLL
  • 03/08/1992 09:50 pm 1,508 WIN87EM.DLL
  • 05/08/1992 12:04 am 3,100 KEYBOARD.DLL
  • 05/08/1992 06:33 pm 270 TOOLHELP.DLL
  • 14/08/1992 07:38 pm 7,891 KERNEL.DLL
  • 14/08/1992 09:40 pm 14,545 USER.DLL
  • 09/09/1992 10:59 pm 209,922 RUN286.EXE
  • 09/09/1992 10:59 pm 229,046 RUN286D.EXE
  • 14/09/1992 11:01 pm 14,024 TLW0LOC.DLL
  • 17/09/1992 07:26 pm 34,152 CVP7.EXE

While CVP7 does come with Phar Lap, you have to run it via run286. As you may have noticed there is a mixture of OS/2 and Windows DLL’s in here, as at this point CodeView was a Windows protected mode debugger. The divorce was in full swing, and Microsoft C/C++ 7.0 had amputated the majority of OS/2 support. I’m sure all this is in the manuals, however all I have is disk images. There was no C 6.0a supported hosted debugger. Maybe it’s in the v1/v2 of Phar Lap 286, but I only have 2.5. There is version 3 files on the internet but I wanted to stick to 2.5.

Exception #13

And this is all I got. Yes, I did recompile with ‘/Od /Zi’ along with using cvpack on the executable. Yes, after copying the source to the PS/2 I was able to see the source line mapping, but it immediately jumps to assembly and GP Faults. All this is fine, but IT RUNS UNDER EMULATION.

What is going on?!

I asked around on discord, and found someone willing to test on their 286. It also crashed. I tried VMware and .. it crashed too! So did 86box! Ok now we’re going somewhere!

Since I had been using MS-DOS player to run the tools, I had an issue with the linker in C 6.0a crashing, so I tried the one from C/C++ 7. It also didn’t work. I tried the one from Visual C++ 1.5. It also failed. Almost giving up on the entire thing, since I had copied the source code to the PS/2, I tried something really silly, I compiled it using the /qc or QuickC flag. I wasn’t too worried about sizes as again I’m going to run in protected mode. It took some 20-30 minutes to compile, as 10Mhz machines are not the best for building software in this modern age. Much to my surprise it actually ran.

First run!

This was kind of shocking as I’m not sure what I screwed up to not get this to work, but it worked! I went ahead and changed the build to not use QuickC, but rebuild with /Os (Optimize for space). It took about an hour. And it too worked.

Phar Lap 286 running on OS/2 2.00 on VMware

Shockingly it runs! I’m not sure what on earth is up with the linking. I did find it easier to just rebuild on Qemu since it can easily map into my source directory and copy everything over and re-build very quickly.

Later I did try copying over compiled objects built using the MS-DOS Player, and linked them natively, and they ran fine.

What is it with the LINK?!

It all fits! Stacker rules!

One weird thing on 86box is my pre-built machine I was using has a 5 1/4″ 1.2Mb floppy for the A: drive. It’s too small to fit MS-DOS, the game data and Phar Lap 286 all on there. Although Stacker to the rescue and it fits!

It can save and even load those saves!

I removed a lot of Unix quality of life, to make it more MS-DOS dumping everything in the same directory so you can save & load games.

Assuming anyone is interested in this at all, I have the source up on github. I’ll follow up with some performance videos showing how much slower real vs protected mode is, along with some binaries/demos. A 5 1/4″ floppy disk image can be downloaded here for any suitable emulator.