Thunking for fun & a lack of profit

So, with a renewed interest in OS/2 betas, I’d been getting stuff into the direction of doing some full screen video. I’d copied and pasted stuff before and gotten QuakeWorld running, and I was looking forward to this challenge. The whole thing hinges on the VIO calls in OS/2 like VioScrLock, VioGetPhysBuf, VioScrUnLock etc etc. I found a nifty sample program Q59837 which shows how to map into the MDA card’s text RAM and clear it.

It’s a 16bit program, but first I got it to run on EMX with just a few minor changes, like removing far pointers. Great. But I wanted to build it with my cl386 experiments and that went off the edge. First there are some very slick macros, and Microsoft C just can’t deal with them. Fine I’ll use GCC. Then I had to get emximpl working so I could build an import library for VIO calls. I exported the assembly from GCC, and mangled it enough to where I could link it with the old Microsoft linker, and things were looking good! I could clear the video buffer on OS/2 2.00 GA.

Now why was it working? What is a THUNK? Well it turns out in the early OS/2 2.0 development, they were going to cut loose all the funky text mode video, keyboard & mouse support and go all in on the graphical Presentation Manager.

Presentation Manager from OS/2 6.605

Instead, they were going to leave that old stuff in the past, and 16bit only for keeping some backwards compatibility. And the only way a 32bit program can use those old 16bit API’s for video/keyboard/mouse (etc) is to call from 32bit mode into 16bit mode, then copy that data out of 16bit mode into 32bit mode. This round trip is called thunking, and well this sets up where it all goes wrong.

Then I tried one of the earlier PM looking betas 6.605, and quickly it crashed!

SYS2070:

Well this was weird. Obviously, I wanted to display help

Explanation:

This ended up being a long winded way of saying that there is missing calls from DOSCALL1.DLL. Looking through all the EMX thunking code, I came to the low level assembly, that actually implemented the thunking.

EXTRN   DosFlatToSel:PROC
EXTRN   DosSelToFlat:PROC

After looking at the doscalls import library, sure enough they just don’t exist. I did the most unspeakable thing and looked at the online help for guidance:

No VIO

So it turns out that in the early beta phase, there was no support for any of the 16bit IO from 32bit mode. There was no thunking at all. You were actually expected to use Presentation Manager.

YUCK

For anyone crazy enough to care, I uploaded this onto github Q59837-mono

It did work on the GA however so I guess I’m still on track there.

Porting GCC to 32bit OS/2

I know what you are going to think, that it was already done, and it was called EMX. Or was it GCC/2? Well sure but what if you are not running the GA (General availability) version of OS/2. For example, years ago I had managed to get Citrix Multiuser 2.0, and it’s not at the GA level. All that is available is some ancient beta version of Microsoft C 5.2 from 1989?!

A little while back I had worked on getting GCC to build and run on the FPU enabled versions of Windows NT from 1991. I had mentioned that it turns out thanks to the Xenix assembler, that GCC had been basically available the entire time Windows NT had been available, but lamented that since the OS/2 compiler is 16/32bit, the 5.2 compiler couldn’t handle compiling GCC without blowing it’s heap. 16bit issues in a 64bit world.

However after doing some research on all the early cl386 compilers I could get my hands on, including the Windows NT Pre-release ones, I’d noticed that if I built CC1.EXE (the actual compiler) first for Win32, then rebuilt those object files with the December NT Pre-release compiler, that some versions of LINK386 from the OS/2 2.1 DDK would actually link with them. And sure enough it worked!

First life of GCC on OS/2

I have to admit I was pretty amazed, I had managed to ‘cross compile’ GCC using quite the tool chain.

First the compiler from the December NT Pre-release CD-ROM is shipped as a 16bit OS/2 compiler, but I’m using Windows 11. First I use the MS-DOS player with a quick fix from crazyc to allow Phar Lap 286|Dos Extender to run, which provides a basic enough OS/2 emulation to allow the compiler to run under ‘dos’. The linker on the DDK suffers the same fate as far as it also being 16bit. However the combination of MS-DOS player & Phar Lap gets stuff working! The only weird catch is that the 386 emulator causes strange floating point related crashes, while the 286 or 486 emulators work fine.

Now targeting OS/2 or running on OS/2 isn’t all that new, but building it from a Microsoft C compiler is. And now of course you’ll ask yourself, who cares? why is it interesting?

Well, the vast majority of the GCC ports to OS/2 don’t support the OMF object file binary standard, instead they used the much outdated a.out format, and rely on tools to convert the objects if needed. Additionally, they have DLL dependencies, and other startup issues with things needing to be setup. And of course they rely on a binary standard that is ‘GA’. *HOWEVER* by using a Microsoft compiler, I have OMF object files that the OS/2 built in system linker LINK386 can understand. So in plain English I can just relink the compiler and it’ll run on a new ‘version’ of unsupported OS/2.

I made a diskette image with my objects & a linker script and in a few moments I had it running!

GCC running on OS/2 2.00 6.123

The substantial thing here is that the binary format for OS/2 changed twice, and each release introduced changes that broke binary compatibility, in an effort to force people onto the new tools. So there is no way that the old ‘LE’ format would ever work. And you can see it’s running! In addition I could take the same object files, and copy them to my Citrix server, and likewise it was just a matter of linking, and it too now has GCC!

Converting a.out to OMF via emxomf

One annoying thing is that the LINK386 that ships with OS/2 2.00 GA doesn’t like the output of the Xenix assembler, so I built the a.out traditional assembler, and the emxomf tool to convert the a.out to OMF, and that worked well.

I still have much to mess with, including the pre-processor & main ‘gcc’ program. I have not built anything beyond a trivial program, so there is indeed much more work to be done before I can even try anything challenging. Some programs like emxomf have portions in the debug support that require the ‘long long’ type, which obviously Microsoft compilers from 1989-1991 don’t have, so I’ll have to re-build them with GCC.

Ive been putting my ports onto github (cl386-research) as it handles the rapid changes well enough. It’s a bit of a complex setup and it involves using a build system that I’ve put over on archive.org here: cl386-research-v2.

Not that I can imagine anyone wanting to try but I’ve uploaded some disks with the objects. Copy them to a hard drive, and run the ‘build.cmd’ command and it’ll link to a native freestanding executable.

I’ll explain it more with a post later, along with going over all the versions of cl386 I’ve acquired, over the years in more of a part 2: Targeting OS/2 with Visual Studio 2003!

So it turns out GCC could have been available on Windows NT the entire time!

This is going to be a bit convoluted but here goes.. GCC isn’t a monolithic compiler, instead it’s various parts are separate programs. This lets us tackle it one part at a time. And/Or bypass a lot of it until I want to tackle it.

Flow of GCC

I’m sure many people have explained this far better than I ever could but in C you write source files (obviously), the pre-processor reads those and ‘header’ files that describe interfaces to libraries, other objects, various macros and definitions (magical numbers) and the pre-processor will read those files, and do simple macro expansion and test insert/replacements to generate a single .i file at the end of it’s run.

The C compiler (cc1) now reads that single .i file and translates it into native assembly. This allows for ‘mid/high level’ aspects of C to be machine independent (portable) but now will be written into a very system dependant assembly file, the single .S file. One thing of note is that so far everything is text files. You can edit the assembly file as you would any document, or even further ‘process’ it if needed/wanted.

The assembler ax386 (GAS) will then read the single assembly file and write a a binary object file hi.OBJ. There typically isn’t all that much to be said about assemblers although fancier ones allow for really strong Macro capabilities like Microsoft MASM.

From here on, it’s all binary objects!

The linker then takes your object files, and links them together with other system objects and system libraries into an executable, in this case. Linkers can build all kinds of other things, but for now we’re just pretending its static C compilation like it’s the 1970’s.

At it’s heart GCC processes text files.

The first part in this insane experiment, is to build GCC 1.40 with Microsoft Visual C++ 1.0. Surprisingly it didn’t take an insane amount of messing with stuff, and I got an executable! But everything it compiled failed to assemble. Looking at this fragment, even if you don’t know i386 assembly you might spot the error:

main:
        pushl %ebp
        a b,c
        pushl %esi
        pushl %ebx

Yeah, it’s the “a b,c” part. Those are NOT valid i386 opcodes!

Just because it compiled didn’t mean it actually worked.

I used MinGW to build the same source, same Makefile, and I got a working executable. Annoyed I started compiling random files with Microsoft C, and finally found the file that broke it all, it turned out to be insn-output.c needing to be compiled with the “/D__STDC__” flags. A quick modification of the Makefile and now I have a working CC1!

Okay, great, it’s well known back in the early dangerous ages of the 1980’s/1990’s that everyone wasn’t running Linux, nor were binary distributions of GCC that far spread, rather I think to re-enforce the source was available it was expected that you’d use your system compiler. Systems like DJGPP/EMX take the path of binding a.out object files into something that MS-DOS can run via a dos extender, or the bind utility to allow you to run the a.out on OS/2. What I wan’t to do is verify that in fact Windows NT was a viable host for GCC back in the public pre-releases of 1991.

I’m sticking with the December build 239 version as it has working floating point. Something that GCC has intrinsic support of, and I don’t feel like trying to work out emulation.

The next step is to try to build it with the family mode-OS/2 version of the C compiler, which of course lead to the real issue of this 16bit hosted cross compiler:

        cl386 /u /Od /Ic:\MSVC32S\C386\INCLUDE /I. /Iconfig /c combine.c
Microsoft (R) Microsoft 386 C Compiler. Version 1.00.075
Copyright (c) Microsoft Corp 1984-1989. All rights reserved.

combine.c
combine.c(1734) : fatal error C1002: compiler is out of heap space in Pass 2
NMAKE : fatal error U1077: 'C:\WINDOWS\system32\cmd.exe' : return code '0x2'
Stop.

Very frustrating. I tried mixing and matching from Visual C++ 1.0 & this old compiler, and while it did compile, it doesn’t run. does it mean anything?!

GCC 1.40 compiled by Microsoft 386 C Compiler. Version 1.00.075

I should point out that this should be an expected working configuration as GCC does build on Xenix using the 32bit Microsoft C 5.1/386 compiler. Furthered again that Xenix and these 1991 versions of NT use the same 32bit OMF object format. And expanding on the Xenixnt experiment using the Xenix’ified GAS assembler with old Visual C++ includes & libraries to produce a possible retro-early port of GCC to NT, the next move is to bulid GAS on NT.

Xenix GAS 1.38 compiled by by Microsoft 386 C Compiler. Version 1.00.075

GAS gave me some weird issues with ctype.h where it runs fine with the one from Visual C++ 1.0 but the OS/2 & NT pre-release both fail. However the old Pre-release compiler cannot deal with the much newer ctype include file. So after much hammering I amputated whatever was bothering it, and it’s just enough to build & run. Great!

Going back to the phases, I used a simple hello world program:

void main() {
 printf("Hello World!\n");
}

While not being a good program, it doesn’t include stdio.h, nor does it return anything. It’s terrible. But in this case it allows me to be lazy and sidestep the pre-processor cpp.exe. This way I can just directly run it through cc1 and get my assembler file hi.S

Next I pass it to ax386 (GAS) and get the resulting object file hi.OBJ

And finally link it with link.exe in this case.

Hello World from GCC 1.40 on NT!

And with all the drama I’ve now compiled a simple hello world program on Windows NT.

If it were 1991, I would hollow out gcc.c so it doesn’t use signals or forks to invoke the needed phases, and of course build the pre-processor. In addition, libgcc needs to be compiled to allow for floating point operations to work correctly. None of which is impossible, although I’m not sure it’s all that needed as it isn’t 1991.

phoon

With a little bit more work, I got the floating point support to compile, which relies on both a working ‘native’ compiler, and a working GCC to compile the 2nd half. I usually use phoon, or Phases of the Moon, to test floating point, and as you can see, it’s working!

I’m not sure if there was a 32bit version of Microsoft C/386 available for Microsoft OS/2 2.00 betas. Also, I don’t know if the Microsoft link386 for OS/2 can also link Xenix 386 object files? Would it have been possible to bootstrap GCC/GAS on Microsoft OS/2 2.00? I really don’t know, and as of this writing no versions of the old Microsoft OS/2 2.00 betas have surfaced.

** update from the future, turns out that I found a way to convince the cl386 compilers from the NT Pre-Releases in 1991 to re-build an existing GCC that was built for NT. The catch is the linker, LINK386 of course, as the format was constantly changing. However the object files are fine, and I was able to just copy them over on diskette and re-link the compiler. It even ran. It’s not tested at all, so it turns out the 1989 compiler wasn’t good enough, but the 1991 was.

GCC 1.40 on OS/2 2.00 beta 6.123

It’s interesting to me to see that even before GCC 2.6, that vintage versions from 1991 would compile and run directly on Windows NT.

I uploaded the source on github, along with some binaries.

Since people were asking for xMach binaries

xMach doing it’s Linux calibration

Since binaries had been requested, along with the old elf cross compiler I thought I’d try that new fangled github binary releases.

This is just taken from old artifacts from the old Building OSkit & xMach adventures.

I had made a vmdk, MachUK22-lites.vmdk.7z as well, not sure if that helps anyone.

Fun with Windows ePDK NT 3.1 build 196 & Some Xenix fun!

I’m not sure if I covered the Windows NT 3.1 build 196 before. First the most obvious is that as of this moment it’s the earliest version of Windows NT available.

So let’s do some obligatory scratch of the surface. Like all the other 1991 pre-releases there really is just a text mode setup install script. Choosing the lesser amount of pain, I went with a MS-DOS hosted install. However, using MS-DOS 6.22 resulted in a broken dual boot system. But we live in the era of virtual machines, so it really doesn’t matter. I’m using Qemu 0.14-rc2, something that is ‘era correct’ for when the first avalanche broke on early Windows NT pre-releases. I’ve had issues with more modern versions of Qemu, and I felt that if we’re using vintage software may as well go in all the way.

The boot loader identifies itself as being 1990 vintage. Pretty sure it doesn’t mean anything, but we haven’t been blessed with the “blue screen” of ARC yet.

The login screen and desktop have a very strong Windows 3.1 beta feel to them. And that would land this where it was, Windows was such a big seller that Maritz had been trying to convince Gates & Balmer to ‘switch to the Windows horse’ in the spring/summer of 1990, culminating in Gate’s July decision to walk away from NT/OS2 and rebrand the new OS as Windows NT. Oddly enough it was Balmer who was in favour of OS/2 & IBM. (Showstopper 89-90)

At first glance the opening window isn’t all that interesting. It’s just very. Windows.

Rest assured Reversi, of course made the this Win32 cut. And it’s full of weird easter eggs, oddly enough in the OS/2 surviving bits.

type OS2LDR.DOS
OS2LDR 01.00.01
by KeithMo 01/08/91

On the Discord there had been a big discussion about early NT executable formats, and the whole COFF vs ECOFF vs PE/PEI. I had tried to hunt down the specific version of GCC I used ages to to build a so called Dec Alpha GCC cross compiler, but the short version is that it didn’t work as we don’t have any assembler/linker for anything GNU targets. There had been a cygwin port and an OpenNT on Alpha, but all that is lost to the winds, minus what few scraps I had saved. I did try building some cross tools to elf hoping to just objcopy out the data and get linkable objects, but that didn’t work either.

So I though this was a perfect opportunity to take a look at this early pre-release version of NT, and although I do know that you have to convert your objects into something the ‘COFF’ linker will accept:

And I never really paid that much attention to the object files. I do know that you can link them with Link386 for OS/2, but the NT object files themselves report:

wsl file SIMPLE.OBJ
SIMPLE.OBJ: Intel 80386 COFF object file, not stripped, 4 sections, symbol offset=0x102, 20 symbols, created Thu Jun  1 14:13:50 2023, 1st section name ".text"

Well, now that is interesting. And of course the COFF thing reminded me of Xenix! And sure enough ages ago I had found the source to a modified version of GNU GAS that outputs COFF. Once more again this is an indication that all 386 roads in Microsoft really did originate with Xenix. It’s too bad there never was a Windows/386 on Xenix/386. What an incredible OS that would have been! There must be some incredible stories from the tool teams that worked on Microsoft C/386 along with other projects. Oddly enough they never get anywhere near as much exposure as Office or OS.

Now this is fun, but nothing takes in these ancient COFF objects, do they? I tried to run LINK 1.0 from Win32s SDK and surprisingly it didn’t complain about the object, rather, it auto converted it in memory:

Microsoft (R) 32-Bit Executable Linker Version 1.00
Copyright (C) Microsoft Corp 1992-93. All rights reserved.

SIMPLE.OBJ : warning LNK4016: unresolved external symbol "__chkstk"
LINK : warning LNK4016: unresolved external symbol "_mainCRTStartup"
SIMPLE.OBJ : warning LNK4016: unresolved external symbol "_printf"
SIMPLE.exe : error LNK1120: 3 unresolved externals

Does this mean that if I give it some libraries it will actually link?

LINK.EXE SIMPLE.OBJ /SUBSYSTEM:CONSOLE /MACHINE:i386 -entry:mainCRTStartup -out:simple.exe libc.lib kernel32.lib
Microsoft (R) 32-Bit Executable Linker Version 1.00
Copyright (C) Microsoft Corp 1992-93. All rights reserved.


C:\temp\nt196\x\dec\x>simple
Win32, it's happenin'!

This was. VERY unexpected.

So I had this crazy idea, what if the Xenix assembler could in fact build objects that are also compaible in this manner? I used the a.out GCC / Linux porting tools I had built so I could compile Linux on Windows NT using the vintage tools as a starting point. I guess I should also add that when people always say ‘use newer version of THING’ this is how you miss out on old stuff like this. If I had been obsessed with using modern tools and modern operating systems, I’d have missed out on this Xenix filled window.

I took the gcc driver & the cc1 compiler from 1.40 and the c pre-processor from 2.5.8 as it can understand C++ comments. First I manually compiled the ‘simple’ example to assembly:

gcc -v -nostdinc -I/xenixnt/h -S SIMPLE.c -O simple.S
gcc version 1.40
 cpp -nostdinc -v -I/xenixnt/h -undef -D__GNUC__ -Dunix -Di386 -D__unix__ -D__i386__ -D__OPTIMIZE__ SIMPLE.c C:/Users/jsteve/AppData/Local/Temp/cca2_048.cpp
GNU CPP version 2.5.8 (80386, BSD syntax)
#include "..." search starts here:
#include <...> search starts here:
 /xenixnt/h
End of search list.
 cc1 C:/Users/jsteve/AppData/Local/Temp/cca2_048.cpp -quiet -dumpbase SIMPLE.c -O -version -o SIMPLE.s
GNU C version 1.40 (80386, BSD syntax) compiled by GNU C version 5.1.0.
default target switches: -m80387
 cpp -nostdinc -v -I/xenixnt/h -undef -D__GNUC__ -$ -Dunix -Di386 -D__unix__ -D__i386__ -D__OPTIMIZE__ simple.S C:/Users/jsteve/AppData/Local/Temp/cca2_048.s
GNU CPP version 2.5.8 (80386, BSD syntax)
#include "..." search starts here:
#include <...> search starts here:
 /xenixnt/h
End of search list.

Which gave me the following assembly:

        .file   "SIMPLE.c"
gcc_compiled.:
.text
LC0:
        .ascii "Win32, it's happenin'!\0"
        .align 2
.globl _main
_main:
        pushl %ebp
        movl %esp,%ebp
        pushl $LC0
        call _printf
        leave
        ret

Now to assemble with the GAS Xenix assembler

C:\temp\nt196\files\MSTOOLS\SAMPLES\SIMPLE>ax386 SIMPLE.s -o SIMPLE.obj

C:\temp\nt196\files\MSTOOLS\SAMPLES\SIMPLE>wsl file SIMPLE.obj
SIMPLE.obj: 8086 relocatable (Microsoft), "SIMPLE.c", 1st record data length 10, 2nd record type 0x88, 2nd record data length 11

Not quite the same. But it does closer resemble the output from the OS/2 bound versions of the Pre-Rease compilers:

file EMPTY.OBJ
EMPTY.OBJ: 8086 relocatable (Microsoft), "empty.c", 1st record data length 9, 2nd record type 0x88, 2nd record data length 7

So will it link?!

C:\temp\nt196\files\MSTOOLS\SAMPLES\SIMPLE>LINK.EXE SIMPLE.OBJ /SUBSYSTEM:CONSOLE /MACHINE:i386 -entry:mainCRTStartup -out:simple.exe libc.lib kernel32.lib
Microsoft (R) 32-Bit Incremental Linker Version 2.50
Copyright (C) Microsoft Corp 1992-94. All rights reserved.

SIMPLE.OBJ : warning LNK4033: converting object format from OMF to COFF

C:\temp\nt196\files\MSTOOLS\SAMPLES\SIMPLE>simple
Win32, it's happenin'!

Well now this is interesting! LONG before MinGW, or the GCC port to Windows NT, it turns out that in fact GCC could target Windows NT the entire time!

So the next thing to do is something not as trivial, like phoon.

I setup some quick script to pre-process, compile, assemble and then try to link, but as this one uses floating point, disaster struck:

phoon.obj : error LNK2001: unresolved external symbol "___fixdfsi"
astro.obj : error LNK2001: unresolved external symbol "___fixdfsi"
phoon.exe : error LNK1120: 1 unresolved externals

Now ages ago while messgin with old GCC & DooM I also had weird math calls not working. In the end I ended up extracting them from libgcc builds, so I thought I’d try the libgcc built during the GCC 2.6.3 on NT adventure.

link /NODEFAULTLIB:libc.lib /NODEFAULTLIB:OLDNAMES.LIB -out:phoon.exe astro.obj date_p.obj phoon.obj -entry:mainCRTStartup libgcc1.lib LIBC.LIB KERNEL32.LIB
Microsoft (R) 32-Bit Incremental Linker Version 2.50
Copyright (C) Microsoft Corp 1992-94. All rights reserved.

And of course:

To try to make the steps make a little more sense, and to allow for some higher level of automation I made a Makefile:

CC=gcc
CC1=cc1
AS=ax386
CPP=cpp


CFLAGS= -O
CPPFLAGS= -lang-c-c++-comments -nostdinc -I/xenixnt/h

OBJ =   astro.obj \
date_p.obj \
phoon.obj

LIBS = libgcc1.lib LIBC.LIB KERNEL32.LIB

phoon.exe: $(OBJ)
        link -out:phoon.exe $(OBJ) -entry:mainCRTStartup $(LIBS)

%.obj: %.c
        $(CPP) $(CPPFLAGS) $< $*.i
        $(CC1) $*.i -quiet $(CFLAGS) -version -o $*.S
        $(AS) $*.S -o $*.obj


clean:
        del $(OBJ) *.i *.S phoon.exe

I’m sure there is better ways to do this, but it breaks the compile up to it’s individual parts:

Run the pre-processor to allow // in the comments, C++ hadn’t been the default thing back when GCC 1.40 was a thing. Also path it to the headers, in this case I’m using the ones from NT 196. Trying to link with the 196 libraries gave me this:

C:\xenixnt\demos\phoon>link /NODEFAULTLIB:LIBC.LIB /NODEFAULTLIB:OLDNAMES.LIB -out:phoon.exe astro.obj date_p.obj phoon.obj -entry:mainCRTStartup base.lib wincrt.lib ntdll.lib \xenixnt\lib\libgcc1.lib
Microsoft (R) 32-Bit Incremental Linker Version 2.50
Copyright (C) Microsoft Corp 1992-94. All rights reserved.

wincrt.lib(maincrt0.obj) : warning LNK4078: multiple ".data" sections found with different attributes (40000040)
astro.obj : error LNK2001: unresolved external symbol "_asin"
astro.obj : error LNK2001: unresolved external symbol "_atan"
astro.obj : error LNK2001: unresolved external symbol "_atan2"
phoon.obj : error LNK2001: unresolved external symbol "_cos"
astro.obj : error LNK2001: unresolved external symbol "_cos"
astro.obj : error LNK2001: unresolved external symbol "_floor"
astro.obj : error LNK2001: unresolved external symbol "_sin"
phoon.obj : error LNK2001: unresolved external symbol "_sqrt"
astro.obj : error LNK2001: unresolved external symbol "_sqrt"
astro.obj : error LNK2001: unresolved external symbol "_tan"
phoon.exe : error LNK1120: 8 unresolved externals

Which is not surprising as there is no FPU/Floating point math support in 196. I tried the December 1991 Pre-Release, but it failed for other reasons:

I did copy over BASE.DLL BASERTL.DLL CSR.DLL DBGDLL.DLL as it wanted, but despite the symbol being in the DLL it didn’t load.

So that’s why I’m using the libraries from the Win32s SDK.

Okay, so far now we have GCC 1.40 compiling to an old Xenix GAS assembler, and linking with Microsoft link from Visual C++ 1.0/2.0 era. The next step is to see if we can just link the objects under 196, and get a running EXE!

I have this tiny fibonacci example program, so with it compiled & assembled by GCC & GAS, I did the final link under 196, and YES it runs!

I then built the ’87 InfoTaskForce, Infocom interpreter, and it was just a simple link, and it’s running!

Possible things to do? GCC should be able to build itself, so it should be possible to build GCC and link that on 196 or December 1991, and get a native version of GCC on NT. The other possibility is to get newer versions of GCC (cc1 drop in replacements) to build for Xenix and / or OS/2. Obviously this Xenix linker is the gateway to older 386 Microsoft based products!

For those interested in such things, I’ve uploaded all of this to archive.org here: windows-nt-196-linking-and-running-gcc

39 years ago, the dawn of home 32bit computing.

Clive on the QL announcement day

Much like the ZX Spectrum the Sinclair QL was a machine largely unknown to me growing up in Canada, then moving to the Miami area as a teenager. While the ZX 80/81 were pioneers in low end home computers barely able to do anything the ZX Spectrum with it’s 80kb of RAM (48k usable, as half the 64kb was defective), the QL announced in the start of 1984 was announced to the world as a very serious business machine. No gaming around to be found here!

While the IBM PC had been released in 1981 with it’s 16bit Intel 8088 processor utilizing a much slower & cheaper 8bit bus, the QL went one step further utilizing the Motorola 68008, a hybrid 16bit processor with 32bit registers, also using an external 8bit bus. However, being always on the cheap sifde, the QL only offered a single expansion slot, unlike the IBM PC. Also it eschewed floppy disks in favour of it’s endless loop ‘micro drive’ cassettes. Every corner that could be cut was, and sadly the resulted in a machine that just wasn’t ready as one has to wonder if the word of the Macintosh launch the following week was out, and Clive knew that it was either announce it now, or be a meetoo going forward. For better or worse he launched.

On paper it sounds fantastic, 128kb of ram, 32bit capable processor, and 2 drives all for £399! The IBM PC was an eye watering £3,325 by comparison, while the similarly spec’d Macintosh was £2,698! Indeed the QL stood for Quantum Leap, as the jump from 8bit to 32bit home computing was going to be phenomenal.

The avid jogger, Clive taking his Quantum Leap.

But how could it all go so wrong? Within a year the price had been slashed to £199, and stores were said to be further marking them down to a mere £99 the year afterwards. How could a seemingly on par machine fail so badly? The 128k Macintosh also was limited to a paultry 128kb on it’s motherboard, while the more expensive, and expandable IBM PC/XT maxed out at infamous 640kb, and it supported up to two floppy drives, and 2 hard disks although the IBM AT would be announced later that year, and it could go well beyond 640Kb, but the lack of protected mode operating systems & software would hinder the platform for quite some time.

If the trades are to be believed it was a combination of announcing too early, and failing to deliver burnt people on the QL. Additionally the ZX Spectrum had been busy wining apps at the time (games), but Sinclair wanted so much to be a serious company, not the man who brought you jet-set “fucking” willy. Sound and video capabilities of the QL were no match for the Spectrum, just weren’t there, and also missing was the incredibly cheap European storage of choice the audio casette. Many people were also dismayed that the operating system was much larger than expected and it needed to occupy both internal ROM sockets, and the cartridge port. And of course, the microdrives themselves were seen as easy to corrupt, stretch and tear. Not the kind of thing someone in business wants to hear. The ironic thing about the QL was that in my opinion it was too cheap. The PC/XT offered plenty of expansion at the base price (albeit a high one), and Apple also quickly added a much more realistic 512kb model Macintosh. The QL never got a ‘big brother’, basically condemning it at launch as nothing more but a toy. Which is a shame.

Hello!

The operating system, burnt into ROM feels kind of 8bit as it has a basic interpreter built in, and it’ll open several hard coded windows in which you are expected to interact with. However, it feels more like a minicomputer with input on the bottom, running lists on the left, and output on the right. Indeed, it can feel outright baffling. And certainly nothing like an 8bit machine, or like the later home 32bit machines like the Amiga, or Mac. Even the TOS based Atari ST felt more ready for the world with its GEM burned into ROM.

Building my dream system

Despite all of these downsides, I was still intrigued by the machine, and I have to admit I really love the look of it. When I’d first read Neuromancer around the time of the video game, and this is what I’d imagined a cyberdeck to look like. While I was wasting my youth with an 8bit machine I wanted to experience this seemingly parallel universe where affordable 32bit micros were a thing.

My QL getting ready for surgery

Since I’ve heard of the machine, I’ve been trying to get one. Surprisingly for such an unloved machine they are incredibly hard to find, and they do go for quite a bit of money. However, thanks to making contacts on the QL User’s forum, I had managed to get my hands on one, so I could start my journey. I sent it off to RudeDog Retros, luckily located across the bridge from where I’m staying and within a week. I was able to get back a working system. I also had managed to get a tetroid ram expansion/CF card addon as well for my QL bringing it up to 880kb of RAM, and an 8MB CF card, making the machine a top of the line experience. Although the card was given to me with known issues, for the most part it worked, except when it didn’t, and it was always the same, bad ram at the 128kb boundary. Which is a shame, having mass storage certainly gets around the microdrives, but 128kb of ram just isn’t enough when factoring in mass storage.

Fully loaded!

Much like other 68000 based machines, the Sinclair QL is no stranger to hardware modifications. I ordered a 512kb memory expansion unit (Sinclair QL 512KB Internal Memory Expansion from *der_englaender*) to fill in the 128k gap from the Tetroid card (plugged into the left). The red 68008 socket is a real pain to deal with, and it took an unreasonable amount of force to get the RAM expansion into the QL. No doubt nearly 40 year old sockets are just not that flexible. I had thought the board was in, but I was getting a buzzing black screen, it only took a bit of reassuring and swapping in another 68008 to verify the machine was fine, and more pressing to get it to seat into the socket.

Sinclair QL effectively with no CPU!

You can also spot the Hermes 2.20 co-processor upgrade to support 19,200 baud serial operations, along with the Minerva OS upgrade, and you can see the numerous RAM chips that had to get replaced to make the unit functional. With an appropriately upgraded system it can begin to feel like a real machine.

Thanks to the folks over at The Sinclair QL Forum, I was able to complete the upgrade!

With enough luck my machine now has reliable memory, mass storage, and faster communications with the outside world. Everything you’d want in a modern computer! I now have a capable machine to do the one thing I always worry about when doing cross compiling, actual hardware verification.

SIxteen/Thirtytwo into EIght dreams…

The 68000 was used by Stanford University in it’s project based motherboards, that gave way to the 68010 based SUN-1/SUN-2 based machines, where other companys also used the 68000 line of processors in their Unix based machines. While the QL with it’s cost conscious 68008 was not going to run a ‘real unix’, it was however capable of running real programs. The big AT&T compilers (PCC!) can target the processor, just as other new and upcoming compilers, even GCC, although even 880kb of ram is probably not enough for running GCC natively. But going back to 1985 this leaves room for something more restricted to mini-computers, Hack.

While Hack had been ported to 16bit machines like the IBM-PC, or sixteen/32bit machines like the Amiga, I was surprised to find that it had not appeared on the QL. For someone like me that first meant getting a cross compiler in place to target the QL. Thankfully xXorAa had done a lot of the hard work in xtc68, qdos-libc, and Dilwyn Jones had saved the GCC patches by Richard Zidlicky, Jonathan Hudson, Thierry Godefroy and Dave Walker. With a working cross compiler, time to get hacking!

From the mini-computer to the 32bit home micro-computer

Doing my thing I put together a simple cross compiler so that I could begin work. With enough RAM, the QL is in the surprising league of many a minicomputer of the early 1980’s. It’s incredible to think of what a missed opportunity this is. When I had decided to try to get Hack-1.03 up and running, I went for the Unix version, bypassing the probably more apt PC port, as despite Hack-1.03 being free enough to be still part of OpenBSD, PC Hack however has a more restrictive license. I know it’s weird.

Using a VT library that gives the QL, something akin to ANSI.SYS functionality and another library to set environment variables (yes QDOS doesn’t have either…) a simple basic program to setup the screen type and point Hack where to find it’s files, and we are suddenly off to the races. I didn’t have to restrict or cut anything down, it’s running the same code that effectively would run on a VAX-11/780 mini-computer, or a SUN-2 workstation. It’s crazy how this machine didn’t fill the home 32bit gap that took Microsoft/Intel quite a few years to fill.

Hack 1.03 on the QL

For those into the javascript, you can download a disk image: hack.win here, and test drive it on ppe/xXorAa‘s emulator here sQLux – MIN198.rom, 4096K (qlforum.co.uk)

Another great program, COM, the 8080 CP/M emulator can also happily run on the QL, again with libvt, emulating a vt52 it opens up an entire ecosystem of software, much like it did on the Commodore-128. If it had been available perhaps it would have greatly helped out things for the platform as it languished.

While the QL was marred with it’s too early announcement/pre-orders, and terrible primary storage medium, and far too restrictive motherboard design, there was great potential in that tiny little machine. I’d like to have thought if I’d known about the QL, I’d have bought one, even though the 128kb is super restrictive, thanks to it’s processor it really was an incredible machine for 1984.

In the days of cheap arm machines, and who knows what the downstream effects of those will be, it feels like there was a much earlier missed window with the Sinclair QL.

When YYLEX shouldn’t be confused with yylex

While trying to build some old stuff I ran into this weird issue where YYLESX is undefined:

gcc -DHAVE_CONFIG_H -I. -I. -I../../include -I../.. -I../.. -I../../include -g -O2 -Wall -c cs_grammar.c
cs_grammar.y: In function ‘cs_parse’:
cs_grammar.y:1074:58: error: ‘YYLEX’ undeclared (first use in this function)
1074 | yychar = YYLEX;
| ^~~~~
cs_grammar.y:1074:58: note: each undeclared identifier is reported only once for each function it appears in
make[3]: *** [Makefile:334: cs_grammar.o] Error 1

Notice it’s all UPPERCASE, but you can find plenty on the lowercase issue where its not being linked correctly. And yeah turns out YYFLEX should define out toe yyflex

# ifndef YYLEX
#  define YYLEX		yylex()
# endif

Add that in the top of whatever source is complaining and you’re golden.

Really.

nonsensical benchmarks

So I was messing with quake1 & DJGPP/DOSBox. So yes that means this table is largely nonsense. My larger goal was to see if a strictly softfloat could run Quake1. The answer, is no.

However I got some weird answers from messing around with the flags & fps from a timedemo of demo1

FPS
67.7
113.4
130.1

131.9
101.3

73.0
71.7
31.1

44.6
32.7
CFLAGS
-O0 -m386 -m80387
-O2 -m486 -m80387
-O2 -m486 -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387
-O2 -m386 -m80387
-O2 -m386 -m80387 -funroll-loops -fomit-frame-pointer -fexpensive-optimizations
-O2 -m486
-O2
-O2 -msoft-float -m386 -funroll-loops -fomit-frame-pointer -fexpensive-optimizations
-O2 -msoft-float -m386
-O2 -msoft-float -m486

This needs to be a table! it’s unreadable!

So surprisingly -O2 -m386 -m80387 produced the fastest code using GCC 2.7.2.3. On DOSBox so yeah that means literally nothing. Rebuilding DOSBox with no floating support code gave a weird error about the pov being out of range.

Obviously the next thing to do is run this stuff natively.. .which means GCC 2.7.2.3 for NT. Oh this is going to be fun, but utterly pointless. Or maybe not.

I re-ran the tests using VMware. There is no audio drivers involved just plain MS-DOS. The red is DOS-Box while green is VMware for the graph with FPS being the measurement. Interesting how the numbers aren’t as varied like DOSBox, however the -m386 -m80387 proved the be the worst for VMware, while the 386 soft float was so incredibly slow on DOSBox but performs great on VMware. yay?

32016 stand alone planetfall!

InfoTaskForce’87 running on a simple NS32016 emulator

What is it?

It sure may not look like much but it was an adventure getting here.

First, what is it? Well it’s the very simple NS32016 from here, with a few minor changes. I expanded the RAM from 256kb to a whopping 8MB. Then I added simple character I/O allowing me to print messages to the screen. Next looking at the toolchain page, I used my old Linux to Windows GCC 4 cross compiler to build the appropriate Canadian cross compiler to the NS3216.

Building the tools

A while back, I had built a cross compiler from Linux to Windows using GCC 4.1 as the basis as it was the last version that didn’t have massive external dependencies. NS32016 support was dropped some time in the late 3.x or early 4.x GCC so it means we need to go old anyways. I arbitrary picked GCC 2.8.1 for this build, while using the recommended Binutils 2.27

I cheated and just downloaded my existing linux-minw32.7z cross compiler as I didn’t feel like rebuilding everything again, although it is all in the Building a MIPS Compiler for Windows on a Linux VM! article. I also used an old Linux to Linux i586 32bit compiler (back from the OSKit build!) although you can use your hosts as well.

configuring Binutils is pretty simple like this:

./configure --prefix=/cross --target=ns32k-pc532-netbsd --host=i686-mingw32 --build=i586-linux

You can try omitting the –build portion, Debian GNU Linux 10 seemed okay with Gcc 8 as the default system compiler.

configuring GCC 2.8.1 was pretty similar:

./configure --target=ns32k-pc532-netbsd --prefix=/cross --disable-libssp --build=i586-linux --host=i686-mingw32

GCC 2.8.1 doesn’t quite know what we are doing so there is some flags we need to run off in auto-config.h namely

  • #define HAVE_BCMP 1
  • #define HAVE_BCOPY 1
  • #define HAVE_BZERO 1
  • #define HAVE_INDEX 1
  • #define HAVE_KILL 1
  • #define HAVE_RINDEX 1
  • #define HAVE_SYS_RESOURCE_H 1
  • #define HAVE_SYS_TIMES_H 1

You can just comment them out, or remove those lines all together.

When it came to building GCC, I did run into issues with GCC 7/8 trying to build GCC 2.8.1. I found it much easier to either have that Linux 4.1 compiler, or if you have access to Wine or WSL you can just run the Win32 binaries for the gen phases.

./configure --prefix=/cross --target=ns32k-pc532-netbsd --host=i686-mingw32
make CC=i686-mingw32-gcc xgcc cccp cc1 cc1obj

If you can run your own Win32 exe’s on Linux it’ll run just fine using the Linux to Windows GCC 4 cross. Otherwise you will need to either patch GCC or make your own GCC 4 hosted Linux to Linux cross compiler like this:

make CC=i686-mingw32-gcc HOST_CC=i586-linux-gcc xgcc cccp cc1 cc1obj

Hopefully that worked enough, and now you have your cross compiler. Now it’s time to build libgcc1.a

cp cccp cpp.exe
cp cc1 cc1.exe
cp xgcc xgcc.exe
cp ../binutils-2.27/gas/.libs/as-new.exe as.exe
cp ../binutils-2.27/binutils/.libs/ar.exe ar
cp ../binutils-2.27/binutils/.libs/ranlib.exe ranlib
make libgcc1.a TARGET_TOOLPREFIX="./" OLDCC=./xgcc.exe

Again you really want to be able to run the resulting programs on Linux but I guess you could script it out. Naturally if you wanted to just use Linux, it’d be easier to make that cross compiler directly, although I’m not sure how much of GCC 2.8.1 I want to fight, or just get GCC 4 running on Linux and use that to port.

crt0, somewhere for C to start

As mentioned a crt0.s is missing but there was enough inspiration to come up with this:

#NO_APP
gcc_compiled.:
.text
        .align 1
.globl _start
_start:
        enter [],0
#APP
#       setting the stack 256k under 8MB
        lprd sp,0x7c0000
        jsr _main
#NO_APP
L1:
        exit []
#       setting the stack 256k under 8MB
        lprd sp,0x7c0000
        bpt
        .align 1

#does nothing
.globl ___main
___main:
        ret 0

.globl _exit
_exit:
        bpt
        ret 0

I used a bit of the C example, and added some hooks that GCC was expecting namely a __main call that is made from main before it does anything (a place to init memory perhaps?), a place to catch an explicit exit call, along with setting the stack of course.

Patching InfoTaskForce without malloc / disk access

It’s not going to win any awards, but it was really great to get it to run a simple program written in GCC. Looking for something more fun, I took the old InfoTaskForce interpreter from ’87, and dug up my modification to run on cisco routers, and cooked up this version, that adds enough of printf from Linux, a bogus malloc that just allocates from a fixed memory array (otherwise you have to actually know about your platform), and a fun trick with later binutils where you can import a binary file directly as an object!

Neat!

Since I don’t have any file I/O being able to have the game data in RAM is crucial. I tried to tweak it so you could build the same working thing on Windows (maybe others?).

So for anyone who wants to look at the standalone adventure Win32 hosted tools are here, although the emulator should be somewhat portable.

GCC 2.5.8 failure on 32bit ARM

This is really nothing more than a placeholder for me… Unless someone else knows the answer, then it’s really ‘how not to cross compile GCC’.

First I’m using the EMX’ified version of GCC from my MinGW to EMX cross. It didn’t require that much massaging to get it to build, the usual unzip as ascii to convert text, and in no time I can build cc1.


root@pinepro:/src/emx/src/gcc-2.5.8# file cc1
cc1: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=42c0c8de7175edade7614dc92d5d13e4421e0e6f, with debug_info, not stripped

and it crashes in what has to be a 2020 most unfortunte name

Reading symbols from cc1...done.
(gdb) r
Starting program: /src/emx/src/gcc-2.5.8/cc1 

Program received signal SIGSEGV, Segmentation fault.
0x004f6b84 in rtx_cost (x=<error reading variable: Cannot access memory at address 0xff7efff0>, 
    outer_code=<error reading variable: Cannot access memory at address 0xff7effec>) at cse.c:667
667     {
(gdb) 

Yes, it really crashes in rtx_cost. Good thing there isn’t a super popular card from Nvidia that is currently being short squeezed by crypto miners right now called the RTX where everyone is looking for a good price. 😐

I had then been thinking perhaps it’s because I’m using GCC 8.3.0, maybe it’s introducing some new and exciting bug? So I cross compiled GCC 4.1.2 as follows:

./configure --target=armeb-linux --host=armeb-linux --build=armeb-linux

Keeping in mind that my knowledge of ARM is pretty much nill, especially on Linux. The compile went mostly okay, just have to remember the gnu inline macro’s as needed from back in the day (-fgnu89-inline) and while it builds, it is insisting on using collect2 which of course is screwing things up. And of course I don’t want it as my system compiler. As a hack I found system gcc 8 can link things fine as I didn’t want to spend all day messing with GCC/collect2

I copied xgcc, cc1 and cpp from 4.1.2 into a /412 directory, and rebuilt 2.5.8 with the following shell:

make CC="/412/xgcc -B/412 -g -O0  -I. \
-I./config \
-I/usr/lib/gcc/arm-linux-gnueabihf/8/include \
-I/usr/local/include \
-I/usr/lib/gcc/arm-linux-gnueabihf/8/include-fixed \
-I/usr/include/arm-linux-gnueabihf \
-I/usr/include" cc1

As you can see the cross wasn’t picking up the right include paths, so I just cheated, and dumped them from 8, and just copied them into this script. I re-ran the build and had 2 issues,

/412/xgcc -B/412 -g -O0  -I. -I./config -I/usr/lib/gcc/arm-linux-gnueabihf/8/include -I/usr/local/include -I/usr/lib/gcc/arm-linux-gnueabihf/8/include-fixed -I/usr/include/arm-linux-gnueabihf -I/usr/include -c  -DIN_GCC   -g -std=gnu89     -I. -I. -I./config local-al.c

....

/tmp/ccMguyhs.s: Assembler messages:
/tmp/ccMguyhs.s:5001: Error: selected processor does not support `fltd f1,r3' in ARM mode
/tmp/ccMguyhs.s:5025: Error: selected processor does not support `fltd f0,r3' in ARM mode
/tmp/ccMguyhs.s:5026: Error: selected processor does not support `dvfd f1,f1,f0' in ARM mode
/tmp/ccMguyhs.s:5027: Error: selected processor does not support `ldfd f0,.L489' in ARM mode
/tmp/ccMguyhs.s:5028: Error: selected processor does not support `mufd f0,f1,f0' in ARM mode

and so on. Also failing was global.c Again the same weird instruction/asm mix being triggered. Other than those two, cc1 will build, but unsurprisingly:

Reading symbols from cc1...done.
(gdb) r
Starting program: /src/emx/src/gcc-2.5.8/cc1 

Program received signal SIGSEGV, Segmentation fault.
0x004f6b84 in rtx_cost (x=<error reading variable: Cannot access memory at address 0xff7efff0>, 
    outer_code=<error reading variable: Cannot access memory at address 0xff7effec>) at cse.c:667
667     {
(gdb) 

Well, at least it’s consistent?

Or a fun way to kill a couple hours.

**EDIT I went ahead and looked in the 4.1 source for ARM stuff..

root@pinepro:/src/gcc-4.1.2# grep arm config*|grep linux
grep: config: Is a directory
configure:  arm*-*-linux-gnueabi)
configure.in:  arm*-*-linux-gnueabi)

it didn’t like the gnueabihf stuff one bit.

I tried to rebuild as linux-gnueabi

./configure --target=arm-linux-gnueabi --host=arm-linux-gnueabi --build=arm-linux-gnueabi

make LANGUAGES=c HOST_CFLAGS='-fgnu89-inline' CFLAGS='-fgnu89-inline'

And then re-built GCC 2.5.8 with the same error, but slightly further into the program:

Starting program: /src/emx/src/gcc-2.5.8/cc1

Program received signal SIGSEGV, Segmentation fault.
0x004f2a20 in rtx_cost (x=0x41, outer_code=PLUS) at cse.c:679
679       code = GET_CODE (x);
(gdb) bt
#0  0x004f2a20 in rtx_cost (x=0x41, outer_code=PLUS) at cse.c:679
#1  0x004f2e20 in rtx_cost (x=0x60c3f8, outer_code=SET) at cse.c:736
#2  0x004ac2dc in init_expmed () at expmed.c:87
#3  0x0045ae28 in compile_file (name=0x5c96ec "stdin") at toplev.c:1648
#4  0x0045f6fc in main (argc=1, argv=0xfffefd04, envp=0xfffefd0c) at toplev.c:3569
(gdb)

The positive thing is that there was no weird register errors while compiling, and it built 100% normally…? “arm-linux-gnueabihf” almost seems right, specs needs fixing to point to “/lib/ld-linux-armhf.so.3” instead of “/lib/ld-linux.so.3” along with the linker target.