A while back I had made a small post about getting Graylog running on Windows. It was fun, as it’s just JAVA so you know it should be portable, and other than some weird disk access thing it does seem to run fine.
Of course, the next step is to create a dashboard to replace what I used on wp-statistics, as it was crashing taking up 100% of my CPU, and exceeding PHP’s 12GB of RAM per process limit. You know things are messed up when I’m replacing you with not one, but 2 Java apps! (Graylog & Opensearch).
magical dashboard
It’s by no means perfect, but the guide How to se -up graylog geoip configuration, is all around great to have. The rest of it is me learning how to do aggregate searches, and simple lists, to see latest hits, 404’s and count the pages and build a graph.
Again, this is all good, now for the real question, how to get this onto the Internet?!
The firs thing to do is enable cors.. It’s for being on the internet!
http_enable_cors = true
Next enable the external URI name
http_external_uri = https://dash.board.com/
And now the changes I had to make in my haproxy config
I kind of wish I saved the logs while going crazy but YES for some reason it’ll try to reference itself as /api/api. I don’t know why, so I had to do some uri regex to fix that. Neat!
Next for some reason Graylog responds that all .js (javascript) files are actually text. Chrome doesn’t allow that to work, so yes you need to set the content type header to “application/javascript” for Chrome to be happy.
I had wasted over an hour with this and couldn’t get it working. So, I walked away for a few hours, and it suddenly was working. I think Cloudflare was doing some caching against it.
This is probably too terse to be really useful, and I lost all the pages I was reading about setting stuff in haproxy as I was doing that incognito. Oops. I picked this config out of fragments from five other people’s stuff. There is other considerations to host it on a subdirectory of a public site, but I just wanted to K.I.S.S.
Sydney’s idea of what Visual C++ for OS/2 should look like
No, it’s not a typo.
This is a long-winded post, but the short version is that I found a working combination to get the C compiler from Visual Studio 2003 targeting OS/2.
Once I’d learned how C compilers are a collection of programs working in concert, I’d always wanted to force Microsoft C to work in that fashion, however it is born to be a compiler that integrates everything but linking. There has been a “/Fa” or output assembly option, but I’ve never gotten it to do anything useful. I’m not that much into assembly but it seemed insurmountable.
But for some reason this time things were different.
This time I used:
Microsoft (R) Macro Assembler Version 6.11
After the great divorce and the rise of Windows NT, Microsoft had shifted from the OMF format to COFF. However somewhere buried in their old tools it still supports it, namely MASM. For example, if I try to run LINK386 (the OS/2 Linker) against output from Visual C++ 2003 I get his
However if I output to assembly and then have MASM assemble that, and try the linker, I’m bombarded with errors like this:
warp.obj(warp.asm) : error L2025: __real@4059000000000000 : symbol defined more than once
warp.obj(warp.asm) : error L2029: '__ftol2' : unresolved external
If I was smart I’d have given up, there is pages and pages of this stuff. But I’m not smart, so instead I decided to something different, and use SED, the stream editor, and try to patch out the errors.
The ftol2 call is for newer CPU’s and any OS/2 library won’t have it. But instead of binary editing symbols we can replace the ftol2 with ftol with this simple line:
sed -e 's/_ftol2/_ftol/g'
For some reason Visual C++ likes to make all it’s reals “public” meaning there can only be one, but yet there is so many. Why not comment them all out?
sed -e 's/PUBLIC\t__real@/;PUBLIC\t__real@/g'
And there are various other annoying things, but again they can be all patched out. Just as the older Windows 1991 Pre-release compilers also have weird syntax that MASM doesn’t understand.
astro.asm(59): error A2138: invalid data initializer
which goes into how Microsoft C used to initialize floating point constants:
This one was a little hard for me as I’m not a sed expert, but I did figure out how to mark the section, and then to replace it
sed -e "s/DQ\t[0-9a-f]r/&XMMMMMMX/g" $.a1 | sed -e "s/rXMMMMMMX/H/g"
And so on. At the moment my ‘mangle’ script is now this:
.c.obj:
$(CC) $(INC) $(OPT) $(DEBUG) /c /Fa$*.a $*.c
wsl sed -e 's/FLAT://g' $*.a > $*.a1
wsl sed -e "s/DQ\t[0-9a-f]*r/&XMMMMMMX/g" $*.a1 \
| wsl sed -e "s/rXMMMMMMX/H/g" \
| wsl sed -e 's/call \t\[/call DWORD PTR\[/g' \
| wsl sed -e 's/PUBLIC\t__real@/;PUBLIC\t__real@/g' \
| wsl sed -e 's/_ftol2/_ftol/g' > $*.asm
ml /c $*.asm
del $*.a $*.a1 $*.asm
This allows me to plug it into a Makefile, so I only have to edit it in one place.
Not surprisingly, this allows the LINK from Visual C++ 1.0 to link the MASM generated object files and get a native Win32 executable. Even from the oldest compiler I have from the Microsoft OS/2 2.00 Beta 2 SDK from 1989!
But now that we have the C compilers being able to output to something we can edit and force into a Win32, there is a few more things and suddenly:
C:\cl386-research\bin.10.6030\cl386 /u /w /G3 /O /c /Faphoon.a phoon.c
C:\cl386-research\bin.10.6030\CL386.EXE: warning: invoking C:\cl386-research\bin.10.6030\CL.EXE
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.6030 for 80x86
Copyright (C) Microsoft Corporation 1984-2002. All rights reserved.
phoon.c
wsl sed -e 's/FLAT://g' phoon.a > phoon.a1
wsl sed -e "s/DQ\t[0-9a-f]*r/&XMMMMMMX/g" phoon.a1 | wsl sed -e "s/rXMMMMMMX/H/g" | wsl sed -e 's/call \t\[/call DWORD PTR\[/g' | wsl sed -e 's/PUBLIC\t__real@/;PUBLIC\t__real@/g' | wsl sed -e 's/_ftol2/_ftol/g' > phoon.asm
ml /c phoon.asm
Microsoft (R) Macro Assembler Version 6.11
Copyright (C) Microsoft Corp 1981-1993. All rights reserved.
Assembling: phoon.asm
del phoon.a phoon.a1 phoon.asm
msdos286 run286 C:\cl386-research\bin\ddk12\LINK386.EXE @phoon.lnk
Operating System/2 Linear Executable Linker
Version 2.01.012 Nov 02 1993
Copyright (C) IBM Corporation 1988-1993.
Copyright (C) Microsoft Corp. 1988-1993.
All rights reserved.
Object Modules [.obj]: astro.obj date_p.obj phoon.obj
Run File [astro.exe]: phoon2 /NOE /NOI /NOD:OLDNAMES
List File [nul.map]: nul.map
Libraries [.lib]: ..\..\lib2\libc.lib +
Libraries [.lib]: ..\..\lib2\os2386.lib
Definitions File [nul.def]: nul.def;
LINK386 : warning L4071: application type not specified; assuming WINDOWCOMPAT
I know it’s a bit of a word salad, but the key thing here is that using Visual C++ 2003’s compiler (version 13.10.6030), and outputting to assembly that we can edit, we can then use MASM to build objects that surprisingly LINK386 version 2.01.012 will link with. I suspect this has to do with device drivers, and probably the majority of the OS/2 operating system.
Anways, we’ve done the incredible, using the same object files, we made both a Win32 application, and an OS/2 application!
phoon-13.10.6030.exe: PE32 executable (console) Intel 80386, for MS Windows phoon2.exe: MS-DOS executable, LX for OS/2 (console) i80386
Phoon compiled by Visual C++ 2003 on OS/2 2.00
Incidentally Happy CNY!
Obviously, this is VERY cool stuff.
I know the next question is do we have to rely on a 16bit linker? How about Watcom?
C:\cl386-research\proj\trek>wlink @trek.wlk
WATCOM Linker Version 10.0
Copyright by WATCOM International Corp. 1985, 1994. All rights reserved.
WATCOM is a trademark of WATCOM International Corp.
loading object files
searching libraries
Warning(1008): cannot open LIBC.lib : No such file or directory
Warning(1008): cannot open OLDNAMES.lib : No such file or directory
creating an OS/2 32-bit executable
Ignore the warnings and YESwe can Link from something much newer & 32bit! In this example I linked the old TREK game, also built with Visual C++ 2003. The response file looks lke:
It’s probably needing additional stack space, maybe some other stuff, or resources, maybe how to flag it’s windowing compatible.
TREK built by Visual C++ 2003, and Linked using Watcom C/C++ 10.0
How do I get started, if I dare?! First download and unpack cl386-research-v2. Ideally on the root of your C: drive, because why not?
run the ‘env’ command to set your environment up. Its pretty complicated but in the proj directly there is currently:
*NOTE that I do use SED scripts, I have it set to use Linux in the WSL package. I tried some Win32 sed but it didn’t work. So you need WSL or a working sed!
and it’ll compile populate a floppy and launch the emulator
Its all good fun.
Read the Makefiles to configure a compiler, how to run it, and if you need to mangle the assembly. The 32bit new stuff needs to be mangled, the older stuff almost always works with just compile.
# Version 6.00.054 1989
# https://archive.org/details/os-2-cd-rom_202401
PLATFORM = ddksrc
In this case it’ll select the platform from the ‘ddksdk’ release. The next is if the compiler is OS/2 based or native win32. Basically 73g / windows 95 & below are native Win32.
In the above example we comment out the dos extended cross
# dos exteded cross
CC = $(EMU) $(DOSX) $(CL386ROOT)$(PLATFORM)\cl386
# native CC
# CC = $(CL386ROOT)$(PLATFORM)\cl386
Next is the mangle strategy. In this case it’s an ancient OS/2 (like) compile so try un commenting the ‘just compile’ line
# must include ONLY ONE strategey..
# for OS/2 it must have been assembled my MASM 6.11
include ..\-justcompile.mak
#include ..\-mangleassembly.mak
#include ..\-plainassembly.mak
save the makefile, and run
nmake os2
You can just close the emulator as after each run it’ll unpack a hard disk image, so nothing will be lost. or saved. It’s just for testing. You may need to periodically clean the floppy drive, as that is the only way to transfer stuff in and out of the VM.
What versions of CL386 have I found? Well, it’s quite a few, although I know I’m missing quite a few.
== c386 ============================
Microsoft C 5 386 Compiler
Microsoft C 5.2 286/386 Compiler -- Driver
@(#)C Compiler Apr 19 1990 11:48:30
Copyright (c) Microsoft Corp
1984-1989. All rights reserved.
(press <return> to continue)
Microsoft 386 C Compiler. Version 1.00.075
Quick C Compiler Version 2.00.000
1.00.075
== ddk12 ============================
C 6.00 (Alpha) Aug 24 1990 19:12:31
Copyright (c) Microsoft Corp
1984-1989. All rights reserved.
(press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.054
Quick C Compiler Version 2.00.000
6.00.054
== ddk20 ============================
C 6.00 (Alpha) Aug 16 1990 23:04:06
Copyright (c) Microsoft Corp
1984-1989. All rights reserved.
(press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.054
Quick C Compiler Version 2.00.000
6.00.054
== ddksrc ============================
C 6.00 (Alpha) Aug 24 1990 19:21:49
Copyright (c) Microsoft Corp
1984-1989. All rights reserved.
(press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.054
Quick C Compiler Version 2.00.000
6.00.054
== nt-sep ============================
@(#)C Compiler 6.00 Feb 06 1991 17:15:19
@(#)C Compiler 6.00 May 13 1991 23:54:12
@(#)C Compiler 6.00 Jun 03 1991 15:16:22
Copyright (c) Microsoft Corp
1984-1991. All rights reserved.
(press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.077
Quick C Compiler Version 2.00.000
6.00.077
== nt-oct ============================
@(#)C Compiler 6.00 Jun 03 1991 15:16:22
@(#)C Compiler 6.00 Jun 13 1991 22:07:23
@(#)C Compiler 6.00 Oct 10 1991 00:42:24
Copyright (c) Microsoft Corp
1984-1991. All rights reserved.
(press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.080
Quick C Compiler Version 2.00.000
6.00.080
== nt-dec ============================
@(#)C Compiler 6.00 Jun 03 1991 15:16:22
@(#)C Compiler 6.00 Jun 13 1991 22:07:23
@(#)C Compiler 6.00 Oct 10 1991 00:42:24
Copyright (c) Microsoft Corp
1984-1991. All rights reserved.
(press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.081
Quick C Compiler Version 2.00.000
6.00.081
== 73g ============================
1984-1993. All rights reserved.
Copyright (c) Microsoft Corp
8.00.3200
32-bit C/C++ Optimizing Compiler Version
Microsoft (R)
== msvc32s ============================
Microsoft 8.00.0000 - Copyright (C) 1986-1993 Microsoft Corp.
Microsoft 8.00.0000 - Copyright (C) 1986-1993 Microsoft Corp.
@(#) Microsoft C/C++ 32 bits x86 Compiler Version 8.00.XXXX
8.00.000
== 13.10.6030 ============================
Microsoft (R) C/C++ Compiler Version 13.10.6030
From my install of Visual Studio 2003 Enterprise
As you can see many of these earlier OS/2 compilers report the same versions but are in fact different builds on the inside. I suspect Microsoft had to support one version, and an Alpha version of version 6 is as good as it got. I would have imagined there were internal 32bit versions of 6 or 7, but I haven’t seen them.
Compiling and running TREK
Hopefully this gives some idea of how I tried to made a probably too modular build system to try all kinds of different compilers. I might have to see if it’s possible to run the tools from the 1992 versions of Windows NT in this setup, perhaps they are interesting as well.
One thing in my porting GCC to OS/2 experience is that the usability of the C compilers from 1991 were dramatically better than what Microsoft had given IBM at the time of the divorce. No doubt the upcoming NTOS/2 project was placing a bigger demand on the tools team.
If anyone has any access to other ‘cl386’ compilers, or early OS/2 2.00 stuff, please let me know! I’d love to do build/tests and see if my idea of distributing objects ‘just works’!
I know what you are going to think, that it was already done, and it was called EMX. Or was it GCC/2? Well sure but what if you are not running the GA (General availability) version of OS/2. For example, years ago I had managed to get Citrix Multiuser 2.0, and it’s not at the GA level. All that is available is some ancient beta version of Microsoft C 5.2 from 1989?!
A little while back I had worked on getting GCC to build and run on the FPU enabled versions of Windows NT from 1991. I had mentioned that it turns out thanks to the Xenix assembler, that GCC had been basically available the entire time Windows NT had been available, but lamented that since the OS/2 compiler is 16/32bit, the 5.2 compiler couldn’t handle compiling GCC without blowing it’s heap. 16bit issues in a 64bit world.
However after doing some research on all the early cl386 compilers I could get my hands on, including the Windows NT Pre-release ones, I’d noticed that if I built CC1.EXE (the actual compiler) first for Win32, then rebuilt those object files with the December NT Pre-release compiler, that some versions of LINK386 from the OS/2 2.1 DDK would actually link with them. And sure enough it worked!
First life of GCC on OS/2
I have to admit I was pretty amazed, I had managed to ‘cross compile’ GCC using quite the tool chain.
First the compiler from the December NT Pre-release CD-ROM is shipped as a 16bit OS/2 compiler, but I’m using Windows 11. First I use the MS-DOS player with a quick fix from crazyc to allow Phar Lap 286|Dos Extender to run, which provides a basic enough OS/2 emulation to allow the compiler to run under ‘dos’. The linker on the DDK suffers the same fate as far as it also being 16bit. However the combination of MS-DOS player & Phar Lap gets stuff working! The only weird catch is that the 386 emulator causes strange floating point related crashes, while the 286 or 486 emulators work fine.
Now targeting OS/2 or running on OS/2 isn’t all that new, but building it from a Microsoft C compiler is. And now of course you’ll ask yourself, who cares? why is it interesting?
Well, the vast majority of the GCC ports to OS/2 don’t support the OMF object file binary standard, instead they used the much outdated a.out format, and rely on tools to convert the objects if needed. Additionally, they have DLL dependencies, and other startup issues with things needing to be setup. And of course they rely on a binary standard that is ‘GA’. *HOWEVER* by using a Microsoft compiler, I have OMF object files that the OS/2 built in system linker LINK386 can understand. So in plain English I can just relink the compiler and it’ll run on a new ‘version’ of unsupported OS/2.
I made a diskette image with my objects & a linker script and in a few moments I had it running!
GCC running on OS/2 2.00 6.123
The substantial thing here is that the binary format for OS/2 changed twice, and each release introduced changes that broke binary compatibility, in an effort to force people onto the new tools. So there is no way that the old ‘LE’ format would ever work. And you can see it’s running! In addition I could take the same object files, and copy them to my Citrix server, and likewise it was just a matter of linking, and it too now has GCC!
Converting a.out to OMF via emxomf
One annoying thing is that the LINK386 that ships with OS/2 2.00 GA doesn’t like the output of the Xenix assembler, so I built the a.out traditional assembler, and the emxomf tool to convert the a.out to OMF, and that worked well.
I still have much to mess with, including the pre-processor & main ‘gcc’ program. I have not built anything beyond a trivial program, so there is indeed much more work to be done before I can even try anything challenging. Some programs like emxomf have portions in the debug support that require the ‘long long’ type, which obviously Microsoft compilers from 1989-1991 don’t have, so I’ll have to re-build them with GCC.
Not that I can imagine anyone wanting to try but I’ve uploaded some disks with the objects. Copy them to a hard drive, and run the ‘build.cmd’ command and it’ll link to a native freestanding executable.
I’ll explain it more with a post later, along with going over all the versions of cl386 I’ve acquired, over the years in more of a part 2: Targeting OS/2 with Visual Studio 2003!
I wanted to get some research into some early space flight, and look into that magical transition when everything went from Cowboys & Indians to moonmen & the race to space. Granted Toy Story covers cultural touchstone pretty well, reading period pieces is fun too. I had a few CD-ROM’s containing the 1980’s National Geographic CD-ROM’s when they were sold up in decade sets, but what always escaped me was the fancy collectors box with the whole thing.
$169!!
I always figured this was going to be one of those weird collectors’ items that probably was under produced, over sold, and lost to the winds of time. Looking on eBay for a 1950’s and 1960’s set .. and thinking about the 1970’s as well, and it was going to get close to £40. Ouch. So for the heck of it, I look for the fancy box set.
wow
I was surprised for as much as I was going to end up paying for one or two sets, I could get the entire thing. In the legendary fancy wooden box. I got mine, shipped for under £20. Much wow!
I always was running mine on MacOS using Cockatrice III with the monitor resolution set to the absolutely absurd resolution of 1152×870 provided by the 1991 21″ monitor. I couldn’t imagine why these CD’s wouldn’t work. And of course the first step was to rip the CD-ROM’s. There was 31 National Geographic, and an additional clip art disc in my box. I fired up my XP machine to have it’s hard disk give up and die on me. Luckily since I had removed the mechanical disk from the iMac G5, I had a spare SATA disk handy. I didn’t feel like fighting the XP installer, and I’m impatient, so I made a netboot.xyz bootable flash drive, and installed Debian 10 over the internet. Very nice. Now I could get down to ripping CD’s. Luckily the drive from the DVD-RAM drive disaster reads at 48x, so it took me under 4 hours to rip them all. In case you are wondering:
32 File(s) 17,894,987,776 bytes
The flash drive I used is 32Gb. I got it in a 3 pack from Tesco. I think I paid £10 for it. That means UTZOO + NatGEO all fit one of the drives. I wonder if they’ll ever offer high resolution scans on a USB drive in a fancy wooden box?
For the hell of it, I used 7zip to decompress all the ISO’s and that’s when I noticed that although the files were spread over discs there was a clean decade break between volumes from the 1900’s, and that they had a logic to them.
On the NGS_1956_1959 CD-ROM there is a hierarchy something like this:
E:\IMAGES\256I
The 2 is for the 20th century, and 56 is the year. I is the month; in this case I is September. Since we live in the future, and rendering jpeg’s is quicker than real-time, 256IC01A.JPG can be shown to be the cover. The next pattern is for the adverts, 256IA02A.JPG – 256IA30A.JPG are the into adverts. Now, we get to the interior content 256I0287.JPG – 256I0426.JPG. Next is the closing/trailer advertisements 256IZ01Z.JPG – 256IZ13Z.JPG. And Finally the back of the issue, 256IB14Z.JPG is the rear of the magazine.
So we now have some understanding of the format. Putting this into order could be done with something simple like this:
Doing so makes a nice list file of what images should go in which order. I could probably use ffmpeg and painstakingly check the images for ‘pullouts/double wide’ ones, and have it stitch the rest together as a two pager ( ffmpeg -i left.jpg -i right.jpg -filter_complex hstack combined.jpg ), but that still sounds like a lot of work. Also the images are very low quality, It’s a shame they didn’t use black & white on the text, and scan the images separately, but that’d require something like PDF, and no doubt a LOT of time. Although Kodak did sponsor the set, the developer, Mindscape didn’t go with fancy PDF technology of the late 1990s, instead it’s just the blurry jpeg scans we have today.
NG July 1981
That’s when I found out about Tesseract. Running it against this one paragraph reveals:
Voyager is watching two small moons that-seem to be playing tag as they race around Saturn in almost the same orbit. The trailing moon is traveling faster than the leader, and should catch up with the leader in January 1982 (pages 20-21). The two pre- sumably have been playing this game for billions of years, Through what sleight of physics do they avoid colliding?
National Geographic, July 1981
I have to admit, that’s pretty good! And how amazing that I have a LOT of files to scan.
188553 File(s) 11,061,111,690 bytes
That is a LOT of files. Okay that’s nice, but can Tesseract read the list that I generated per issue? YES. The only thing that I’d love to see Tesseract do is create PDF’s with the scanned text embedded. OH WAIT, IT ALREADY DOES THAT!
How on earth did I not know this?
I put together a few scripts, and I was able to separate out all the images into years & months, then I created the needed list files with all the images in the correct order. It’s not a fast process I think it may take me a week or so to do this.
My CPU hates me
So far, I’m up to 1903, so I’ll update with some rough idea of when this finished.
So, the applications needed are old and obsolete Win16 or Classic MacOS needed machines, with an optical drive. Yes they still work (with emulation) on modern machines, although you still need to read the physical discs. Thankfully the images are easily mapped into the right order, and you can map them as your own. Neat!
Because I hate myself, I tried to get the Microsoft OS/2 Beta 2 SDK’s C compiler building simple stuff for text mode NT. Because, why not?!
Since the object files won’t link, we have to go in with assembly. And that of course doesn’t directly assemble, but it just needs a little hand holding:
Microsoft (R) Program Maintenance Utility Version 1.40
Copyright (c) Microsoft Corp 1988-93. All rights reserved.
cl386 /Ih /Ox /Zi /c /Fadhyrst.a dhyrst.c
Microsoft (R) Microsoft 386 C Compiler. Version 1.00.075
Copyright (c) Microsoft Corp 1984-1989. All rights reserved.
dhyrst.c
wsl sed -e 's/FLAT://g' dhyrst.a > dhyrst.a1
wsl sed -e "s/DQ\t[0-9a-f]*r/&XMMMMMMX/g" dhyrst.a1 | wsl sed -e "s/rXMMMMMMX/H/g" > dhyrst.asm
ml /c dhyrst.asm
Microsoft (R) Macro Assembler Version 6.11
Copyright (C) Microsoft Corp 1981-1993. All rights reserved.
Assembling: dhyrst.asm
del dhyrst.a dhyrst.a1 dhyrst.asm
link -debug:full -out:dhyrst.exe dhyrst.obj libc.lib
Microsoft (R) 32-Bit Executable Linker Version 1.00
Copyright (C) Microsoft Corp 1992-93. All rights reserved.
I use sed to remove the FLAT: directives which makes everything upset. Also there is some weird confusion on how to pad float constants and encode them.
MASM 6.11 is very update with this. I just padded it with more zeros, but it just hung. I suspect DQ isn’t the right size? I’m not 386 MASM junkie. I’m at least getting the assembler to shut-up but it doesn’t work right. I’ll have to look more into it.
Xenix 386 also includes an earlier version of Microsoft C / 386, and it formats the float like this:
So I had thought maybe if I replace the ‘r’ with a ‘H’ that might be enough? The only annoying thing about the Xenix compiler is that it was K&R so I spent a few minutes porting phoon to K&R, dumped the assembly and came up with this sed string to find the pattern, mark it, and replace it (Im not that good at this stuff)
wsl sed -e "s/DQ\t[0-9a-f]r/&XMMMMMMX/g" $.a1 \
| wsl sed -e "s/rXMMMMMMX/H/g" > $*.asm
While it compiles with no issues, and runs, it just hangs. I tried the transplanted Xenix assembly and it just hangs as well. Clearly there is something to do with how to use floats.
I then looked at whetstone, and after building it noticed this is the output compiling with Visual C++ 8.0
Great they look nothing alike. So something it totally broken. I guess the real question is, does it even work on OS/2?
Since I should post the NMAKE Makefile so I can remember how it can do custom steps so I can edit the intermediary files. Isn’t C fun?!
INC = /Ih
OPT = /Ox
DEBUG = /Zi
CC = cl386
OBJ = dhyrst.obj
.c.obj:
$(CC) $(INC) $(OPT) $(DEBUG) /c /Fa$*.a $*.c
wsl sed -e 's/FLAT://g' $*.a > $*.a1
wsl sed -e "s/DQ\t[0-9a-f]*r/&XMMMMMMX/g" $*.a1 \
| wsl sed -e "s/rXMMMMMMX/H/g" > $*.asm
ml /c $*.asm
del $*.a $*.a1 $*.asm
dhyrst.exe: $(OBJ)
link -debug:full -out:dhyrst.exe $(OBJ) libc.lib
clean:
del $(OBJ)
del dhyrst.exe
del *.asm *.a *.a1
As you can see, I’m using /Ox or maximum speed! So how does it compare?
Dhrystone(1.1) time for 180000000 passes = 20
This machine benchmarks at 9000000 dhrystones/second
And for the heck of it, how does Visual C++ 1.0’s performance compare?
Dhrystone(1.1) time for 180000000 passes = 7 This machine benchmarks at 25714285 dhrystones/second
That’s right the 1989 compiler is 35% the speed of the 1993 compiler. wow. Also it turns out that MASM 6.11 actually can (mostly) assemble the output of this ancient compiler. It’s nice when something kind of work. I can also add that the Infocom ’87 interpreter works as well.
I had a small twitter account, and I tried not to get dragged into anything that would just be basically wasting my time. Just stay focused and on topic. FINE. I just wanted to see if anyone ever saw it, if it was even worth the effort of doing WIP’s as I didn’t want to make it super annoying.
I logged on to post a fun update that I’d finally gotten a Phar Lap 386 version 4.1 app to do something halfway useful, the sairen AGI interpreter up and running in the most basic sense.
Talking about DOS Extenders is spammy and manipulation!
I don’t get what triggered it, but oh well there was a ‘have a review’ and yeah that was fine. Great. So I’m unlocked so I go ahead and post with the forbidden topic, as I’m clearly dumb, and forgetting that Twitter is for hate mobs & posting pictures of food, and cat pictures.
The Sairen AGI interpreter built with Watcom 386/7.0 & Phar Lap 386 4.1
So yes, that was a line too far, and now that’s it.
Now some of you may think, if you buy ‘the plan’ you’ll no doubt be exempt from the heavy hands of Twitter
3 squids a month
But I already was and had been for a while.
Your account is suspended
So that’s the end of that. I guess it’s all too confusing for a boomer like me.
Cancel me, cancel you
So needless to say I cancelled Twitter as well. Kind of sneaky they didn’t auto-cancel taking money.
So yeah, with that out of the way, let’s continue into DOS Extender land. I added just enough 386 magic, onto github: neozeed/sarien286. Yes I see now it really was a poorly named repo. Such is life.
There is 3 main things for porting old programs where they take care of all the logic, it’s going to be File I/O, Screen I/O, and timers. Luckily this time it was easier than I recalled.
Over on usenet (google groups link) Chris Giese shared this great summary on direct memory access from various methods:
/* 32-bit Watcom C with CauseWay DOS extender */
int main(void) {
char *screen = (char *)0xA0000;
initMode13();
*screen = 1;
return 0;
}
/* 32-bit Watcom C with DOS/4GW extender
(*** This code is untested ***) */
int main(void) {
char *screen = (char *)0xA0000;
initMode13();
*screen = 1;
return 0;
}
/* 32-bit Watcom C with PharLap DOS extender
(*** This code is untested ***) */
#include <dos.h> /* MK_FP() */
#define PHARLAP_CONVMEM_SEL 0x34
int main(void) {
char far *screen = (char far *)MK_FP(PHARLAP_CONVMEM_SEL, 0xA0000);
initMode13();
*screen = 1;
return 0;
}
/* 16-bit Watcom C (real mode) */
#include <dos.h> /* MK_FP() */
int main(void) {
char far *screen = (char far *)MK_FP(0xA000, 0);
initMode13();
*screen = 1;
return 0;
}
It is missing the Phar Lap 286 method:
/* Get PM pointer to text screen */
DosMapRealSeg(0xb800,4000,&rseg);
textptr=MAKEP(rseg,0);
But it’s very useful to have around as documentation is scarce.
Which brings me to this (again?)
Phar Lap 386|Dos-Extender 4.1
Years ago, I had managed to score a documentation set, and a CD-ROM with a burnt installed copy of the extender. I didn’t know if it was complete, but of course these things are so incredibly rare I jumped on the chance to get it!
2011!
Unfortunately, I didn’t feel right breaking the books apart, and scanning them, then add in some bad life choices on my part, and I ended up losing the books. Fast forward *years* later and Foone uploaded a document set on archive.org. GREAT! As far as I can tell the only difference in what I had is that I’ve got a different serial number. Thankfully I was smart enough to at lest email myself a copy of the CD-ROM contents! And this whole thing did inspire me to gut and upload the Phar Lap TNT 6.0 that I had also managed to acquire.
Although unlocking the video RAM wasn’t too bad, once I knew what to do, the other thing is to hook the clock for a timer. ISR’s are always hell, but at least this is a very simple one:
The methodology is almost always the same, as always, it’s the particular incantation.
So yeah, it’s super simple, but the 8086/80286 calling down to DOS/BIOS from protected mode via the int86 just had to be changed to int386, and some of the register structs being redefined. I’m not sure why but the video/isr code compiled with version 7 of Watcom, but crashes. I think its more drift in the headers, as the findfirst/findnext/assert calls are lacking from Watcom 7, so I just cheated and linked with Watcom 10. This led to another strange thing where the stdio _iob structure was undefined. In Watcom 10 it became __iob, so I just updated the 7 headers, and that actually worked. I had to include some of the findfirst/next structures into the fileglob.c file but it now builds and links fine.
Another thing to do differently when using Watcom 7, is that it doesn’t include a linker, rather you need to use 386LINK. Generating the response file, as there is so many objects didn’t turn out too hard once I realized that by default everything is treated as an object.
Another fun thing is that you can tell the linker to use the program ‘stub386.exe’ so that it will run ‘run386’ on it’s own, making your program feel more standalone. From the documentation:
386 | LINK has the ability to bind the stub loader program, STUB386.EXE, to
the front of an application .EXP file. The resulting .EXE file can be run by
typing the file name, just like a real mode DOS program. The stub loader
program searches the execution PATH for RUN386.EXE (the
386 | DOS-Extender executable) and loads it; 386 | DOS-Extender then loads
the application .EXP file following the stub loader in the bound .EXE file.
To autobind STUB386.EXE to an application .EXP file and create a bound
executable, specify STUB386.EXE as one of the input object files on the
command line.
So that means I can just use the following as my linker response file.
One interesting observation is that the 386 extender is actually smaller than the 286 one. And being able to compile with full optimisations it is significantly faster.
16bit on the left, 32bit on the right.
I ran both the prior 16bit protected mode version (on the left), and 32bit version (on the right), on the same IBM PS/2 80386DX 16Mhz machine. You can see how the 32bit version is significantly faster!.
I really should profile the code, and have it load all the resources into RAM, it does seem to be loading and unloading stuff, which considering were in protected mode, we should use all ram, or push the VMM386 subsystem to page, and not do direct file swapping, like it’s the 1970s.
Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.
-Ian Malcolm
Ever since I got my first 286 board way back in the early 90’s (1990? 1991?) I have been intrigued by the whole protected mode of operation. Unfortunately, in the era the required tools were way out of my reach, and of course were not available at retail. But now I live in the future where I have all the parts! Let’s look at the needed parts
I have to once more again thank my patrons, and people tolerating the google ads as it made all the difference in being able to buy all this stuff. And now is as good a time as any other to put it all together.
I stumbled upon this repo on sourceforge, Sarien. It included a Turbo C++ port, which is pretty exciting! So, this became my goal, get Sarien running on Phar Lap 286.
Installing Microsoft C 6.0a
Installing Microsoft C requires you to pick and choose both hosting, targeting environments, along with what the preferred libraries are. In the business we call this foreshadowing as this can be such a giant PITA. At least virtual machines are fast, plentiful, and cheap. In addition I had been using MS-DOS player to host the tools on Windows 11. This of course proved weird later.
The first step was getting it running on MS-DOS using Microsoft C 6.0a. This was actually pretty easy, the hard part was working out the makefile, as some files don’t compile with optimisations. And overall, the project doesn’t seem to work with /Ox at all. I haven’t spent enough time mixing and matching settings to find what actually doesn’t work, but I’m in a hurry, and /Os seems to work just fine.
In no time I had both the CGA & VGA drivers up and running and verified working on my PS/2. Great!
Now comes the fun, getting it ready to run on Pharlap.
The magic!
Phar Lap’s 286|DOS-Extender is pure magic. A DOS extender is a special program that can load a protected mode program into memory on a 286 or better computer and run it. At it’s heart, it can proxy MS-DOS functionality from protected mode to real mode, allowing you to use a lot of methodology and code from traditional real mode code. Phar Lap, goes beyond that by providing a pseudo OS/2 1.2 environment on MS-DOS, including advanced features like DLL’s, and being able to use ALL the RAM in your computer. Of course on the 286 there is a massive caveat:
The 286 has no built-in function for switching from protected mode to real mode. This makes programs that rely a LOT on MS-DOS potentially very slow. You can absolutely feel the difference between the real mode and the protected mode version of Sarien.
Phar Lap does include a test program, swtest which can benchmark the switching methods, so let’s run it and get some scores.
Switch code version = 1.14
BIOS signature: BA66CC86
BIOS date: 02/13/87
Machine ID = 0, A20 method = PS2, Reset method = Standard
Starting test for Switch Mode 3 (SLOW) ... Test complete.
Avg switch time (usecs): To prot = 34, To real = 101, Total = 135
Min switch time (usecs): To prot = 32, To real = 98, Total = 130
Max switch time (usecs): To prot = 35, To real = 103, Total = 138
Machine ID = 0, A20 method = PS2, Reset method = Standard
Starting test for Switch Mode 2 (AT) ... Test complete.
Avg switch time (usecs): To prot = 34, To real = 86, Total = 120
Min switch time (usecs): To prot = 33, To real = 83, Total = 116
Max switch time (usecs): To prot = 36, To real = 88, Total = 124
Machine ID = 0, A20 method = PS2, Reset method = Standard
Starting test for Switch Mode 1 (SURE) ... Test complete.
Avg switch time (usecs): To prot = 34, To real = 70, Total = 104
Min switch time (usecs): To prot = 32, To real = 68, Total = 100
Max switch time (usecs): To prot = 35, To real = 72, Total = 107
For those of you wondering what the timing is like on a 386, here is my 16Mhz PS/2 Model 80 board (now with fully 32bit memory)
Switch code version = 1.14
BIOS signature: 039D2DB4
BIOS date: 03/30/87
Machine ID = 0, A20 method = PS2, Reset method = Standard
Starting test for Switch Mode 5 (386) ... Test complete.
Avg switch time (usecs): To prot = 31, To real = 22, Total = 53
Min switch time (usecs): To prot = 30, To real = 20, Total = 50
Max switch time (usecs): To prot = 32, To real = 23, Total = 55
I’m honestly surprised the 286 switches from protected back to real so quickly! Although as you can see from the 386 timings it’s significantly faster than the 286.
Here is a quick video, real mode on the left, protected mode on the right. Yes I need to get a VGA capture card. Sorry.
Real mode on the left, Protected on the right. Running on the 386
Being a DOS extender it does have built in functions for things like hooking interrupts like this:
Using pointers into things like video ram do require ‘asking for permission’ but it’s not too involved:
int rseg;
/* Get PM pointer to text screen */
DosMapRealSeg(0xb800,4000,&rseg);
textptr=MAKEP(rseg,0);
with the segment mapped, and a pointer to the segment, and now I can read/write directly into video RAM!
/* save text screen */
memcpy(textbuf,textptr,4000);
Just like that!
I’m not sure what I screwed up on the VGA graphics, as it doesn’t work correctly, but oddly enough CGA does work.
And now this is where everything goes off the rails.
Sairen running on Qemu & DOSbox
It ran fine on emulation. So all excited I fired up the PS/2 and….
General protection fault
This lead me to more fun in how on earth to debug this. Of course Phar Lap 286 version 2.5 requires me to have Microsoft C/C++ 7.0. I shamelessly downloaded a disk set from pcjs.org. You actually need to install it, to copy out the files required:
28/05/1991 05:37 pm 47,216 CFIG286.EXE
26/11/1991 11:19 am 13,531 GORUN286.EXE
19/03/1992 04:00 am 42,720 shw0.dll
19/03/1992 04:00 am 105,039 eew0cxx.dll
19/03/1992 04:00 am 410,112 cvw4.exe
19/03/1992 04:00 am 91,118 eew0can.dll
19/03/1992 04:00 am 74,400 emw0w0.dll
03/08/1992 09:34 pm 2,649 INT33.DLL
03/08/1992 09:40 pm 2,718 MSG.DLL
03/08/1992 09:40 pm 1,702 NAMPIPES.DLL
03/08/1992 09:42 pm 2,073 NLS.DLL
03/08/1992 09:43 pm 5,184 PTRACE.DLL
03/08/1992 09:45 pm 2,320 SESMGR.DLL
03/08/1992 09:50 pm 1,508 WIN87EM.DLL
05/08/1992 12:04 am 3,100 KEYBOARD.DLL
05/08/1992 06:33 pm 270 TOOLHELP.DLL
14/08/1992 07:38 pm 7,891 KERNEL.DLL
14/08/1992 09:40 pm 14,545 USER.DLL
09/09/1992 10:59 pm 209,922 RUN286.EXE
09/09/1992 10:59 pm 229,046 RUN286D.EXE
14/09/1992 11:01 pm 14,024 TLW0LOC.DLL
17/09/1992 07:26 pm 34,152 CVP7.EXE
While CVP7 does come with Phar Lap, you have to run it via run286. As you may have noticed there is a mixture of OS/2 and Windows DLL’s in here, as at this point CodeView was a Windows protected mode debugger. The divorce was in full swing, and Microsoft C/C++ 7.0 had amputated the majority of OS/2 support. I’m sure all this is in the manuals, however all I have is disk images. There was no C 6.0a supported hosted debugger. Maybe it’s in the v1/v2 of Phar Lap 286, but I only have 2.5. There is version 3 files on the internet but I wanted to stick to 2.5.
Exception #13
And this is all I got. Yes, I did recompile with ‘/Od /Zi’ along with using cvpack on the executable. Yes, after copying the source to the PS/2 I was able to see the source line mapping, but it immediately jumps to assembly and GP Faults. All this is fine, but IT RUNS UNDER EMULATION.
What is going on?!
I asked around on discord, and found someone willing to test on their 286. It also crashed. I tried VMware and .. it crashed too! So did 86box! Ok now we’re going somewhere!
Since I had been using MS-DOS player to run the tools, I had an issue with the linker in C 6.0a crashing, so I tried the one from C/C++ 7. It also didn’t work. I tried the one from Visual C++ 1.5. It also failed. Almost giving up on the entire thing, since I had copied the source code to the PS/2, I tried something really silly, I compiled it using the /qc or QuickC flag. I wasn’t too worried about sizes as again I’m going to run in protected mode. It took some 20-30 minutes to compile, as 10Mhz machines are not the best for building software in this modern age. Much to my surprise it actually ran.
First run!
This was kind of shocking as I’m not sure what I screwed up to not get this to work, but it worked! I went ahead and changed the build to not use QuickC, but rebuild with /Os (Optimize for space). It took about an hour. And it too worked.
Phar Lap 286 running on OS/2 2.00 on VMware
Shockingly it runs! I’m not sure what on earth is up with the linking. I did find it easier to just rebuild on Qemu since it can easily map into my source directory and copy everything over and re-build very quickly.
Later I did try copying over compiled objects built using the MS-DOS Player, and linked them natively, and they ran fine.
What is it with the LINK?!
It all fits! Stacker rules!
One weird thing on 86box is my pre-built machine I was using has a 5 1/4″ 1.2Mb floppy for the A: drive. It’s too small to fit MS-DOS, the game data and Phar Lap 286 all on there. Although Stacker to the rescue and it fits!
It can save and even load those saves!
I removed a lot of Unix quality of life, to make it more MS-DOS dumping everything in the same directory so you can save & load games.
I’d never seen this one before, but attempting to boot up PC-DOS 4.00 or 4.01 on my PS/2 model 60, with 7 virtual disks attached, all I get is a single smiley/happy face, ASCII 1 ‘☺’ on boot.
MS-DOS 5.00 doesn’t care.
too many disks!
While trying the Apricot MS-DOS 4 set I got on eBay, it boots from floppy disks, crashes trying to boot from the hard disk, and trying to run fdisk just causes a divide by zero error.
– integer divide by 0
run-time error R6003
- integer divide by 0
I’m a bit reluctant to rip the whole machine apart as SD card extension cables don’t work for me which is even more annoying. Didn’t people buy big machines and put in a LOT of disks to just run MS-DOS? Even a Netware server still requires MS-DOS to boot.
Is my 286 just too weird?!
I’m not sure if it’s worth following up, but it is perplexing. Maybe I need to rename all my disks, and stick with whatever was actually selling in 1987. And sadly that means not fully loading it out.
I’m looking forward to how the series progresses, even DJ has dropped on in on the article!
So much trauma, 64kb segments, wrapping around, a20 gates, and overlaid virtual machines. And that’s not even really going into extenders. Looking forward to segmentation vs paging, interrupt reflection, and bi-modal handlers! Don’t cross the ISA 16Mbit DMA boundary!
I saw this video and I was like sold! I have this PowerMac 6400/180 so I figured this would be good. The problem was my network card was acting up so I figured instead of troubleshooting it I’ll just format it and go from there.
the machine is very much an Old World Macintosh, so that limits me from OS X. It’s 603ev CPU it’s not all that advanced either. I have an 8.1 ISO that I’ve been using under 68k emulation but the limit it has is old multimedia stuff ins t 68k compatible as nobody would imagine emulation putting 68k at speeds above a gigahertz.
I went looking for a 8.6 ISO, and that is where the fun hit me again that many so-called ISO images aren’t. Rather they are giant floppy disk images with the media headers and/or partition tables being obliterated. As an ISO they don’t detect at all, and as a giant floppy, of course they don’t boot as MacOS checks if it is on read-only media.
This ISO isn’t an ISO
very annoying
I did manage to finally find one that does work however!
I should also add the MacOS 8.1 CD-ROM image Ive been using as again,l I have the same issue where so many are headderless ‘floppies’ and not actual CD-ROM’s that don’t work in Cockatrice III or an actual Mac using BlueSCSI.
MacOS 8.1 CD-ROM on Cockatrice III
Sorry the image shows in black & white, but as you can see from the CD-ROM background it is in fact booted from the CD-ROM. You can download it from here: https://archive.org/details/mac-os-8.1-iso_202401
In no time, I was able to get online only to find that the power Mac plugin’s seem to be unavailable for anything and unsupported.
old netscape website
but the rendition of the old Netscape page was a treat!
Now I do have a Windows Surface RT tablet, and sure enough pluggin the proxy values, and YES the video site does work!
Warpstream on Windows RT
Very cool! So it turns out Protoweb can actually save all those old devices that work fine enough, but not fine enough for ‘Modern platforms’.
