Targeting OS/2 with Visual Studio 2003

Sydney’s idea of what Visual C++ for OS/2 should look like

No, it’s not a typo.

This is a long-winded post, but the short version is that I found a working combination to get the C compiler from Visual Studio 2003 targeting OS/2.

Once I’d learned how C compilers are a collection of programs working in concert, I’d always wanted to force Microsoft C to work in that fashion, however it is born to be a compiler that integrates everything but linking. There has been a “/Fa” or output assembly option, but I’ve never gotten it to do anything useful. I’m not that much into assembly but it seemed insurmountable.

But for some reason this time things were different.

This time I used:

Microsoft (R) Macro Assembler Version 6.11

After the great divorce and the rise of Windows NT, Microsoft had shifted from the OMF format to COFF. However somewhere buried in their old tools it still supports it, namely MASM. For example, if I try to run LINK386 (the OS/2 Linker) against output from Visual C++ 2003 I get his

abandon.obj :  fatal error L1101: invalid object module
 Object file offset: 1  Record type: 4c

However if I output to assembly and then have MASM assemble that, and try the linker, I’m bombarded with errors like this:

warp.obj(warp.asm) :  error L2025: __real@4059000000000000 : symbol defined more than once
warp.obj(warp.asm) :  error L2029: '__ftol2' : unresolved external

If I was smart I’d have given up, there is pages and pages of this stuff. But I’m not smart, so instead I decided to something different, and use SED, the stream editor, and try to patch out the errors.

The ftol2 call is for newer CPU’s and any OS/2 library won’t have it. But instead of binary editing symbols we can replace the ftol2 with ftol with this simple line:

 sed -e 's/_ftol2/_ftol/g'

For some reason Visual C++ likes to make all it’s reals “public” meaning there can only be one, but yet there is so many. Why not comment them all out?

sed -e 's/PUBLIC\t__real@/;PUBLIC\t__real@/g'

And there are various other annoying things, but again they can be all patched out. Just as the older Windows 1991 Pre-release compilers also have weird syntax that MASM doesn’t understand.

astro.asm(59): error A2138: invalid data initializer

which goes into how Microsoft C used to initialize floating point constants:

CONST   SEGMENT  DWORD USE32 PUBLIC 'CONST'
$T20000         DQ      0a0ce51293ee845c8r    ; 1.157407407407407E-05
$T20001         DQ      0bffffd3441429ec5r    ; 2440587.499999667
CONST      ENDS

while I found that the C compiler in Xenix 386 initialises them like this:

$T20000         DQ      0a0ce51293ee845c8h    ; 1.157407407407407E-05
$T20001         DQ      0bffffd3441429ec5h    ; 2440587.499999667

This one was a little hard for me as I’m not a sed expert, but I did figure out how to mark the section, and then to replace it

sed -e "s/DQ\t[0-9a-f]r/&XMMMMMMX/g" $.a1 |  sed -e "s/rXMMMMMMX/H/g"

And so on. At the moment my ‘mangle’ script is now this:

.c.obj:
        $(CC) $(INC) $(OPT) $(DEBUG) /c /Fa$*.a $*.c
        wsl sed -e 's/FLAT://g' $*.a > $*.a1
        wsl sed -e "s/DQ\t[0-9a-f]*r/&XMMMMMMX/g" $*.a1 \
        | wsl sed -e "s/rXMMMMMMX/H/g" \
        | wsl sed -e 's/call \t\[/call DWORD PTR\[/g' \
        | wsl sed -e 's/PUBLIC\t__real@/;PUBLIC\t__real@/g' \
        | wsl sed -e 's/_ftol2/_ftol/g' > $*.asm
        ml /c $*.asm
        del $*.a $*.a1 $*.asm

This allows me to plug it into a Makefile, so I only have to edit it in one place.

Not surprisingly, this allows the LINK from Visual C++ 1.0 to link the MASM generated object files and get a native Win32 executable. Even from the oldest compiler I have from the Microsoft OS/2 2.00 Beta 2 SDK from 1989!

But now that we have the C compilers being able to output to something we can edit and force into a Win32, there is a few more things and suddenly:

        C:\cl386-research\bin\13.10.6030\cl386 /u /w /G3 /O  /c /Faphoon.a phoon.c
C:\cl386-research\bin\13.10.6030\CL386.EXE: warning: invoking C:\cl386-research\bin\13.10.6030\CL.EXE
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.6030 for 80x86
Copyright (C) Microsoft Corporation 1984-2002. All rights reserved.

phoon.c
        wsl sed -e 's/FLAT://g' phoon.a > phoon.a1
        wsl sed -e "s/DQ\t[0-9a-f]*r/&XMMMMMMX/g" phoon.a1  | wsl sed -e "s/rXMMMMMMX/H/g"  | wsl sed -e 's/call \t\[/call DWORD PTR\[/g'  | wsl sed -e 's/PUBLIC\t__real@/;PUBLIC\t__real@/g'  | wsl sed -e 's/_ftol2/_ftol/g' > phoon.asm
        ml /c phoon.asm
Microsoft (R) Macro Assembler Version 6.11
Copyright (C) Microsoft Corp 1981-1993.  All rights reserved.

 Assembling: phoon.asm
        del phoon.a phoon.a1 phoon.asm
        msdos286 run286 C:\cl386-research\bin\ddk12\LINK386.EXE @phoon.lnk

Operating System/2 Linear Executable Linker
Version 2.01.012 Nov 02 1993
Copyright (C) IBM Corporation 1988-1993.
Copyright (C) Microsoft Corp. 1988-1993.
 All rights reserved.

Object Modules [.obj]: astro.obj date_p.obj phoon.obj
Run File [astro.exe]: phoon2 /NOE /NOI /NOD:OLDNAMES
List File [nul.map]: nul.map
Libraries [.lib]: ..\..\lib2\libc.lib +
Libraries [.lib]: ..\..\lib2\os2386.lib
Definitions File [nul.def]: nul.def;
LINK386 :  warning L4071: application type not specified; assuming WINDOWCOMPAT

I know it’s a bit of a word salad, but the key thing here is that using Visual C++ 2003’s compiler (version 13.10.6030), and outputting to assembly that we can edit, we can then use MASM to build objects that surprisingly LINK386 version 2.01.012 will link with. I suspect this has to do with device drivers, and probably the majority of the OS/2 operating system.

Anways, we’ve done the incredible, using the same object files, we made both a Win32 application, and an OS/2 application!

phoon-13.10.6030.exe: PE32 executable (console) Intel 80386, for MS Windows
phoon2.exe: MS-DOS executable, LX for OS/2 (console) i80386
Phoon compiled by Visual C++ 2003 on OS/2 2.00

Incidentally Happy CNY!

Obviously, this is VERY cool stuff.

I know the next question is do we have to rely on a 16bit linker? How about Watcom?

C:\cl386-research\proj\trek>wlink @trek.wlk
WATCOM Linker Version 10.0
Copyright by WATCOM International Corp. 1985, 1994. All rights reserved.
WATCOM is a trademark of WATCOM International Corp.
loading object files
searching libraries
Warning(1008): cannot open LIBC.lib : No such file or directory
Warning(1008): cannot open OLDNAMES.lib : No such file or directory
creating an OS/2 32-bit executable

Ignore the warnings and YES we can Link from something much newer & 32bit! In this example I linked the old TREK game, also built with Visual C++ 2003. The response file looks lke:

SYS os2v2
NAME trek2w
FILE abandon.obj,attack.obj,autover.obj,capture.obj,cgetc.obj,checkcond.obj
FILE check_out.obj,compkl.obj,computer.obj,damage.obj,damaged.obj,dcrept.obj
FILE destruct.obj,dock.obj,dumpgame.obj,dumpme.obj,dumpssradio.obj,events.obj
FILE externs.obj,getcodi.obj,getpar.obj,help.obj,impulse.obj,initquad.obj
FILE kill.obj,klmove.obj,lose.obj,lrscan.obj,main.OBJ,move.obj
FILE nova.obj,nullsleep.obj,out.obj,phaser.obj,play.obj,ram.obj
FILE ranf.obj,rest.obj,schedule.obj,score.obj,setup.obj,setwarp.obj
FILE shield.obj,snova.obj,srscan.obj,systemname.obj,torped.obj,utility.obj
FILE visual.obj,warp.obj,win.obj
LIBR ..\..\lib2\LIBC.LIB
LIBR ..\..\lib2\OS2386.LIB

It’s probably needing additional stack space, maybe some other stuff, or resources, maybe how to flag it’s windowing compatible.

TREK built by Visual C++ 2003, and Linked using Watcom C/C++ 10.0

How do I get started, if I dare?! First download and unpack cl386-research-v2. Ideally on the root of your C: drive, because why not?

run the ‘env’ command to set your environment up. Its pretty complicated but in the proj directly there is currently:

*NOTE that I do use SED scripts, I have it set to use Linux in the WSL package.
I tried some Win32 sed but it didn’t work. So you need WSL or a working sed!

you can then go into each directory and run

make os2

and it’ll compile populate a floppy and launch the emulator

Its all good fun.

Read the Makefiles to configure a compiler, how to run it, and if you need to mangle the assembly. The 32bit new stuff needs to be mangled, the older stuff almost always works with just compile.

# Version 6.00.054      1989
# https://archive.org/details/os-2-cd-rom_202401
PLATFORM = ddksrc

In this case it’ll select the platform from the ‘ddksdk’ release. The next is if the compiler is OS/2 based or native win32. Basically 73g / windows 95 & below are native Win32.

In the above example we comment out the dos extended cross

# dos exteded cross
CC =  $(EMU) $(DOSX) $(CL386ROOT)\$(PLATFORM)\cl386
# native CC
# CC =  $(CL386ROOT)\$(PLATFORM)\cl386

Next is the mangle strategy. In this case it’s an ancient OS/2 (like) compile so
try un commenting the ‘just compile’ line

# must include ONLY ONE strategey..
# for OS/2 it must have been assembled my MASM 6.11

include ..\-justcompile.mak
#include ..\-mangleassembly.mak
#include ..\-plainassembly.mak

save the makefile, and run

nmake os2

You can just close the emulator as after each run it’ll unpack a hard disk image, so nothing will be lost. or saved. It’s just for testing. You may need to periodically clean the floppy drive, as that is the only way to transfer stuff in and out of the VM.

What versions of CL386 have I found? Well, it’s quite a few, although I know I’m missing quite a few.

== c386 ============================
Microsoft C 5 386 Compiler

Microsoft C 5.2 286/386 Compiler -- Driver

@(#)C Compiler Apr 19 1990 11:48:30

Copyright (c) Microsoft Corp
1984-1989. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 1.00.075
Quick C Compiler Version 2.00.000
1.00.075

== ddk12 ============================

C 6.00 (Alpha) Aug 24 1990 19:12:31

Copyright (c) Microsoft Corp
1984-1989. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.054
Quick C Compiler Version 2.00.000
6.00.054

== ddk20 ============================

C 6.00 (Alpha) Aug 16 1990 23:04:06

Copyright (c) Microsoft Corp
1984-1989. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.054
Quick C Compiler Version 2.00.000
6.00.054

== ddksrc ============================

C 6.00 (Alpha) Aug 24 1990 19:21:49

Copyright (c) Microsoft Corp
1984-1989. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.054
Quick C Compiler Version 2.00.000
6.00.054

== nt-sep ============================

@(#)C Compiler 6.00 Feb 06 1991 17:15:19
@(#)C Compiler 6.00 May 13 1991 23:54:12
@(#)C Compiler 6.00 Jun 03 1991 15:16:22


Copyright (c) Microsoft Corp
1984-1991. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.077
Quick C Compiler Version 2.00.000
6.00.077

== nt-oct ============================

@(#)C Compiler 6.00 Jun 03 1991 15:16:22
@(#)C Compiler 6.00 Jun 13 1991 22:07:23
@(#)C Compiler 6.00 Oct 10 1991 00:42:24

Copyright (c) Microsoft Corp
1984-1991. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.080
Quick C Compiler Version 2.00.000
6.00.080

== nt-dec ============================
@(#)C Compiler 6.00 Jun 03 1991 15:16:22
@(#)C Compiler 6.00 Jun 13 1991 22:07:23
@(#)C Compiler 6.00 Oct 10 1991 00:42:24

Copyright (c) Microsoft Corp
1984-1991. All rights reserved.
  (press <return> to continue)
Microsoft 386 C Compiler. Version 6.00.081
Quick C Compiler Version 2.00.000
6.00.081

== 73g  ============================
1984-1993. All rights reserved.
Copyright (c) Microsoft Corp
8.00.3200
32-bit C/C++ Optimizing Compiler Version
Microsoft (R)


== msvc32s ============================

Microsoft 8.00.0000 - Copyright (C) 1986-1993 Microsoft Corp.
Microsoft 8.00.0000 - Copyright (C) 1986-1993 Microsoft Corp.
@(#) Microsoft C/C++ 32 bits x86 Compiler Version 8.00.XXXX

8.00.000

== 13.10.6030 ============================

Microsoft (R) C/C++ Compiler Version 13.10.6030
From my install of Visual Studio 2003 Enterprise

As you can see many of these earlier OS/2 compilers report the same versions but are in fact different builds on the inside. I suspect Microsoft had to support one version, and an Alpha version of version 6 is as good as it got. I would have imagined there were internal 32bit versions of 6 or 7, but I haven’t seen them.

Compiling and running TREK

Hopefully this gives some idea of how I tried to made a probably too modular build system to try all kinds of different compilers. I might have to see if it’s possible to run the tools from the 1992 versions of Windows NT in this setup, perhaps they are interesting as well.

One thing in my porting GCC to OS/2 experience is that the usability of the C compilers from 1991 were dramatically better than what Microsoft had given IBM at the time of the divorce. No doubt the upcoming NTOS/2 project was placing a bigger demand on the tools team.

If anyone has any access to other ‘cl386’ compilers, or early OS/2 2.00 stuff, please let me know! I’d love to do build/tests and see if my idea of distributing objects ‘just works’!

4 thoughts on “Targeting OS/2 with Visual Studio 2003

  1. Heh, I should have thought of that.

    Out of interest, why not try a compiler and assembler from the same era? Either the 1993 compiler, or use the 2003 assembler. It looks like the one included with Visual C++ 2003 defaults to COFF but the /omf switch is still there and appears to work. I’d hope if they’re from the same generation the syntax mismatches would be reduced.

    • the newer assembler hits the same road blocks..

      phoon.obj(phoon.asm) : error L2025: __real@3f50624dd2f1a9fc : symbol defined more than once

      from the duplicates to the ftol2

      Error! E2028: __ftol2 is an undefined reference

      and the old 1991 cl386

      astro.asm(59) : error A2138: invalid data initializer

      basically the same issues..

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.