Porting Quake II to MS-DOS pt1

(In this series, I’m going to go over the process of porting Quake II to MS-DOS.  Now of course the question will pop up why? And the answer is simple enough, after [HCI]Mara’akate added in gamespy support, something was noticeably off, and that is fewer and fewer people are playing Quake 1 these days.  So what to do?

Well apparently Quake II has an active following, so it’s time to move our creeky but favorite OS, MS-DOS into the Quake II Arena!

So the first thing to do is to grab a copy of the GPL source to Quake 2, along with a copy of the game, and get started on making a ‘null’ version of the game.  Null versions of the game have no graphical output, no sound, and ‘function’ at a very basic level.  It’s something to strive for as a first base in hitting that home run of a functional port.  Thankfully iD wrote really portable and modular software.  Unfortunately they tended to let their ports drift as they were writing the engine so the null code doesn’t actually work out of the box.  And the project makfiles leave a little to be desired for me, as they have a bunch of i386/Dec Alpha magic which doesn’t leave much in the room for weird ports like MS-DOS.

Now I should mention that before I’d gotten started the first thing I decided that like QuakeWorld for MS-DOS (and OS/2) I would use GCC as it is a known working compiler out of the box.  If you can, don’t fight so many battles of unknowns at once.  Another thing is that I am going to cross compile from OS X and test with DOSBox.  Of course you may want to use something else, and I know my tastes drift, but for now this is what I’m using. I’m using my old OSX to DJGPP cross compiler with GCC 2.95.3, which serves me well.

So the first thing was to compile and run the null version with native tools on OS X.  After a bit of struggle I got here:

$ ./q2
Added packfile ./baseq2/pak0.pak (3307 files)
Added packfile ./baseq2/pak1.pak (279 files)
Added packfile ./baseq2/pak2.pak (2 files)
Console initialized.
Sys_Error: ref_soft::R_BeginFrame() – catastrophic mode change failure

Well that’s great, how to figure this one out?

One thing I did to make it easier to work with the flow of Quake is to make the Sys_Error procedure contain a divide by zero.  Now why would I purposely put a divide by zero in the code?  Simple it lets me back trace the code when Quake catches it’s own faults so I can see what went wrong where, vs what would look like a clean exit.  From my example:

(lldb) bt
* thread #1: tid = 0xf14f9, 0x000c0853 q2`Sys_Error(error=0x000c17d4) + 98 at sys_null.c:28, queue = ‘com.apple.main-thread’, stop reason = EXC_ARITHMETIC (code=EXC_I386_DIV, subcode=0x0)
* frame #0: 0x000c0853 q2`Sys_Error(error=0x000c17d4) + 98 at sys_null.c:28
frame #1: 0x0003a0cb q2`Com_Error(code=0, fmt=0x000c17d4) + 368 at common.c:215
frame #2: 0x000c0523 q2`VID_Error(err_level=0, fmt=0x000cc8a8) + 81 at vid_null.c:45
frame #3: 0x000a6d38 q2`R_BeginFrame(camera_separation=0) + 592 at r_main.c:1145
frame #4: 0x000a4cfc q2`R_Init(hInstance=0x00000000, wndProc=0x00000000) + 276 at r_main.c:333
frame #5: 0x000c0740 q2`VID_Init + 362 at vid_null.c:122
frame #6: 0x00007900 q2`CL_Init + 55 at cl_main.c:1795
frame #7: 0x0003c2cb q2`Qcommon_Init(argc=1, argv=0xbffffc40) + 754 at common.c:1469
frame #8: 0x000c09ac q2`main(argc=1, argv=0xbffffc40) + 24 at sys_null.c:128
frame #9: 0x000019ad q2`_start + 212
frame #10: 0x000018d8 q2`start + 40
(lldb)

Now I know that R_BeginFrame was where the error was occurring, and looking at the code:

if ( ( err = SWimp_SetMode( &vid.width, &vid.height, sw_mode->value, vid_fullscreen->value ) ) == rserr_ok )
{

else
{
ri.Sys_Error( ERR_FATAL, “ref_soft::R_BeginFrame() – catastrophic mode change failure\n” );
}

We can see that the vid.width/vid.height aren’t being setup correctly.  Turns out there was a bunch more work to be done setting up vid_null!

 

After looking closer at the files, I notice as I’m stitching them together is that Quake II relies on DLL’s as part of it’s base functionality.  I drifted out of Quake after Quake 1, so I never played II before.  So I didn’t know this.  Obviously DJGPP doesn’t support DLL’s that can be loaded and un-loaded at will (Yes I know about DXE’s, but as the FAQ states, they cannot be un-loaded.  And I’m not going to fight DJGPP’s LIBC).  So looking further in the source, I saw these fun defines:

-DREF_HARD_LINKED -DGAME_HARD_LINKED

So at one point there was support for ‘hard linking’ in the ‘REF’ video driver, and the ‘game’ logic driver.  But it did kind of drift out of the code.  But looking at the Win32 version I could see that putting this functionality back in should be easy.  And to be honest if I learned any lesson from this, is that I should have been pulling the Win32 version apart by injecting null into it until it ran as a null platform, then use that as the basis.  Lesson learned.  Always start with a known good!  Quake II was built with Visual C++ 6, but I only have Visual C++ 4.2 installed on Crossover.  Yes I know again this is me being difficult.  But it didn’t take much time to get a simple project that has two DLL’s and a Win32 exe running.  Then I took on the ‘ref’ video driver and got that linking inside of the main EXE.  Now with one DLL ‘eliminated’ it was time to work on the game dll.

The game DLL posed the biggest challenge because it passes a reference to internal functions to it, and exports various functions back to the game engine.  So I ended up altering the engine to not call the game import/export directly but setting it up myself.  The hardest thing was that I couldn’t pull in the game header file, but rather I had to copy the prototypes myself.  Another interesting thing with the way Quake II works is that the game dll has to be able to be unloaded and loaded at will.  It wasn’t hard to simulate this, but I wasn’t expecting it.  Again this is probably because I never really played Quake II.

Now that I had Quake II building without DLL’s I could then take the next step of removing all the IO and re-replacing it with the null code, and now I had something that looked like it was doing something.

Added packfile ./baseq2/pak0.pak (3307 files)
Added packfile ./baseq2/pak1.pak (279 files)
Added packfile ./baseq2/pak2.pak (2 files)
execing default.cfg
execing config.cfg
NULLsock Initialized
Console initialized.
768k surface cache
ref_soft version: SOFT 0.01
——- sound initialization ——-
Sys_FindFirst [./baseq2/autoexec.cfg]: UNIMPLEMENTED!
==== InitGame ====
——- Server Initialization ——-
0 entities inhibited
0 teams with 0 entities
————————————-
====== Quake2 Initialized ======
loopback: client_connect
==== ShutdownGame ====
Sys_Error: Draw_Pic: bad coordinates

This turns out because I didn’t allocate the screen properly.  Looking at the code:

if ((x < 0) || (x + w > vid.width) ||
(y + h > vid.height))
{
ri.Sys_Error (ERR_FATAL,”Draw_Pic: bad coordinates”);
}

We can see it pretty plainly.

Now since we were going somewhere I started to write some MS-DOS code, and switch out of the null set of mind!

First a simple VGA mode 13 setup which gives us 320×200 resolution with 256 available colours.  And for good measure I did a simple VGA palette setup that I knew worked from a prior program.  Next we just blit our buffer onto the screen, and we get this:

Quake II on MS-DOS first screen

Quake II on MS-DOS first screen

Which is exciting and disappointing at the same time.  I then took the palette code from DOS Quake, and got something just as ugly.  I tried the code from OS/2. Same thing.  I tried all kinds of things and was going nowhere.

At this point Mara’akate added in the Linux clock code, and now we had this!

Quake II with a bad palette

Quake II with a bad palette

It wasn’t until much more digging around I saw some 320×240 screen setups that I realized there was something wrong there, and then I saw this gem in the linux port’s code:

/*
** SWimp_SetPalette
**
** System specific palette setting routine. A NULL palette means
** to use the existing palette. The palette is expected to be in
** a padded 4-byte xRGB format.
*/

In traditional VGA palette setups it’s 768 bytes that needs to be read/and pushed to the card.  I even checked Quake 1 is 768 bytes, but now in Quake II, it’s 1024 bytes!  OOPS!  Sometimes (ok a lot of times) you really need to check other ports or a ‘known good’ to see how they did things.

Quake II running on MS-DOS!

Quake II running on MS-DOS!

Pretty awesome!

So where to go from here? Obviously things like better keyboard input, the mouse, sound and networking need to be done.

Continued in pt2, pt3, pt4, and part5.

Announcing DJGPP 2.05 Beta 1

Yes, you read that right, not only is DJGPP still alive, but after the much stalled 2.04 beta cycle that has been going on for more than a decade, they have decided to push forward with 2.05 to get a release out there!

 

This is announcement of DJGPP 2.05 beta 1

It has numerous changes since previous DJGPP 2.04 beta 1 release in 2003.
(http://www.delorie.com/djgpp/mail-archives/browse.cgi?p=djgpp-announce/2003/12/06/22:18:05)
Unfortunately DJGPP v2.04 was never released and old beta version slowly
became almost unusable together with other newer DJGPP packages.

More information about changes in DJGPP 2.05 beta 1 is available at

http://www.delorie.com/djgpp/doc/kb/

both in sections about changes in 2.04 and 2.05. The same information is also
available in file info/kb.inf in djdev205.zip

It needs a lot of testing. Please test it if you have time and/or are
interested in any of the above features. Any level of testing would be
appreciated.

The beta is not available via the Zip Picker interface. You can download it
from here:

ftp://ftp.delorie.com/pub/djgpp/beta/v2

Additionally RPM packages (source and binary packages for i686 and x86_64) are
available from

ftp://ftp.delorie.com/pub/djgpp/rpms

Please see the README file for instructions on how to install the beta:

ftp://ftp.delorie.com/pub/djgpp/beta/v2/readme.1st

You can also download DJGPP 2.05 beta 1 packages from DJGPP mirror sites:

http://www.delorie.com/djgpp/getting.html

Thanks for all who have contributed to development of DJGPP

Andris Pavenis

I haven’t used it yet, but I see the following files in the beta directory:

djdev205 DJGPP V2 Development Kit and Runtime
djlsr205 DJGPP V2 Base Library Sources
djtst205 DJGPP V2 Test Programs (for testing the C library)
djcrx205 DJGPP V2 Cross-to-DOS Compiler Support Files (from djlsr/djdev)
djtzn205 DJGPP V2 Timezone Files
djtzs205 DJGPP V2 Timezone Sources

also floating around in the beta directory is GCC 5.1.0!

No doubt it’ll be a good excuse to rebuild and update some Quake related things to see how we fare in 21 century MS-DOS!

Impuse Tracker open sourced

I’ve never been musically inclined, but for those who loved the whole tracker scene of 20 years ago, Jeffrey Lim has released the source code to Impulse Tracker!  It features an impressive sound card support list:

  • Sound Blaster
  • Sound Blaster Pro
  • Sound Blaster 16
  • Sound Blaster AWE 32
  • Terratec EWS64
  • Hoontech Soundtrack 97 PCI
  • Gravis UltraSound, PnP, Max
  • Pro Audio Spectrum
  • Pro Audio Spectrum 16
  • Interwave
  • PC Speaker
  • DAC on LPT

It’s all in assembly and can be built with Turbo Assembler 4.1

You can find the source code on bitbucket here. Github mirror!

Netkeen

I’d never heard about this before, but a group reverse engineered Commander Keen, and re-built it as a network playable game!

Check it out at their site.

To build the source, you’ll need Borland C++ 5.0.2 and TASM 5.  Earlier versions may work but I haven’t tested.  The interesting thing with the networking is that they use the same code out of DOOM.

NetKeen

NetKeen

OpenWatcom v2

I know what you are thinking, wouldn’t it be great if you could create MS-DOS executables directly from a Win64 desktop with no MS-DOS needed?

Well, I just found out about this unofficial Open Watcom v2 project that targets the usual suspects, allows you to compile from Win64!

Hello World!

Hello World!

Some of the features of this fork include:

  • New 2-phase build system, OW can be build by platform native C/C++ compiler or by itself
  • Code generator properly initialize pointers by DLL symbol addresses
  • DOS version of tools now support long file names (LFN) if appropriate LFN driver is loaded by DOS
  • OW is ported to 64-bit hosts (WIN64, Linux X64)
  • Librarian support X64 CPU object modules and libraries
  • RDOS 32-bit C run-time compact memory model libraries are fixed
  • Resource compiler and Resource editors support WIN64 executables
  • OW text editor is now self containing, it can be used as standalone tool without any requirements for any additional files or configuration
  • Broken C++ compiler pre-compiled header template support is fixed
  • Many C++ compiler crashes are fixed
  • Debugger has no length limit for any used environment variable

Binaries are available on sourceforge.

So how does it fare?  I thought I’d take the old Wolf4GW, and compile it with this toolset.  The first hurdle I hit was this fun feature:

  • The C++ compiler now treats warning W737, implicit conversion of pointers to integral types of same size, as an error.

Which is an integral part of wl_menu.cpp .  So this was somewhat problematic, until I just commented out that block, and while I was expecting no working keyboard, I’m able to play, and load/save games…. Even the boss key works.

Wolf4GW

Wolf4GW

So with the W737 taken care of, I have to say this thing compiles FAST.  Incredibly FAST.  If for some reason you have to build 16bit or 32bit anything, you should look at a 64bit tool chain, well assuming you have a 64bit computer by now.

If anyone want’s to build their own Wolf4GW with the newer OpenWatcom, my source drop is here.

Today marks 20 years of FreeDOS!

From the original announcement:

A few months ago, I posted articles relating to starting a public
domain version of DOS.  The general support for this at the time was
strong, and many people agreed with the statement, “start writing!”
So, I have…

And how does it measure up?  Well the current release version is v1.1 and I tested it in both Qemu and VMWare Player.  First the installation process is SLOW.  Extraordinarily slow.  I really have no idea why, but 10+ minute installs in either environment were the best I could do.  The one thing I did do on installation was to disable a FAT32 drive, defaulting to the older FAT16 file-system.  Why? well because ..

Stacker 4 on FreeDOS

Stacker 4 on FreeDOS

Once installed, I thought I’d torture the environment, and install Stacker 4.0.  The installation went ok (I modified my system to boot up with option #3 as the default (Load FreeDOS including XMGR XMS-memory driver).  The installation went smooth, I setup a 300MB compressed drive, rebooted and the drive was not available.  I think this was because MS-DOS uses a config.sys file, and FreeDOS uses fdconfig.sys.  Although at the same time the autoexec.bat file was not modified as I had requested.  So I ran ‘config.exe’ and it walked me though additions to the c:\stacker.ini and what needed to be added to the (fd)config.sys (device=c:\stacker\stacker.com).  I did that, and rebooted, and I had my compressed D drive.

Then I installed DOOM v1.1 onto the compressed drive, and it runs perfectly fine! (is a screen shot really necessary?)  Seeing that DOS4G/W works (I also tested Descent, and Wolf4GW).  Oddly enough Wolf3d just hangs at a black screen, and Descent runs too fast although I think both issues are more specific to the games running on Qemu/VMWare than FreeDOS.  And if you want to skip the lengthy install, you can download my stock VMDK right here.

Seeing that 32bit DOS extenders seem OK, I thought I’d try TradeWars 2002 with it’s 16bit DPMI version, and it ran fine as well.

TradeWars on FreeDOS

TradeWars on FreeDOS

Now when it came to multitasking, Windows 3.0 won’t work outside of realmode.  Windows 3.1 however will run in standard mode.  386 Enhanced mode still doesn’t work correctly (is that even an issue)… So to use this for a BBS you’d need a network and some nodes… maybe fun for another day.  Excel 3.0c works under Windows 3.1 Standard mode, while Word 2.0c does not.

Another weird thing is that the FreeDOS partitioning program (fdisk) chooses a NON-DOS partition type by default, so if you were expecting to dualboot between MS-DOS and FreeDOS I think you need to install MS-DOS first, although I guess in the age of virtual machines does it matter that much?

For basic MS-DOS program compatibility it seems pretty much there, even with weird stuff like stacker.

MS-DOS Player updates

Poorly translated from TAKEDA toshiya’s blog..


2014/4/15
I has integrated source of i386 and i286 edition edition. 
In addition, in the i286 version, I added support for int 10h/16h. equivalent to 0.149 MAME, I was replaced with a 0.152 equivalent MAME core i386 i286 core. However, the i386 core, I have omit the TLB around.

Which is very cool, although I wasn’t sure about the MAME source code being open to other projects…?  I tried to contact the i86/i386 author vlinde but he then pulled his contact page.  I wanted to use i386 for something of my own, but the whole “Redistributions may not be sold, nor may they be used in a commercial product or activity.” really puts the damper on it.

I was able to get some simple XMS test program to run, but nothing of any substance.  No DOS4G/W or anything like that.  But if you re-build it specifying MS-DOS version 5.0, some of the MS-DOS utils and even command.com work!

The weird issue I had was running out of conventional RAM, because this program gives you nearly 1MB of conventional RAM… I was surprised, as I wasn’t expecting that much!

George R R Martin Writes with WordStar 4!

So apparently it’s all the news, The best selling series, Game of Thrones was written using Wordstar 4.  On a dedicated MS-DOS computer of all things.

Game of thrones..

What George probably sees..

Well isn’t that kinda cool.

As he says, he likes Wordstar because it doesn’t try to think for him, and he likes MS-DOS because there is no distractions.

“I actually like it, it does everything I want a word processing program to do and it doesn’t do anything else,” Martin told Conan O’Brien. “I don’t want any help. I hate some of these modern systems where you type a lower case letter and it becomes a capital letter. I don’t want a capital. If I wanted a capital, I would have typed a capital. I know how to work the shift key.”

He best says it back in 2007:

I do my writing on a completely different computer than the one I use for email and the internet, in part to guard against viruses, worms, and nightmares like this. My work machine does not even use Windows (which I loathe). I write with WordStar 4.0 on a pure DOS-based machine. Mock if you must… but WordStar and DOS are both stable as rocks, and never give me the sort of headaches I get from Windows. (I won’t even talk about Microsoft Word, about which I have nothing printable to say).

For anyone chasing Wordstar nostalgia, you can leaf through the manual.

Terminator Rampage.

terminator rampage front

Terminator Rampage

Back in October of 1993, this cool looking game, The Terminator Rampage was released.

But sadly I had a lowly 4MB 386sx-16 with CGA, so things like this game with it’s awesome VGA graphics were an impossibility.

Even more sad at the time was that ‘primitive’ 3d games like wolfenstein 3d also required VGA.

But as we all know a few short months later, DOOM was released, and then Terminator Rampage was quickly swept off the face of the earth.

I recently came across this page,  and I thought I’d give it a shot.  The requirements are pretty ‘minimal’:

  • Minimal 386DX-25 with 4MB of Ram, VGA & 18MB of disk space
  •  Recommended 386DX-33 or 486DX-33

So I was thinking Qemu could easily run this game.  Long story short, it doesn’t work.  Turns out Rampage needs EMS.  And for whatever reasons, running emm386.exe on Qemu (I tried a handful of versions) just crash on Qemu after initialization.  Failing with stock Microsoft EMM386, I tried JEMM, which loaded, and ran it’s built in EMS diagnostics OK, but trying to run Rampage resulted in a nice crash.

Qemu crash

Qemu crash

So giving up on Qemu, I tried it on DOSBox.  It runs but it is incredibly jerky.  So I thought I’d try PCem, and see how it runs there.

So the plus side is that PCem, is able to run MS-DOS & EMM386.EXE without issues.  It only took a few minutes to install MS-DOS 5.0 and Rampage on my ‘virtual’ 486DX2/66 with 8MB, of ram, and load up Rampage to be greeted with it’s jerky motion.

Thinking its something with emulation in general I fire up Norton SI to get some PCem scores how it benches against known good samples.

pcem 8.1 386DX33 Norton SI score

386DX-33Mhz 37.4

pcem 8.1 486DX2-66 Norton SI score

486DX/2-66Mhz 136.8

pcem 8.1 WinChip90 Norton SI score

WinChip-90 186

Then comparing the scores to this handy (if not ancient) Norton Si benchmark spread we see:

Pentium 60mhz
=============
IBM Clone               P5/60              256k              187.2
w/ Premiere MB

486 DX2/66mhz
=============
Elitegroup (ECS)       iDX2/66             256k              147.3
SA486P AIO-II
INTEL CDC,SIO&DPU

386 DX 33mhz
=============
Forex 36C100 iDX/33 128k 35.9

Well now that interesting, so at a ‘raw’ CPU level, PCem is delivering on what would be classical performance.  So for the heck of it, I load up DOOM, and it runs a bit choppy on the 386, but flies on the 486 & Winchip emulation.  Now that is strange.  And just to confirm…

Terminator Rampage Box (back)

Terminator Rampage Box (back)

They really thought this would be playable on a 486 @ 33Mhz.

So how does it choke?  While going straight is ‘ok’ turning around is so utterly sluggish that there is no feeling of immersion.  It feels like you are driving an incredibly slow tank.  At the same time, the more realistic sprites, and textures serve to make it look even more unrealistic.

So what am I talking about?

Well here is a screenshot of Wolfenstein 3d on the 386DX-33 (and more than playable).

Wolfenstein 3d

Wolfenstein 3d

As you can see, there is no ceiling, and no floor textures.  The walls are all uniform height, and the textures were clearly drawn by hand, giving it a very fake and ’16bit’ feeling.  I should also add on a capable 286, this game is playable.

Terminator Rampage

Terminator Rampage

Now at first it looks like it has a lot in common with the soon to be release DOOM, with textured ceilings and floors.

Doom

Doom

Now as you can see the difference in DOOM is the 2.5D effect of there being lower areas so you can go up and down stairs (while you cannot go under them).  Also Doom introduces dynamic lighting, and better sound rendering.

While I do like Rampage’s upfront map, as you can see thought, it is very square. In a small effort to ‘speed’ up Rampage you can turn off the ceilings and floors revealing a very Wolf3d like environment.  Unfortunately the more they tried to give the world  detail, the more it well just looks flat.

No ceilings, No floors.

No ceilings, No floors.

Which kind of kills the whole thing.  Maybe they should have left out things like water fountains.

High detail sprites

High detail sprites

Then you get things like this computer setup (one of the programmers? The accountant’s lamp is a nice touch) but it’s a sprite, so as you rotate around it, you always see the same face.

Ironically it’s these high resolution background sprites that make the environment feel less real, as they make the rooms feel too open, and too sparse.

Too open, and yet too sparse

Too open, and yet too sparse

It is the real paradox that in a good shooter you have lots of room, and things to duck behind, but the rooms feel too large, and look bizarre with the massive open spaces.  But it is more so a limitation of the time, with the engine being more of an improved Wolf3d engine, than taking a larger leap into being something more 2.5d or 3d like Doom (or the distant Quake).

Another thing that really bugged me was the doors.

Doom door

Doom door

In doom, the doors felt more ‘natural’ in that they weren’t super wide.

Screen Shot 2014-04-28 at 10.44.09 PM

Generic office door in Rampage

But in rampage they are stretched wide giving the impression of why you can’t turn is you are incredibly wide..

Screen Shot 2014-04-28 at 10.43.16 PM

Rampage ‘exit’ door

Even the ‘exit’ door texture still feels too wide.

I could probably get by the empty spaces, but it takes so long to turn around, and the controls feel so unnatural (they don’t even try to be a Wolf3d control-a-like) that it really feels klunky.  No matter what speed you play it at.

It really was an exercise in frustration.