Porting Quake II to MS-DOS pt2

Continuing in this series on porting Quake II to MS-DOS, we get to touch some of the fun stuff.  The first big ‘fun’ thing is networking.

Now in my prior work with the MS-DOS version of Quake, I had used the WATTCP library to bring networking to the otherwise Windows/UNIX specific fun of network deathmatch back to MS-DOS.  Quake by default had support for the Beame & Whiteside’s TCP/IP stack which for all intents and purposes has vanished from the face of the Earth (does anyone have a copy?!).  So, at the time, I thought it’d be cool to try to interface WATTCP with Quake, and it worked on the first attempt as WATTCP is a very competent TCP/IP stack.

So I took the Linux networking file net_udp, and compiled it, and I got an executable!

When it comes to testing WATTCP though, I prefer to use Qemu instead of DOSBox as it can not only emulate various network cards to which I can find packet drivers (yes even the evil PCI NE2000!) but it has a built in SLiRP network stack that let’s me NAT on my desktop without any crazy network configs.

And for the sake of testing, I setup a ‘null’ text mode server, figured out some flags, and I was able to connect!

Quake II for MS-DOS running on Qemu connected to a dedicated OS X server.

Quake II for MS-DOS running on Qemu connected to a dedicated OS X server.

Very exciting stuff indeed!

Now for some interesting stuff.  First, I noticed that MS-DOS 5.00 with himem.sys is almost unplayable because it is so slow.  MS-DOS 4.01 without himem.sys is actually faster.  No, I’m not kidding.

Next is that some levels LOVE to gobble up RAM.  Maps like city1 will need at least 192MB of ram.  I haven’t even tried playing with the virtual memory of DJGPP, and I really don’t want to.  And let’s face it, if you even try to load Quake II on a MS-DOS machine, it better be a ‘big’ one.  This means you should be using the ‘dos’ from Windows 98, or perhaps FreeDOS, although I haven’t tested that at all.

So far from our limited testing the networking seems to be pretty good.   And at least that is one function we didn’t have to really pour a lot of effort into.  Although the payoff of being able to connect to servers on the LAN and even the internet is a good thing.

Continued in pt3, pt4, and part 5.

Porting Quake II to MS-DOS pt1

(In this series, I’m going to go over the process of porting Quake II to MS-DOS.  Now of course the question will pop up why? And the answer is simple enough, after [HCI]Mara’akate added in gamespy support, something was noticeably off, and that is fewer and fewer people are playing Quake 1 these days. So what to do?

Well apparently Quake II has an active following, so it’s time to move our creeky but favorite OS, MS-DOS into the Quake II Arena!

So, the first thing to do is to grab a copy of the GPL source to Quake 2, along with a copy of the game, and get started on making a ‘null’ version of the game.  Null versions of the game have no graphical output, no sound, and ‘function’ at a very basic level.  It’s something to strive for as a first base in hitting that home run of a functional port.  Thankfully iD wrote really portable and modular software.  Unfortunately, they tended to let their ports drift as they were writing the engine, so the null code doesn’t actually work out of the box.  And the project makfiles leave a little to be desired for me, as they have a bunch of i386/Dec Alpha magic which doesn’t leave much in the room for weird ports like MS-DOS.

Now I should mention that before I’d gotten started the first thing, I decided that like QuakeWorld for MS-DOS (and OS/2) I would use GCC as it is a known working compiler out of the box.  If you can, don’t fight so many battles of unknowns at once.  Another thing is that I am going to cross compile from OS X and test with DOSBox.  Of course you may want to use something else, and I know my tastes drift, but for now this is what I’m using. I’m using my old OSX to DJGPP cross compiler with GCC 2.95.3, which serves me well.

So, the first thing was to compile and run the null version with native tools on OS X.  After a bit of struggle, I got here:

$ ./q2
Added packfile ./baseq2/pak0.pak (3307 files)
Added packfile ./baseq2/pak1.pak (279 files)
Added packfile ./baseq2/pak2.pak (2 files)
Console initialized.
Sys_Error: ref_soft::R_BeginFrame() – catastrophic mode change failure

Well that’s great, how to figure this one out?

One thing I did to make it easier to work with the flow of Quake is to make the Sys_Error procedure contain a divide by zero.  Now why would I purposely put a divide by zero in the code?  Simple it lets me back trace the code when Quake catches its own faults so I can see what went wrong where, vs what would look like a clean exit.  From my example:

(lldb) bt
* thread #1: tid = 0xf14f9, 0x000c0853 q2`Sys_Error(error=0x000c17d4) + 98 at sys_null.c:28, queue = ‘com.apple.main-thread’, stop reason = EXC_ARITHMETIC (code=EXC_I386_DIV, subcode=0x0)
* frame #0: 0x000c0853 q2`Sys_Error(error=0x000c17d4) + 98 at sys_null.c:28
frame #1: 0x0003a0cb q2`Com_Error(code=0, fmt=0x000c17d4) + 368 at common.c:215
frame #2: 0x000c0523 q2`VID_Error(err_level=0, fmt=0x000cc8a8) + 81 at vid_null.c:45
frame #3: 0x000a6d38 q2`R_BeginFrame(camera_separation=0) + 592 at r_main.c:1145
frame #4: 0x000a4cfc q2`R_Init(hInstance=0x00000000, wndProc=0x00000000) + 276 at r_main.c:333
frame #5: 0x000c0740 q2`VID_Init + 362 at vid_null.c:122
frame #6: 0x00007900 q2`CL_Init + 55 at cl_main.c:1795
frame #7: 0x0003c2cb q2`Qcommon_Init(argc=1, argv=0xbffffc40) + 754 at common.c:1469
frame #8: 0x000c09ac q2`main(argc=1, argv=0xbffffc40) + 24 at sys_null.c:128
frame #9: 0x000019ad q2`_start + 212
frame #10: 0x000018d8 q2`start + 40
(lldb)

Now I know that R_BeginFrame was where the error was occurring, and looking at the code:

if ( ( err = SWimp_SetMode( &vid.width, &vid.height, sw_mode->value, vid_fullscreen->value ) ) == rserr_ok )
{

else
{
ri.Sys_Error( ERR_FATAL, “ref_soft::R_BeginFrame() – catastrophic mode change failure\n” );
}

We can see that the vid.width/vid.height aren’t being setup correctly.  Turns out there was a bunch more work to be done setting up vid_null!

After looking closer at the files, I notice as I’m stitching them together is that Quake II relies on DLL’s as part of its base functionality.  I drifted out of Quake after Quake 1, so I never played II before. Â So I didn’t know this.  Obviously DJGPP doesn’t support DLL’s that can be loaded and un-loaded at will (Yes I know about DXE’s, but as the FAQ states, they cannot be un-loaded.  And I’m not going to fight DJGPP’s LIBC). Â So looking further in the source, I saw these fun defines:

-DREF_HARD_LINKED -DGAME_HARD_LINKED

So, at one point there was support for ‘hard linking’ in the ‘REF’ video driver, and the ‘game’ logic driver. But it did kind of drift out of the code.  But looking at the Win32 version I could see that putting this functionality back in should be easy.  And to be honest if I learned any lesson from this, is that I should have been pulling the Win32 version apart by injecting null into it until it ran as a null platform, then use that as the basis.  Lesson learned.  Always start with a known good!  Quake II was built with Visual C++ 6, but I only have Visual C++ 4.2 installed on Crossover.  Yes, I know again this is me being difficult.  But it didn’t take much time to get a simple project that has two DLL’s and a Win32 exe running.  Then I took on the ‘ref’ video driver and got that linking inside of the main EXE.  Now with one DLL ‘eliminated’ it was time to work on the game DLL.

The game DLL posed the biggest challenge because it passes a reference to internal functions to it, and exports various functions back to the game engine.  So, I ended up altering the engine to not call the game import/export directly but setting it up myself.  The hardest thing was that I couldn’t pull in the game header file, but rather I had to copy the prototypes myself.  Another interesting thing with the way Quake II works is that the game DLL has to be able to be unloaded and loaded at will.  It wasn’t hard to simulate this, but I wasn’t expecting it.  Again, this is probably because I never really played Quake II.

Now that I had Quake II building without DLL’s I could then take the next step of removing all the IO and re-replacing it with the null code, and now I had something that looked like it was doing something.

Added packfile ./baseq2/pak0.pak (3307 files)
Added packfile ./baseq2/pak1.pak (279 files)
Added packfile ./baseq2/pak2.pak (2 files)
execing default.cfg
execing config.cfg
NULLsock Initialized
Console initialized.
768k surface cache
ref_soft version: SOFT 0.01
——- sound initialization ——-
Sys_FindFirst [./baseq2/autoexec.cfg]: UNIMPLEMENTED!
==== InitGame ====
——- Server Initialization ——-
0 entities inhibited
0 teams with 0 entities
————————————-
====== Quake2 Initialized ======
loopback: client_connect
==== ShutdownGame ====
Sys_Error: Draw_Pic: bad coordinates

This turns out because I didn’t allocate the screen properly.  Looking at the code:

if ((x < 0) || (x + w > vid.width) ||
(y + h > vid.height))
{
ri.Sys_Error (ERR_FATAL,”Draw_Pic: bad coordinates”);
}

We can see it pretty plainly.

Now since we were going somewhere I started to write some MS-DOS code, and switch out of the null set of mind!

First a simple VGA mode 13 setup which gives us 320×200 resolution with 256 available colours.  And for good measure I did a simple VGA palette setup that I knew worked from a prior program.  Next we just blit our buffer onto the screen, and we get this:

Quake II on MS-DOS first screen

Quake II on MS-DOS first screen

Which is exciting and disappointing at the same time.  I then took the palette code from DOS Quake and got something just as ugly.  I tried the code from OS/2. Same thing.  I tried all kinds of things and was going nowhere.

At this point Mara’akate added in the Linux clock code, and now we had this!

Quake II with a bad palette

Quake II with a bad palette

It wasn’t until much more digging around I saw some 320×240 screen setups that I realized there was something wrong there, and then I saw this gem in the linux port’s code:

/*
** SWimp_SetPalette
**
** System specific palette setting routine. A NULL palette means
** to use the existing palette. The palette is expected to be in
** a padded 4-byte xRGB format.
*/

In traditional VGA palette setups it’s 768 bytes that needs to be read/and pushed to the card.  I even checked Quake 1 is 768 bytes, but now in Quake II, it’s 1024 bytes!  OOPS!  Sometimes (ok a lot of times) you really need to check other ports or a ‘known good’ to see how they did things.

Quake II running on MS-DOS!

Quake II running on MS-DOS!

Pretty awesome!

So where to go from here? Obviously things like better keyboard input, the mouse, sound and networking need to be done.

Continued in pt2, pt3, pt4, and part5.

Announcing DJGPP 2.05 Beta 1

Yes, you read that right, not only is DJGPP still alive, but after the much stalled 2.04 beta cycle that has been going on for more than a decade, they have decided to push forward with 2.05 to get a release out there!

 

This is announcement of DJGPP 2.05 beta 1

It has numerous changes since previous DJGPP 2.04 beta 1 release in 2003.
(http://www.delorie.com/djgpp/mail-archives/browse.cgi?p=djgpp-announce/2003/12/06/22:18:05)
Unfortunately DJGPP v2.04 was never released and old beta version slowly
became almost unusable together with other newer DJGPP packages.

More information about changes in DJGPP 2.05 beta 1 is available at

http://www.delorie.com/djgpp/doc/kb/

both in sections about changes in 2.04 and 2.05. The same information is also
available in file info/kb.inf in djdev205.zip

It needs a lot of testing. Please test it if you have time and/or are
interested in any of the above features. Any level of testing would be
appreciated.

The beta is not available via the Zip Picker interface. You can download it
from here:

ftp://ftp.delorie.com/pub/djgpp/beta/v2

Additionally RPM packages (source and binary packages for i686 and x86_64) are
available from

ftp://ftp.delorie.com/pub/djgpp/rpms

Please see the README file for instructions on how to install the beta:

ftp://ftp.delorie.com/pub/djgpp/beta/v2/readme.1st

You can also download DJGPP 2.05 beta 1 packages from DJGPP mirror sites:

http://www.delorie.com/djgpp/getting.html

Thanks for all who have contributed to development of DJGPP

Andris Pavenis

I haven’t used it yet, but I see the following files in the beta directory:

djdev205 DJGPP V2 Development Kit and Runtime
djlsr205 DJGPP V2 Base Library Sources
djtst205 DJGPP V2 Test Programs (for testing the C library)
djcrx205 DJGPP V2 Cross-to-DOS Compiler Support Files (from djlsr/djdev)
djtzn205 DJGPP V2 Timezone Files
djtzs205 DJGPP V2 Timezone Sources

also floating around in the beta directory is GCC 5.1.0!

No doubt it’ll be a good excuse to rebuild and update some Quake related things to see how we fare in 21 century MS-DOS!

Impuse Tracker open sourced

I’ve never been musically inclined, but for those who loved the whole tracker scene of 20 years ago, Jeffrey Lim has released the source code to Impulse Tracker!  It features an impressive sound card support list:

  • Sound Blaster
  • Sound Blaster Pro
  • Sound Blaster 16
  • Sound Blaster AWE 32
  • Terratec EWS64
  • Hoontech Soundtrack 97 PCI
  • Gravis UltraSound, PnP, Max
  • Pro Audio Spectrum
  • Pro Audio Spectrum 16
  • Interwave
  • PC Speaker
  • DAC on LPT

It’s all in assembly and can be built with Turbo Assembler 4.1

You can find the source code on bitbucket here. Github mirror!

Netkeen

I’d never heard about this before, but a group reverse engineered Commander Keen, and re-built it as a network playable game!

Check it out at their site.

To build the source, you’ll need Borland C++ 5.0.2 and TASM 5.  Earlier versions may work but I haven’t tested.  The interesting thing with the networking is that they use the same code out of DOOM.

NetKeen

NetKeen

OpenWatcom v2

I know what you are thinking, wouldn’t it be great if you could create MS-DOS executables directly from a Win64 desktop with no MS-DOS needed?

Well, I just found out about this unofficial Open Watcom v2 project that targets the usual suspects, allows you to compile from Win64!

Hello World!

Hello World!

Some of the features of this fork include:

  • New 2-phase build system, OW can be build by platform native C/C++ compiler or by itself
  • Code generator properly initialize pointers by DLL symbol addresses
  • DOS version of tools now support long file names (LFN) if appropriate LFN driver is loaded by DOS
  • OW is ported to 64-bit hosts (WIN64, Linux X64)
  • Librarian support X64 CPU object modules and libraries
  • RDOS 32-bit C run-time compact memory model libraries are fixed
  • Resource compiler and Resource editors support WIN64 executables
  • OW text editor is now self containing, it can be used as standalone tool without any requirements for any additional files or configuration
  • Broken C++ compiler pre-compiled header template support is fixed
  • Many C++ compiler crashes are fixed
  • Debugger has no length limit for any used environment variable

Binaries are available on sourceforge.

So how does it fare?  I thought I’d take the old Wolf4GW, and compile it with this toolset.  The first hurdle I hit was this fun feature:

  • The C++ compiler now treats warning W737, implicit conversion of pointers to integral types of same size, as an error.

Which is an integral part of wl_menu.cpp .  So this was somewhat problematic, until I just commented out that block, and while I was expecting no working keyboard, I’m able to play, and load/save games…. Even the boss key works.

Wolf4GW

Wolf4GW

So with the W737 taken care of, I have to say this thing compiles FAST.  Incredibly FAST.  If for some reason you have to build 16bit or 32bit anything, you should look at a 64bit tool chain, well assuming you have a 64bit computer by now.

If anyone want’s to build their own Wolf4GW with the newer OpenWatcom, my source drop is here.

Today marks 20 years of FreeDOS!

From the original announcement:

A few months ago, I posted articles relating to starting a public
domain version of DOS.  The general support for this at the time was
strong, and many people agreed with the statement, “start writing!”
So, I have…

And how does it measure up?  Well the current release version is v1.1 and I tested it in both Qemu and VMWare Player.  First the installation process is SLOW.  Extraordinarily slow.  I really have no idea why, but 10+ minute installs in either environment were the best I could do.  The one thing I did do on installation was to disable a FAT32 drive, defaulting to the older FAT16 file-system.  Why? well because ..

Stacker 4 on FreeDOS

Stacker 4 on FreeDOS

Once installed, I thought I’d torture the environment, and install Stacker 4.0.  The installation went ok (I modified my system to boot up with option #3 as the default (Load FreeDOS including XMGR XMS-memory driver).  The installation went smooth, I setup a 300MB compressed drive, rebooted and the drive was not available.  I think this was because MS-DOS uses a config.sys file, and FreeDOS uses fdconfig.sys.  Although at the same time the autoexec.bat file was not modified as I had requested.  So I ran ‘config.exe’ and it walked me though additions to the c:\stacker.ini and what needed to be added to the (fd)config.sys (device=c:\stacker\stacker.com).  I did that, and rebooted, and I had my compressed D drive.

Then I installed DOOM v1.1 onto the compressed drive, and it runs perfectly fine! (is a screen shot really necessary?)  Seeing that DOS4G/W works (I also tested Descent, and Wolf4GW).  Oddly enough Wolf3d just hangs at a black screen, and Descent runs too fast although I think both issues are more specific to the games running on Qemu/VMWare than FreeDOS.  And if you want to skip the lengthy install, you can download my stock VMDK right here.

Seeing that 32bit DOS extenders seem OK, I thought I’d try TradeWars 2002 with it’s 16bit DPMI version, and it ran fine as well.

TradeWars on FreeDOS

TradeWars on FreeDOS

Now when it came to multitasking, Windows 3.0 won’t work outside of realmode.  Windows 3.1 however will run in standard mode.  386 Enhanced mode still doesn’t work correctly (is that even an issue)… So to use this for a BBS you’d need a network and some nodes… maybe fun for another day.  Excel 3.0c works under Windows 3.1 Standard mode, while Word 2.0c does not.

Another weird thing is that the FreeDOS partitioning program (fdisk) chooses a NON-DOS partition type by default, so if you were expecting to dualboot between MS-DOS and FreeDOS I think you need to install MS-DOS first, although I guess in the age of virtual machines does it matter that much?

For basic MS-DOS program compatibility it seems pretty much there, even with weird stuff like stacker.

MS-DOS Player updates

Poorly translated from TAKEDA toshiya’s blog..


2014/4/15
I has integrated source of i386 and i286 edition edition. 
In addition, in the i286 version, I added support for int 10h/16h. equivalent to 0.149 MAME, I was replaced with a 0.152 equivalent MAME core i386 i286 core. However, the i386 core, I have omit the TLB around.

Which is very cool, although I wasn’t sure about the MAME source code being open to other projects…?  I tried to contact the i86/i386 author vlinde but he then pulled his contact page.  I wanted to use i386 for something of my own, but the whole “Redistributions may not be sold, nor may they be used in a commercial product or activity.” really puts the damper on it.

I was able to get some simple XMS test program to run, but nothing of any substance.  No DOS4G/W or anything like that.  But if you re-build it specifying MS-DOS version 5.0, some of the MS-DOS utils and even command.com work!

The weird issue I had was running out of conventional RAM, because this program gives you nearly 1MB of conventional RAM… I was surprised, as I wasn’t expecting that much!

George R R Martin Writes with WordStar 4!

So apparently it’s all the news, The best selling series, Game of Thrones was written using Wordstar 4.  On a dedicated MS-DOS computer of all things.

Game of thrones..

What George probably sees..

Well isn’t that kinda cool.

As he says, he likes Wordstar because it doesn’t try to think for him, and he likes MS-DOS because there is no distractions.

“I actually like it, it does everything I want a word processing program to do and it doesn’t do anything else,” Martin told Conan O’Brien. “I don’t want any help. I hate some of these modern systems where you type a lower case letter and it becomes a capital letter. I don’t want a capital. If I wanted a capital, I would have typed a capital. I know how to work the shift key.”

He best says it back in 2007:

I do my writing on a completely different computer than the one I use for email and the internet, in part to guard against viruses, worms, and nightmares like this. My work machine does not even use Windows (which I loathe). I write with WordStar 4.0 on a pure DOS-based machine. Mock if you must… but WordStar and DOS are both stable as rocks, and never give me the sort of headaches I get from Windows. (I won’t even talk about Microsoft Word, about which I have nothing printable to say).

For anyone chasing Wordstar nostalgia, you can leaf through the manual.