Adding the missing part of DoomNew – audio

DoomNew, is a rather ambitious project by Maraakate, to attempt to revert the old linuxdoom-1.10 to something more akin to what shipped for DooM 1.9 using Hexen/Heretic source code to fill in many of the blanks in a very Jurassic Park like manipulation of it’s DNA (source code). It’s great and gives you a very cool MS-DOS based engine using the original Watcom tools. But there is always the one catch, which is that it relies on the original sound library, DMX.

And unfortunately, nobody has been able to get ahold of Paul Radek to see if he’d be okay with any kind of open-source license. So, sadly DMX has been a long-standing stumbling block for that ‘authentic’ super vanilla DOS DooM.

Enter the Raptor

Raptor: Call of the Shadows

Fast forward to a few days ago, and I come across dosraptor on github. I had a copy of this back in the day, it was bundled on CD-ROM or something. I am absolutely terrible at games like this, but I did remember this one being incredibly fluid, and fun despite me having no skill. Raptor was written by Scott Host, and it’s still on sale over on steam! The source had been cleaned up with help from skynettx, nukeykt and NY00123.

I went ahead and built it from source, and in no time I was up and running. I found Watcom 9.5 was the best path to go with. I even made a ‘release‘ for those who don’t want the joy of building from source, and of course picked up a copy on both steam and his site. While building the source code, and looking at the directory tree that’s when I noticed apodmx:

This is a DMX sound library wrapper, which is powered by the
Apogee Sound System, the latter being written by Jim Dose for
3D Realms. When used together, they form a replacement for DMX.

The DMX wrapper was written by Nuke.YKT for PCDoom, a DOS port
of id Software's Doom title from the 90s.

It also includes the mus2mid converter, contributed by Ben Ryves for
Simon Howard's Chocolate Doom port, as well as the PC Speaker frequency table, dumped by Gez from a DOOM2.EXE file and later also added to Chocolate Doom.

A few years later, this wrapper was modified by NY00123; Mostly to be built as a standalone library, while removing dependencies on game code.

So it turns out that Raptor used DMX, just like DooM!

Well, isn’t that incredible!

Now the first question I had, was apodmx a direct drop-in replacement for DMX? Well basically, yes! Let’s check out the Adlib driver!

DooM 1.9 intro
NewDoom intro

As you can hear, the intro is very different. But it’s playing at least. Ok how about E1M1?

DooM 1.9 E1M1
NewDoom E1M1

The Apogee Sound System is softer, and not quite the same as DMX, but compared to nothing I’m more than happy with it. The AdLib is kind of a weird card to drive, and I guess it’s not to surprising that there is such a variance.

How about a Roland Sound Canvas?

Nuked-SC55: Roland SC-55 emulation

Sadly, mine is inaccessible, but thanks to nukeykt there is the Nuked-SC55: Roland SC-55 series emulation. I had to setup the MidiLoop as expected, and configure DosBox for the Loop and now I have a virtual Sound Canvas. So let’s see how the two engines deal with a common instrument!

DooM 1.9 Sound Canvas 55
NewDoom Sound Canvas 55

Pretty cool, if I do say so myself.

ZeeDooM

I’ve uploaded my modifications to github, along with a copy of that old ZeeDooM I had slapped together ages ago. I’d taken the map source code from Romero, and the graphical/audio resources from Freedoom and slapped them together.

zeeDooM Xenix!

So a few months ago, gattilorenz had told me he managed to work out how to do direct video under Xenix, and was able to get DooM running! He made the source available, and I meant to do something back then, but I must have gotten distracted.

So I went ahead and added the zeeDooM thing I had been working on a while ago which is a combination of the John Romero released maps, and the FreeDoom assets. So it’s not 100% the released version, it just looks that way.

I’ve gone ahead and created a qemu disk image (if you convert it, it does run on VMware, so probably far more 386 capable emulators than I can imagine), along with an old Qemu exe for some stand alone fun on archive.org.

There is no sound, nor music. I should look closer one day, and see if I can drive some direct music to an Adlib since it’s just IO ports, and maybe grab all the direct sound code from viti95’s FastDoom?

Anyways, it’s zeeDooM!.. on Xenix!

How much DooM is there in the Doom 3: BFG Edition version?

Another follow up to my ages old “Just how ‘original’ is the Ultimate Doom on steam?“, I thought I’d follow up with the DooM 3 BFG version.

I already have DooM 3, but this is apparently remastered, and includes the previous versions!?

It’s not like it really offers anything I don’t have, but it’s on sale so whatever let’s go.

And in the base/wads directory there they are!

fb35c4a5a9fd49ec29ab6e900572c524 DOOM.WAD
c3bea40570c23e511a7ed3ebcd9865f7 DOOM2.WAD

And sure enough the are the latest versions of the game files to be found according to doom.fandom.com. Great! So to further the abuse I tried them under my mutilated DooM.

Ultimate Doom seems to work just fine on it’s own I tested it briefly warping to a few levels but yeah it just works! Doom2 however bombs out that the resource TITLEPIC is missing from the wad. How disappointing!

Naturally I just took the easy way out, and basically checked for the resource, and load another if it’s missing.

@@ -477,7 +477,11 @@
  else
  pagetic = 170;
  gamestate = GS_DEMOSCREEN;
- pagename = "TITLEPIC";
+ /* the Doom 3 BFG EDITION version of Doom 2 is lacking the titlepic */
+ if(W_CheckNumForName("TITLEPIC")>0)
+ pagename = "TITLEPIC";
+ else
+ pagename = "DMENUPIC";
  if ( gamemode == commercial )
  S_StartMusic(mus_dm2ttl);
  else

It’s a shamefully basic patch. But it works.

Another interesting thing is that DooM 3 BFG also includes the gravis ultrasound bank data, so you could load them up into some other emulator and enjoy that gravis experience. I don’t know if it’s licensed or what, but it’s a nice touch.

AVGN reviews Chex Quest

Okay that was funny. I never thought of even trying Brutal DooM + Chex Quest. Sounds awesome.

Although I’d played a little with Chex Quest before, I never tried it on the DooM source. Oddly enough it’s Ultimate DooM. In d_main.c you can do some simple test for chex.wad and pass it off as the ‘retail’ version.

	if ( !access (chex,R_OK) )
	{
		gamemode = retail;
		D_AddFile (chex);
		return;
	}

Or you can simply just rename chex.wad to doom.wad or doomu.wad. Many of the strings for DooM are compiled into the EXE, not taken from the WAD file (although it could have been, I guess it was to prevent people from making overtly cheeky mods?).

So firing up the wad under my crappy DooM port thing to MS-DOS (for the sane people just use some other Win64/OSX/Linux thing like zdoom), and when selecting an episode you’ll see the Ultimate DooM levels.

Chex Quest, Thy Flesh Consumed!

Wait? What? I though Chex Quest was a ‘total conversion’ WAD. Well it turns out that it is, and it isn’t. They replaced a lot of the default stuff from a retail version of Ultimate DooM. And what they didn’t replace, well it’s still there. And yes that does mean everything outside of the first 5 levels are the original DooM levels. And that includes the music as well!

E2M2 from Chex Quest

Well isn’t that surprising! And yes that means that it’s possible to just replace the first 5 levels with the default DooM levels and have that reverse conversion. In the same way the menu screens are very Chexy too:

It certainly gives that kid like feeling to it. Although the replacement Barrons of Hell are a little too big so they do look kind of silly.

E1M8

So as always I’m late to the party. I’m sure someone out there didn’t have the retail version of DooM and instead used Chex Quest for those LAN games. Although it does detect that the WAD has been modified so I don’t think it would just be all that fine.

Not that finding the original WAD files, or source to the maps, and just compiling them yourself is all that difficult, but I guess it is something else to load up.

Imagemagick really is.. Magic!

Ok, so since I’ve been playing around with the Freedoom assets, I wanted to process all the assets and then make them into an iwad.  And for the graphics this meant generating a simple color cube palette and then transforming all of the images to match that palette.  And the results were, while recognizable as DooM, they are drab.

Gray world

And yes, it’s gray, and drab.  So UK.  And there is another problem, many of the ‘graphics’ assets were a mix of PNG and GIF, and it turns out that in the GIF format you usually have a single color set as your transparent color, and it’ll get cut out automatically.  In this case the transparent color is cyan.  However there is some cyan still in the image!

Cyan artifact

So the best way I figured to ‘fix’ this was to do a straight conversion using Imagemagick.  So I loaded up paintbrush of all things and noticed that for some reason the colors had bled on the gif’s I had from the Freedoom pack I’d downloaded.  And that there were actually 3 cyan colors that needed to be purged.  In this case in hex they are 0x00fefe, 0x00f2f2, 0x02f6f5, and the one that they should have been, 0x00ffffff

convert ..\graphics_freedoom\old\wia20200.gif -transparent #02f6f5 -transparent #00fefe -transparent #00f2f2 -transparent #00ffff oldpng\wia20200.png

So I had to run convert like this against all the GIFs that I needed to fill in the graphics that I’m currently not processing in Python.  Now the images actually look right, no surprise cyan, but my palette still sucks

Although the Freedoom team has told me that it’s far easier to just use the DooM palette as their assets use that palette and it’ll just work.  But I’m too stupid for that.  One great feature of Imagemagick is that you can hand it an image, and ask it to reduce it to any arbitrary number of colors, and it’ll do a great job of it.  And while that is great for a single image, that doesn’t help me when I’m talking about thousands of images.  Except Imagemagick also has another great ability which is to paste a new image to the right of an existing one.  So with a little creative use of the make command I can then build a single giant image that contains all of the artwork.  Isn’t that great?

Although great care and detail went into the original DooM palette selection we can throw all of that away, and let a program stitch everything together, and then have it analyze the entire mess, and come up with the ‘ultimate palette’ that works best with everything.  Although one word of warning it takes well over an hour on an i7 to just stitch the images together (I should have setup a RAM disk) but it only took about 10 minutes for Imagemagick to process the blob image to come up with 255 colors that work best across the entire image set.

With the reduction done, the next thing to do is to create a ‘palette image’, which is one pixel for each color.  This is the palette that we will use to ‘reduce’ all the images against.  It’s more so to let Imagemagick do the hard work of selecting a palette.

convert slug256.png -unique-colors -scale 100% slug256_table.png

Imagemagick optimal palette

And the next thing is to extract the RGB set, which DooM uses for it’s palette.  In this case each color is is represented by three bytes.

convert -size 256×1 -depth 8 slug256_table.png -append rgb:playpal-base256.lmp

And then the next step is to process this palette with dmutils dcolors.  While it is primarily designed to use the LMB file format from Amiga fame, it wasn’t too hard to modify it to read a palette file directly, and let it add in the ‘red pain’ color shift, along with the green ‘bio hazard suit’ effect.  The color map it generates is totally corrupt at the moment, so I’m using the old perl program to generate one based off of the palettes.

Indeed it is something so crazy that I really don’t want to even do it a second time to make sure my process was reproducible.  However compared to before, I think the results speak for themselves:

new palette

So while it does take the better part of forever to go through all the images like this, it certainly gives zeeDoom a little more ‘building the world from scratch’ type feel.

 

Putting the 2015 Romero DooM level release to use

or how I learned to love Freedoom.

E1M1

So back in the distant past of 2015, John Romero had released a bunch of assets to DooM, including the source to the maps. And like a fool I never really did anything with them.  But I decided to do something about that.  Obviously just compiling a map isn’t anywhere near good enough, you need graphics, music and sound right?

Enter Freedoom

Now what is great is that the Feeedoom folks have had the whole ‘apple pie’ stance to their project, with everything included.  However they have drifted tools, build processes and other methodologies through the years. But thankfully the full archives are online, so I could go through and piece stuff together to my liking.  Of course this all needs various tools, and oh boy does it ever need tools.  You need:

  • Perl
  • Python
  • a C pre-processor
  • ImageMagick
  • idbsp10
  • deutex-4.4.902
  • midi3mus
  • sox-14.4.2
  • A Unix’y build environment, I used the ancient MSYS

And probably many more I’m forgetting.

The first was to compile the levels.  I used Ron Rossbach’s ancient IDBSP, a C port of iD’s command line Objective C level compiler.  I did run into two slight issues, the first is that Ron’s tool expects their to be a line in the level’s dwd file telling us the name of the wad it’ll compile to, which of course is missing.  The other is that I suck a makefiles, and just cheated forcing the tool to use a .wad extension on whatever .dwg it converts.  And with that out of the way, I had the levels all built.

However that is where I found out the hard way that Freedoom doesn’t target something like my goal of being able to run this wad with the original DooM v1.1 release.  The first problem is that Freedoom uses a different palette set, where it looks like it should be deeper?  I’m not sure, but one thing was for sure everything looked like a rainbow sea of wrong and any vanilla or chocolate engines.  And that is where I got a fun chance to play with ImageMagik.  It really is quite powerful.

The first problem of course is the palette.  Buried in the Python build scripts is a Perl fragment for generating a palette, and the all important playpal.lmp & colormap.lmp. I also got to spend a lot of time trying to extract that palette and do something useful with it to no avail.  But googling led me to VGA_palette_with_black_borders.png, on Wikipedia of all things. So using this as a base I could convert, dither, and re-map the higher color Freedoom artwork into something that would play nice with Vanilla Doom.  The downsite is that I didn’t try to find out exactly what assets a DooM version 1 wad needs, so I converted them all.  On a good Xeon or i7 it takes about 10-20 minutes all the images.  I can’t even think about how slow this would be using a 486, 68040 or MIPS.

ImageMagick-7.0.7-18-Q16/convert ../graphics_freedoom/wia00001.gif -dither Riemersma -remap ..\playpal\VGA_palette_with_black_borders.png wia00001.gif

Imagine running that some 1,558 times.

Vanilla DooM v1.1 only plays audio at 11,000 Hz, 8bit deep.  All the later ones needless to say use better sampling, and once more again Freedoom was at a higher level.  I used sox to re-sample all the audio into the lower frequencies.

sox “../sounds_freedoom/dsslop.wav” -b 8 -r 11025 -c 1 “dsslop.wav”

Later engines also are capable of directly playing MIDI files, and taking advantage of OPL3 chips, instead of v1.1’s OPL2 level. Thankfully Natt’s midi3mus is still online, and I was able to use this to convert Freedoom’s MIDI into the more restricted MUS format.

midi3mus ../musics_freedoom/d_map11.mid d_map11.mus

The last tool is deutex, which is not only capable of building iwads, but also decomposing them.  I ended up having to add the ‘musid’ flag to force it to honor the older sound format.

deutex -v0 -fullsnd -rate accept -rgb 0 255 255 -doom2 bootstrap/ -iwad -musid -textures -lumps -patch -flats -sounds -musics -graphics -sprites -levels -build wadinfo_ult.txt wads/doom.wad

But because I wanted to verify my build against DooM v1.1 I also had to address one other thing, which is the status bar for DooM version 1.1, which I had patched around in my lame source port. I was able to just decompose the shareware 1.0 wad, and extract the following:

  • stabarl.bmp
  • stabarr.bmp
  • starms.bmp
  • stbar.bmp
  • stmbarl.bmp
  • stmbarr.bmp

And injecting them into the final wad gives me something that the Vanilla DooM  v1.1 engine can actually run.

I’ll put up something online later, and a video of me playing, I guess.  It’s a weird Doom, I mean it is the “normal” DooM, just looks different.

As suggested it’s now called zeeDoom. I’ve uploaded the project, some of the 3rd party programs to build the project, and of course the built wad file.

Download zeeDoom

And as mentioned a play through E1M1 video.