Rebuilding Darwin from source: Part 3 Debian makes the world go ’round

In the previous posts, we’ve gone through the excruciating fun of installing Rhapsody DR2, and of course built the Rhapsody kernel from source. But now it’s time to build the software that can build software.

Enter the apt*

Of course we can’t just start building apt, rather we need to start at the 1990’s super star scripting language that revolutionized massive, shared code libraries, accelerating web development, and building the modern web, of course I’m talking about PERL.

Even back in the original effort building Perl was a slog. Even with temporary wins with miniperl it quickly fell apart from missing symbols. When it comes to the system libraries Darwin is not complete, rather it’s a lot of amputations from OPENSTEP. Which of course, itself was amputated from NeXTSTEP. I’m not sure what held back NeXTSTEP from being ‘open’ back when ‘open’ meant published specifications, not open source, or open in any other sense of the way, like The Open Group being a gatekeeping organization that is NOT open at all.

Anyways, Perl from the Darwin 0.1 downloads & the 0.3 CD-ROM don’t build. I gave up and moved to the OS X Server version and that one did build! As much as I could diff them out and find the breaking difference, honestly, it’s just easier to stick with what works.

/pub/Darwin/PublicSource/Darwin

I should point out that the source to Darwin was preserved on this now defunct site “next.68k.org”, which in turn was also preserved on the defunct site “nextftp.onionmixer.net“. Amazing how mirrors go. Other fun things on the ftp site include MacOSX-Server public sources, which did include the Perl that we built.

Darwin 0.3

Darwin 0.3 however was pressed onto CD-ROM, and distributed out to the masses. It took me a short while to get a tip to a hidden server that had a copy, which really was a massive breakthrough as it had a far more complete set of files than the 0.1 ftp dump. However at the same time there are files in the 0.1 dump that are not in the 0.3. Was there ever a 0.2? I have no idea, the mailing lists don’t seem to have been preserved so I really don’t know. Does anyone have any other ftp site archives? Not to my knowledge but I’d be more than happy to be wrong.

With a working Perl the next thing to do is to patch the buildtools & dpkg to not be PowerPC centric. It’s no secret that all the official effort going on was to get OpenSTEP up and running on Apple PowerPC’s and to transition away from OPENSTEP to Rhapsody, the MacOS 8 Platinum feeling type OS, where everyone was going to love the ‘Coca’ API, and dump the old MacOS stuff or be forced to run it in the ‘BlueBox’ MacOS 8 emulator. Obviously this future didn’t happen as developers were not interested in rewriting for Steve’s decade+ fever dream of a Unix for the ‘rest of us’, instead they wanted their existing software to ‘just work’ and the Carbon API had to be created, along with a drastically different and modern looking OS X.

-    $flags->{'RC_CFLAGS'} = '-arch i386 -arch ppc ' . &liststring (@cflags);
-    $flags->{'RC_ARCHS'} = 'i386 ppc';
+    $flags->{'RC_CFLAGS'} = '-arch i386 ' . &liststring (@cflags);
+    $flags->{'RC_ARCHS'} = 'i386';

Frequently in the build tools it’s a matter of looking for ppc/powerpc and replacing them with i386. It’s really no surprise that Darwin always built on intel, and it had to, as it’s life depended on it. Back when NeXT hit their first real big stumbling blocks of being a vertical platform is that they just couldn’t compete in the hardware space. But dumping the 68k based black boxes, they could now take their software and port it to the much more coveted RISC platforms, and shipping with NeXTSTEP 3.3 they supported both SPARC & HPPA. There had always been talks of further platforms like MIPS or DEC Alpha, but these never materialized, much like the i860 which had been relegated to a simple Postscript co-processor.

Anyways.. Keeping with yesterday’s setup, and of course the .darwin-builder-04-23-2024.iso CD-ROM with all the stuff we need, let’s DOIT!

phase 2 completed

With phase 3 completed we are now ready to build the rest of the system. I hope you are excited! As I’m sure hoping you kept the original disk layout from the prior setup, or I’ve totally trashed your system by now.

I should say that deb files are just specially tagged ‘ar’ archives, that contain a data & control files telling apt how to process them. In this case I’ve taken the cc_783.1-1_i386-apple-darwin.deb file and modified it to contain the OS X 10.0 modified CC compiler. Apt had stumbled on building it, and I’m not interested into troubleshooting why or how. Basically, use ar to extract the contents of the deb, then tar to expand the data, replace the files, tar to put the files back into the data tar file, and ar to rebuild the archive.

ar r cc_783.1-1_i386-apple-darwin.deb debian-binary control.tar.gz data.tar.gz

In this case, debian-binary is a text file that simply contains ‘2.0’. Amazing!

The first thing to do is build a manifest of what needs to be built. I just simply extract all the ‘fixed’ source that I’d used last time to build Darwin, apply patches were needed, and then kick off the process with a darwin-buildall.

ls -l | awk '{print "dir /usr/src/"$9 " all"}' | tail -n +2 | grep -v gdb |grep -v cc- > /tmp/manifest.txt

In this case I skip building the C compiler, as it takes too long, and I’ve already done it. If you really want to do it, you can do so at your own leisure. GDB has issues building, and I haven’t even begun to tackle them. As you can guess the format of the manifest is pretty simple:

dir /usr/src/CoreOSMakefiles-1/ all
dir /usr/src/Csu-1/ all
dir /usr/src/Libc-1/ all
dir /usr/src/LibcAT-1/ all

Debian uses deb’s to populate a fake root directory in order to cross compile the packages. This is like installing multiple copies of the operating system, and that is why we use a separate scratch disk. This can consume several gigabytes when it’s done.

Also this presented the chicken/egg problem with how do you make debs from a system that needs debs? Thankfully NCommander had done extensive work with Debian / Canonical and was able to fake enough of a ‘build-base’ fakery that satisfied the build system just enough to start building stuff. In this weird way all roads lead back to the first build-base. Thankfully we live in the future where VMs are fast, and virtual disks are cheap.

I then create the /built directory, where the compiled deb’s will be populated, and copy in my modified compiler Debian into the built directory so that it’s used in all the compiling. On the CD-ROM I have 2 selections of deb files, the ‘deb’ directory from when I’d originally done this back in 2017, and the ‘fresh’ directory that I’ve just built. You can always manually source where the debs some come.

Kicking off the build is as simple as running:

darwin-buildall /tmp/manifest.txt /source/fresh /built

This will take.. a while. It’s a lot of files to copy & expand, and compiling takes a fair bit of a toll on the olde CPU.

By default, 118 of the 127 can be built.

  • boot-2 the sarld won’t compile as.. there is nothing to compile. I’m lost. Also some driverkit headers didn’t make it into the packe!
  • cvs-1 is upset about bison grammar not being in usr local lib?!
  • flex is also upset about bison grammar locations.
  • libgpp ppc/ppc-nextstep/_G_config.h missing?!
  • perl.. should be patched __environ vs _environ
  • bootstrap-cmds “multiple definitions of symbol _migcom_untypd_VERS_STRING”
  • volfs seems plain broken but subdirecotories okay?
  • netinfo missing netinfo.defs and headers?! arpa/nameser.h missing (can just touch)

That just leaves AppleTalk & HFS not building. Which I believe is period correct.

The good news is that the kernel that was built boots up seemingly fine.

Rhapsody Kernel 5.5

The NeXTSTEP, of course is to now setup a new disk image, and see what is involved in booting up!

Rebuilding Darwin from source: Part 2 Building the kernel

Re-creating the steps from 7 years ago the first phase was to build the Darwin kernel. Like everything else, once you know what is involved, it’s not all that difficult. But as always finding out the steps to get there is half the fun!

I’m going to assume if you want to follow along, that you’ve completed the first part of this exercise, and you have a Rhapsody DR 2 system up and running. Due to some issues I’ve had with creating a lot of files & filesystem corruption, we are going to create and add two more disks to the system. On Qemu we need to add them via the CLI:

qemu-img create -f vmdk source.vmdk 8G
qemu-img create -f vmdk scratch.vmdk 8G

Adding them to the command line gives us something like this:

qemu -L pc-bios -m 512 ^
-k en-us ^
-rhapsodymouse ^
-hda rhapsody.vmdk ^
-hdb source.vmdk ^
-hdd scratch.vmdk ^
-cdrom darwin-builder-04-23-2024.iso ^
-fda nic.flp ^
-net nic,model=ne2k_pci,vlan=1 ^
-net socket,udp=127.0.0.1:5001,remote=127.0.0.1:5000,vlan=1 ^
-boot c ^
%1 %2 %3 %4 %5 %6 %7

Additionally you’ll also need to download the current ‘darwin builder’ ISO that I’ve put up on sourceforge. As of today it is darwin-builder-04-23-2024.iso

Step one is to boot into single user mode. As we need to prep & format the disks under Darwin before the system starts up.

We need to check the hard disk, and then create the device names for the third hard disk.

fsck -y /dev/rhd0a
mount -w /
cd /dev
./MAKEDEV hd2

Now we need to run the ‘disk’ command which will abstract the whole volume creation. There are numerous flags, but we don’t need all that many.

disk -i -l 'src' /dev/rhd1a
disk -i -l 'scratch' /dev/rhd2a

The output scrolls off the screen, so I didn’t capture it, but you’ll see all the inodes being created, it’s a lot of output!

With the disks created, we can now shut down the VM

shutdown -h now

and then restart Qemu, and let it boot up normally. We’ll get to the login screen, login as the root user.

The first thing I’d recommend is to drag the Terminal.app from /System/Administration to the desktop to make it easier to get to. Rhapsody, unlike NeXTSTEP & OPENSTEP doesn’t have any dock, as the goal back then was to make Rhapsody look and feel more like Platinum MacOS.

The next thing to do is to make the system very insecure by allowing remote root logins. It’s just easier to deal with. You could use sed or just copy the one I provided from the CD-ROM.

cp /source/ttys /etc

And with that in place, its easy enough to telnet into the VM so you can copy/paste stuff in and out with ease!

You should now be able to verify that all 3 disks are mounted:

# mount | grep hd.a
/dev/hd0a on / (local)
/dev/hd1a on /src (local)
/dev/hd2a on /scratch (local)

From here it should be very simple to kick off the build process:

/source/phase1.sh

And this will kick off the build, recreating all the fun steps I’d gone through so many years ago. These projects now are building in the following order:

  • kernel-1
  • driverkit-139.1-1
  • cc-798
  • bootstrap_cmds-1
  • objc-1

The first phase of the script will unpack both the kernel & driverkit and install their respective header files into the OS. NeXT a bunch of symlinks are created to link the system to the driverkit. Next I decided to build the ObjectiveC compiler from 10.0, hoping it’s more bugfixed and slightly more optimized than what was available back in 1999. Building the compiler is a little involved, as a good GCC tradition is to be cross compiled first, then re-compile itself with itself, then do that again and verify that the 3rd recompile outputs the same as the second one. Yes it’s a thing. Yes it’s slow. Yes you are lucky to live in the future, this was really painful back in the day.

With the kernel compiled, we can then compile the bootstrap commands, and the objectiveC runtime that is used by the kernel. Nothing too exciting here.

DriverKit however….

The PCMCIA code was not included in any of the 0.x Darwins, so for laptop enthusiasts you are basically SOL. As a matter of fact, a lot of weird stuff was pruned out, that either could be ‘touched’ or borrowed from the PowerPC port and massaged into place. Luckily I had at least figured out a simple fix for PCIKernBus.h so at least PCI works.

Likewise for the kernel, there was some guessing on the EISA config, which also overlaps ISA, along with having to remove the PCMCIA cardbus .. bus.

APM crash

I had issues with the APM (Advanced Power Management), another laptopisim I suspect. I had to amputate that.

for testing purposes

Naturally the cpuid code is broken much like early NT (I wonder if both were contributed by intel?), so it doesn’t detect any half way modern Pentium processors correctly which causes it to fall all the way back to the i386, which unfortunately, Rhapsody is compiled as 486 (remember NeXTSTEP had fat binaries allowing you to recompile for different processors and ship a single binary that can be ‘lipo’d into the appropriate one for the host). So being degraded to a 386 means nothing works.

bad CPU type in library!

yay.

Luckily patching the cpuid was pretty simple just force it always to be a Pentium. It is 1999 afterall.

I’ve done my best to make this a single script to run, and all being well you’ll get something that looks like errors, but it should be fine?!

System/Library/Frameworks/System.framework/Versions/B/Headers/bsd/i386/table.h
System/Library/Frameworks/System.framework/Versions/B/Headers/bsd/i386/types.h
System/Library/Frameworks/System.framework/Versions/B/Headers/bsd/i386/user.h
System/Library/Frameworks/System.framework/Versions/B/Headers/bsd/i386/vmparam.h
private/
private/dev/
private/dev/MAKEDEV
private/tftpboot/
private/tftpboot/mach_kernel
mach_kernel
tar: private/dev: Could not change access and modification times: Permission denied
tar: private/dev: Cannot change mode to 0755: Permission denied
tar: private/dev: Cannot chown to uid 0 gid 0: Permission denied
tar: Error exit delayed from previous errors

A quick look around shows that there is tgz files indicating that things have been compiled. I did backup the old original kernel as “rhapsody-gcc.tgz” in case you ever need it. Can’t imagine why but who knows?

qemu:13# ls -l /usr/src/*.tgz
-rw-r--r--  1 root  wheel   173706 Apr 23 15:25 /usr/src/bootstrap_cmds.bin.tgz
-rw-r--r--  1 root  wheel  2184460 Apr 23 15:33 /usr/src/cc-798-bin.tgz
-rw-r--r--  1 root  wheel  2747289 Apr 23 15:36 /usr/src/driverkit-kern-bin.tgz
-rw-r--r--  1 root  wheel  1264957 Apr 23 15:26 /usr/src/kernel-driverkit-hdrs.tgz
-rw-r--r--  1 root  wheel   116343 Apr 23 15:26 /usr/src/objc-bin.tgz
-rw-r--r--  1 root  wheel  2173005 Apr 23 15:26 /usr/src/rhapsody-gcc.tgz
qemu:14# ls -l /mach_kernel*
-r--r--r--  2 root  wheel  1459520 Apr 23 15:36 /mach_kernel
-r--r--r--  1 root  wheel  1404116 Apr 23 15:38 /mach_kernel-rhapsody

You should now be able to reboot into the kernel that you’d compiled!

Next up is Phase 2, where we compile the tools to enter the dark magic that is the Debian build system. Yes, you read that right, Apple/NeXT was all in on Debian.

Rebuilding Darwin from source: Part 1 Qemu

Back in the old old..

7 years ago!

It’s hard to believe it’s been 7 years ago since I reproduced Ncommander‘s adventure in building the Mach kernel from Darwin 0.1 sources that had been found years ago. At one point we’d managed to build enough of Darwin to do a dump/restore onto a new disk image, and have a mostly built Darwin system save for a hand full of files.

Time goes on by, and memories fade, and I thought it’d be worth going over the adventure, yet again. Just as it was true back then, I thought I’d reproduce the same setup that I’d been using back then. Qemu was a new and exciting thing back then, and

the Disk driver is VERY picky and honestly ancient Qemu is a pretty solid option to emulate NeXTSTEP/OPENSTEP/Rhapsody with some patches to both 0.8 & 0.9 by Michael Engel, which change a nested interrupt and add support for a busmouse, as the PS/2 mouse doesn’t work for some unknown reason. I know many are scared of old Qemu, but the disk support is pretty solid and the CPU recompilation is very fast, so having to rely on MinGW v3 isn’t so terrible.

While I had hid away a lot of these resources on archive.org, I thought it was best to just go back to the oldest post I had where I had painfully documented how to compile Qemu and get it working with NeXTSTEP, back on BSDnexus. I’m so glad I took the time so long ago to not only write it down, and add screenshots, but also tag the version numbers. Software drift, especially free software can be so difficult to pin down, and it’s nice to be able to return to a known good value. I went ahead and placed the recreated toolchain over on archive.org.

Rhapsody is a weird OS, in that NeXT had kind of given up on the OS market after their NeXT RISC Workstation had basically died with the 88k, and even their early abandoned PowerPC 601 aka MC98000/98601 port. Apple had left a few of the changelogs, in the source code. It’s very interesting stuff! I guess to go off on my own tangent NeXT was just too early, the cube with it’s Unix & magnetic optical media and great audio DSP capabilities was just too ahead of the curve. What the cube couldn’t pull off in 1988, the iMac and OS X sure did in 2001.

I also added UDP support to this Qemu so I could use the HecnetNT bridge trick giving me the ability to telnet/ftp into the VM greatly reducing my pain. Back in the day I had used NFS and the network slirp redirection. But I like having direct access so much more!

Rhapsody throws up yet another fun ‘road block’ in that the mouse buttons map backwards for some reason. It’s a trivial fix in the source code, but I made it a runtime option in case I needed or wanted to run NeXTSTEP. And it was a good thing, as I did need to find the NXLock.h file for building one part of Darwin.

When building Darwin, I started with the last x86 version of NeXTSTEP that was availble, and that was Apple ”Rhapsody” / Titan1U x86 Developer Release 2 from WinWorld. My thinking at the time and still is that the closer you can get your build to whatever they were using as ‘current’ the easier this will be.

Rhapsody can support an 8GB disk, so let’s go with that. This always has an issue with people that try much larger, and just fail, so for my sake and yours let’s just go with 8.

qemu-img.exe create -f vmdk rhapsody.vmdk 8G

And to simply start off the installer:

qemu -L pc-bios -m 128 ^
-k en-us ^
-rhapsodymouse ^
-hda rhapsody.vmdk ^
-cdrom rhapsody_dr2_x86.iso ^
-fda rhapsody_dr2_x86_InstallationFloppy.img ^
-boot a

You may be wondering, why only use 128MB of RAM? Well there is a bug in the shipping Rhapsody kernel that prevents booting on machines with more than 192? MB of RAM. Naturally, once we are to the point of building our own kernels this won’t be a problem but for now we are limited.

The bootloader will give us a few seconds to do anything fancy, I just hit -v so I always have the verbose boot.

From here it’s just a few options to go thru the installer

And a disk change is required

CONTROL+ALT+2 will bring up the monitor prompt, where we can change the disk

CONTROL+ALT+1 will return us to the console. Now we have to go through all the SCSI cards, and kind of compatible IDE cards

Further..

Further still…

And how select the Primary/Secondary(DUAL) EIDE/ATAPI Device Controller (v5.01)

We only need the one driver, so we’re good to go!

Continuing onwards will now start the kernel, along with a change to graphical mode. Just like a NeXT!

Now we can confirm again we want to install

As you can see there is our hard disk!

In the future we don’t need to dual boot so, just give it the entire disk.

A few more 1’s and we are finally installing!

Trust me it’s fast on Qemu!

And just like that, we’ve completed the first part of the install.

You can use CONTROL+ALT+2 to toggle back to the monitor and type in

eject fda

to eject the floppy, then it’s CONTROL+ALT+1 to return to the display and have it reboot. Qemu won’t try to boot to the hard disk, and with no disk in the drive, it’ll hang at the BIOS. Now is a good time to close Qemu and backup the hard disk. Mostly because I hate repeating this stuff.

To try to make this a bit easier, I’ve made a floppy disk with the NE2000 driver and updated kernel in one place where they can be injected via single user mode. A simple enough config file:

qemu -L pc-bios -m 128 ^
-k en-us ^
-rhapsodymouse ^
-hda rhapsody.vmdk ^
-cdrom rhapsody_dr2_x86.iso ^
-fda nic.flp ^
-boot c

Type in -s for SINGLE USER MODE. This is where a lot of Unix problems got solved in the old days

It’s a little tricky as this does involve typing. As instructed we need to check the hard disk prior to mounting it read/write

run the commands:

fsck -y /dev/rhd0a
mount -w /

Now we can mount the floppy disk

mkdir /mnt
mount /dev/fd0a /mnt
ls /mnt

I’ve included both the kernel & NE2000 driver in one file, and JUST the NE2000 driver in the other. For my sake I use the first one, update.tar.gz as I wanted to use the newer kernel ASAP. tar -zxvf didn’t want to run, so I did a rather awkward version to achieve the same thing.

cat /mnt/update.tar.gz | gzip -dc | tar -xvf -

With the files in place, you can now shut down the system with a simple

shutdown -h now

Once more again, I’d shut down the emulator, and backup the hard disk. If anything goes wrong you can restore your backups at any phase, at least saving some time!

Next is the graphical install. In this case I use the hecnet bridge to give full access, you can setup with the slirp driver with a simple “-net user” but it doesn’t matter at the moment Since I opted for the newer kernel, I can take advantage of the 512MB RAM.

qemu -L pc-bios -m 512 ^
-k en-us ^
-rhapsodymouse ^
-hda rhapsody.vmdk ^
-cdrom rhapsody_dr2_x86.iso ^
-fda rhapsody_dr2_x86_InstallationFloppy.img ^
-net nic,model=ne2k_pci,vlan=5 ^
-net socket,udp=0.0.0.0:5001,remote=127.0.0.1:5000 ^
-boot c ^

Booting Rhapsody

Now we have to setup the hardware. It should be somewhat straight forward, first we start with the monitor. The mouse should be working although I find that I have to move slowly. Sorry it tracks weird on real hardware too.

The Cirrus Logic GD5446 should be detected automatically, I just go with the default 1MB

Next under the mice, select the PS/2 mouse and remove it

This leaves us with the Bus Mouse driver.

Under the network tab, the NE2000 should be automatically detected.

On the last tab, I make a habit of removing the Parallel port freeing IRQ 7, in case I wanted it for something else.

Back to the summary, we now have Cirrus Logic video, NE2k networking, with no SCSI, no Audio.

With the config saved, now we can just install as is. I un-installed Japanese, but it doesn’t matter. You absolutely need the Development Tools, so may as well go with eveything.

The installation only takes a few minutes and we are ready to reboot

The kernel will now shut down.

Once more again this is a great time to make another backup of the hard disk. At this point this is a great backup to save, as we’ve installed the OS, and selected drivers, in the next reboot we’ll be personalizing the operating system.

We can re-run the last config once the disk has been saved. We’ll be greeted with a message that the Server isn’t responding. Answer YES to continue without networking.

From here we are in the Setup Assistant, taking a nod from MacOS 8

I use USA keyboards. The best keyboards! I touch type so I don’t have to deal with weird layouts.

Specify for a LAN connection

There is no DHCP support so specify a manual IP address.

The default stuff is just wrong.

On my LAN this is good enough. DONT add a router. It’ll just confuse it.

We don’t have or want NetInfo. This would be the server to give out IP addresses, and authentication. We don’t need it!

Leave the DNS servers blank

You can setup any location, it doesn’t matter.

Likewise, with NTP, turn it off.

We will then get the option to set the date/time manually. The default time is far too old and it’ll break stuff.

Next up is user accounts.

I always make local accounts.

Come up with some creative name

and a password

I don’t have it doing any automatic login. You can if you want, it’s all up to you, this one doesn’t matter.

Next is an Administrator password

And now we can apply our changes.

It takes seconds!

And now we’ere done. You may want yet another backup as this is tedious!

With the final configured backup in hand, now we can boot back one more time, and now we’re in SVGA mode!

Rhapsody DR2 Login Screen

Logging in as root will give me the desktop

Desktop

And there we are, all installed.

For anyone brave enough to have read all of this, but wants the quick and easy version, it’s up on archive.org!

In part two I’ll pick up with the source CD-ROM I’ve prepared, so we can start compiling Darwin!

Looking at UnixWare 7.1.1 on VMware & Qemu yet again!

Same old OS, same old problems.

My UnixWare 7.1.1 box

The UnixWare 7.1.1 install program has a date & time Y2k problem. And this always ends in whatever licensing you give it to install will expire and be nullified. Luckily this time while re-installing on VMware I saw if you defer the license on install, It’ll grant you a temporary eval license. It’s not going to matter as it’ll immediately expire, but it get’s us past the install.

Qemu however let’s you rev up the time machine and specify a starting time

-rtc base=1999-09-29T15:00:00

As simple as that. I found for installing with Qemu 8.0 (Latest win32) binary it worked well enough like this:

"c:\Program Files\qemu\qemu-system-x86_64.exe" ^
-m 1024 ^
-hda UnixWare711.vmdk ^
-cdrom SCO_UnixWare711.iso ^
-net nic,model=pcnet ^
-rtc base=2010-09-29T15:00:00

Installation in a stock boring VM goes fine, there is built in support for the AMD PCnet driver, so things ‘just work’. And then on the VMware reboot it never launches X11.

Starting Desktop works fine on Qemu

Under Qemu, I’m greeted by CDE and the login page. On VMware however…

Starting Desktop never starts

I know t his used to work on VMware, but there is some regression in the VESA video driver. The fix was to use scoadmin and knock the video settings down to stock VGA. Luckily I have an X server running on Windows, so I could just export the display and set it up.

Video Configuration on Qemu

Whereas I had to set VMware to VGA:

Video Configuration on VMware

And one more reboot, and I was at least given a graphical console:

Now able to login to VMware graphically

Inputting the licenses

Being a commercial Unix from back in the day, it relies on License Numbers, and activation codes to actually use the software. I have the box, so I have codes so yay me. Post install, I could remove the expired licenses, and then input the ones that were in the box.

These are 5 user licenses, just enough to show off the system, I suppose: The advanced features of the day are nothing special these days, but it’s nice to have the PCC derived compiler, if not to compile GCC but more so for SYSV code from back in the day.

Networking

Networking for VMware is straightforward, I use the NAT interface VMnet8 that is installed by default, selecting a valid Tcp/IP address on the interface range gives me not only full internet access, but also allows me to easily telnet into the VM.

Qemu however…

"c:\Program Files\qemu\qemu-system-x86_64.exe" ^
-m 1024 ^
-hda UnixWare711.vmdk ^
-net nic,model=pcnet ^
-net user,hostfwd=tcp::42323-:23

I had been using the user mode SLiRP code for ages, but after all the MIT PC/IP fun, I was thinking I bet modern Qemu supports UDP transport for network traffic, and that it’d just integrate with HecnetNT just fine. And it does!

"c:\Program Files\qemu\qemu-system-x86_64.exe" ^
-m 1024 ^
-hda UnixWare711.vmdk ^
-net nic,model=pcnet,netdev=hecnet ^
-netdev socket,id=hecnet,udp=127.0.0.1:5001,localaddr=127.0.0.1:5000

Configuring the HetnetNT bridge was simple, as always make sure you have Wireshark/pcapng installed and simply run ethlist to get the list of interfaces:

D:\qemu>ethlist.exe
Network devices:
  Number       NAME                                     (Description)
  0  \Device\NPF_{E79F6278-3E7E-4547-955A-2080A0473AD6} (Local Area Connection* 8)
  1  \Device\NPF_{1D960E08-2A3A-43F7-BAD6-21FCB466717B} (Local Area Connection* 7)
  2  \Device\NPF_{98053A85-B049-45A0-AC33-961E2C136FCA} (Local Area Connection* 6)
  3  \Device\NPF_{BFA868ED-E508-4436-B085-EC815C4C544C} (LoopBack)
  4  \Device\NPF_{C75EAF23-0FA3-433B-B271-9CB0F5EB92D0} (VMware Network Adapter VMnet8)
  5  \Device\NPF_{B615DE21-AEC3-4347-916C-332AC4A4DA70} (VMware Network Adapter VMnet1)
  6  \Device\NPF_{82E5A370-6D3D-40AD-A9D5-C4E0E0C50F0D} (Ethernet)

And then create a simple bridge.conf file with the VMnet8 adapter and the UDP pariing to talk to Qemu:

[bridge]
update 127.0.0.1:5000
vmnet8 \Device\NPF_{C75EAF23-0FA3-433B-B271-9CB0F5EB92D0}


[tcpip]
update
vmnet8

Then launch the bridge program listening on port 5001:

D:\qemu>hecnet.exe 5001
Config filename: bridge.conf
Adding router ''update''. 0100007f:5000
Opening pcap \Device\NPF_{C75EAF23-0FA3-433B-B271-9CB0F5EB92D0}
Adding router ''vmnet8''. 00000000:0
Host table:
0: update 127.0.0.1:5000 (Rx: 0 Tx: 0 (Drop rx: 0)) Active: 1 Throttle: 0(000)
1: vmnet8 0.0.0.0:0 (Rx: 0 Tx: 0 (Drop rx: 0)) Active: 1 Throttle: 0(000)
Hash of known destinations:
Adding new hash entry [52:54:00:12:34:56]. Port is 0
Adding new hash entry [00:50:56:c0:00:08]. Port is 1
Adding new hash entry [00:50:56:f1:dd:d0]. Port is 1
Adding new hash entry [00:0c:29:9a:2b:fb]. Port is 1

It’s a little bit more involved to setup as we have to link the 2 programs via UDP, but I can say it’s totally worth it.

“It just works!” – Sydney

I can now easily FTP files into Qemu, and of course telnet as much as I want to. I don’t see why NFS wouldn’t work either.

Which brings us to the bigger elephant in the room, which one is ‘worth the squeeze’?!

I thought it’d be fun to do a totally unfair CPU intensive thing like building GCC. I would do a quick stage 3 compile blindly running this:

./configure --host=i386-sysv4 --target=i386-sysv4 --prefix=/usr/local/gcc-2.5.8
make
make stage1
make CC="stage1/xgcc -Bstage1/" CFLAGS="-g -O"
make stage2
make CC="stage2/xgcc -Bstage2/" CFLAGS="-g -O"
make stage3
make CC="stage3/xgcc -Bstage3/" CFLAGS="-g -O"

This way we can just look at the timestamps between completed releases. It does build C++ & ObjectiveC as well, and compared to machines from 1999 this is amazing!

-rwxr-xr-x 1 neozeed other 3495688 Mar 29 12:42 ./cc1
-rwxr-xr-x 1 neozeed other 2646888 Mar 29 12:37 ./stage1/cc1
-rwxr-xr-x 1 neozeed other 3495720 Mar 29 12:39 ./stage2/cc1
-rwxr-xr-x 1 neozeed other 3495688 Mar 29 12:40 ./stage3/cc1

Qemu timing

-rwxr-xr-x 1 neozeed other 3884076 Mar 28 20:12 ./cc1
-rwxr-xr-x 1 neozeed other 2647116 Mar 28 20:11 ./stage1/cc1
-rwxr-xr-x 1 neozeed other 3884124 Mar 28 20:11 ./stage2/cc1
-rwxr-xr-x 1 neozeed other 3884076 Mar 28 20:12 ./stage3/cc1

VMware timing

As you can see VMware is substantially faster when it comes to computation. This shouldn’t come to anyone as any surprise. And this isn’t a fair competition, but it does show that you can stage GCC on Qemu just fine, so that’s actually great!

Now let’s mix in some more nonsense, I have a Merge license so let’s try it! First off it really wants Windows 95 from CD-ROM. It will not accept anything else. I have a hacked-up copy of the floppy version of Windows 95 on CD-ROM, and it accepted that just fine, it appears to search through.CAB files looking for files to setup it’s preferred environment. I’m not all that familiar with the whole thing as PC’s are cheap, and virtual machines are even cheaper!

Merge setup on VMware

After the setup completed, I thought I’d try my Sarien 286/386 ports.

no DPMI for Merge

Sadly, neither worked. Maybe it’d have better luck with Windows 95 actually installed. I wanted some high colours so I went over to Qemu and found out that it cannot run Merge.

No Merge!

The error lies in a missing opcode 000000FF. Maybe it’s invalid to trigger an exception to call between DOS and the supervisor?

unknown opcode 000000FF

Either way it doesn’t matter, it doesn’t work. I did get feedback that it does run under KVM.

I don’t know why I didn’t think about doing the HecnetNT bridge earlier as it gives things far more flexibility for tapping into networks, or even being transported. I guess I should look at other transport mechanisms besides UDP since it’s 1:1. Also, it might be worth dropping the protocol restrictive filters to allow everything on the wire to flow.

Installing NetManage Chameleon on Windows 3.0!

After seeing the spotlight on twitter from WinWorld, on NetManage Chameleon, an old TCP/IP stack that supported Windows 3.0! With more details over on the forum. I was inspired to set it up myself.

I did go a bit overboard showing how to install MS-DOS & Windows 3.0 on Qemu. Maybe it’ll help someone who wants to try to use Qemu, but is too scared? Maybe I moved too quickly.

One thing I did do differently in this run, is launching the monitor and a serial port as tcp servers so I could telnet into the VM, effectively having a way to share text like a clipboard back and forth. I’m kind of surprised I hadn’t really started using Qemu in this manner much earlier.

qemu.exe -L pc-bios ^
-m 16 ^
-hda apricot.vmdk ^
-net nic,model=pcnet -net user ^
-monitor telnet:127.0.0.1:4000,server,nowait ^
-serial telnet:127.0.0.1:4001,server,nowait ^
-fda yourdisk_here.vfd

Surprisingly it went surprisingly well, other than my goof of having the OS/2 driver instead of the MS-DOS driver for the nic.

Sadly, the tn3270 program bundled with Chameleon doesn’t work properly with Hercules.

As always I’ve uploaded it to archive.org: apricot-dos4-win3-chameleon3.7z

Citrix 2 on Qemu!

Something weird happened when I was stress testing a build platform, I scripted out a build of all the old Qemu I could with GCC 3.4.5, and I had pointed them to a random disk. Turns out it was a Citrix Multiuser 2 disk. And something weird happened, a few of them actually booted!

I setup a simple script to have it listen on both serial ports, one for the ICA Citrix client, the other accepting a more generic serial terminal.

qemu -L pc-bios -m 64 -M isapc ^
-fda dummy.vfd ^
-hda citrix2-1b.vmdk ^
-serial telnet:127.0.0.1:5523,server,nowait ^
-serial telnet:127.0.0.1:5524,server,nowait ^
-net nic,model=ne2k_isa ^
-net user

Just very simple stuff.

I can then just boot a simple PC qemu and attach it to the other side of the serial port

qemu.exe -L pc-bios -m 8 ^
-serial telnet:127.0.0.1:5523 ^
-fda "i:\Citrix Multiuser Link 1.0 (3.5-720k)\Images\disk01.img" ^
-hda msdos5.vmdk

Sadly it’s only text mode, and at 9600 baud, which you can really feel. Sadly OS/2 PC Serial port drivers sucked.

this kind of setup for me was a bit easier to mess around with linkers as I have multiple screens to view stuff, along with a terrible modem option to transfer files. Only the console can run graphical programs, so this isn’t the one to build a Windows 3.0 farm on

If anyone was crazy enough to try this, I’d suggest the 1.0 manuals. Sorry I didn’t get a chance to scan the 2.0 stuff.

The serial ports lock & jam a lot so I made some scripts to try to deal with it, just rserial1/2 to reset the port, and logouts1/2 to logout the serial connection. Yes it’s that fragile. Being a pre-release version of OS/2 I had zero luck getting any lan client going. If I had it’d make dealing with this thing, even with NetBEUI a lot more tollerable.

I did load up Windows 3.0 by the directions so it does work, albeit standard mode only. Looking back at OS/2 6.123, while it has the OS/2 1.2 interface, it’s MS-DOS was restricted to real mode only. So this was also a nice step up.

One thing to point out is that it’ll hang at the logo screen for literally a minute. I’m not anywhere near as smart enough to debug the loop and ‘fix’ it. Sometimes it just won’t make it, so close it down, and try again. It may seem hopeless sometimes but it does work. Although this unpredictability is why there is no public Citrix 2.0 on demand. I haven’t bothered tryng to build Qemu 0.8.2 on Linux, but if it were more reliable booting and networking, combining it with xinetd having a LAN backing store of homes & apps would make for a neat Citrix on Deman farm thing.

I can’t imagine anyone wanting to play but I did upload it to archive.org.

Microsoft’s Netware emulators

First thing to take care of, is if you have the old pcap on Windows running around. If you have it, you’ll know as you’ll get spammed with “FATAL Bad Memory Block.”, although things will continue to operate just fine.

Win10Pcap!
C:\dynamips\netware\qemu-0.90-pcap-client>qemu -m 16 -L pc-bios -M isapc -hda client.disk -soundhw sb16,adlib -net nic,macaddr=52:24:00:22:00:01 -net pcap,devicename={BFA868ED-E508-4436-B085-EC815C4C544C}
Eth: opened {BFA868ED-E508-4436-B085-EC815C4C544C}
Could not open '\\.\kqemu' - QEMU acceleration layer not activated
FATAL Bad Memory Block.
FATAL Bad Memory Block.
FATAL Bad Memory Block.
FATAL Bad Memory Block.

So be sure to dump that for the one over on npcap!

The old times, actually running Netware 3.12

There was a time when Windows NT didn’t dominate the 1990’s data centre. Instead as a carryover from the 1980’s the majority of corporate LANS were instead based on Netware. And the only way Windows NT was going to make space in this environment was to dress up in sheep’s clothes and mingle among them unnoticed. That brings us to this GEM:

Services for NetWare

This fun CD will let our NT 4.0 server emulate a NetWare server! The first thing in one of these stealth migrations was to just join the existing network.

The existing network is 0C0FFCAB

In order to do this, the two bits of information we need is the frame type, since NetWare supports so many, and the network address. In this case its 0C0FFCAB.

default IPX is no good

By default the NT server will just listen to the network, and participate on what it sees. This is fine if you are just playing along as a dynamic node, but being a NetWare node requires you to step it up, and have these values set, as it is very possible that you could be the first one (or only one) live on the network, and you don’t want clients trying to think on their own.

I also gave mine an internal network number of 1381, because you know, it’s NT 4.0.

To add the FPNW, you need to add it as a new service. Just tell it you have a disk

You’ll then have to point it to the path of the install. This is honestly the hardest part.

Selecting the first option will install the NetWare Server emulation on the NT server.

I went ahead and named my NetWare emulation as SHEEP, as I NT to blend into the existing NetWare network, with nobody being the wiser.

indeed, on our client that was already connected to the Qemu server before I built WOLF, I ran an slist command to show all the servers on the network, and there is my Wolf in Sheep’s clothes.

Creating NetWare compatible volumes is done in the Server Manager, under the FPNW option. It’s pretty self explanatory, nothing too exciting there.

The truth is during the period where this was important the NT 3.51-40 timeframe, NetWare was still a dominant force. But once Windows 95 had launched, and the explosion of people wanting MORE, the natural interest of people going to NT was just amazing to see in corporate space. While there was an early beta of the newshell for NT 3.51, when NT 4.0 shipped it was just amazing as all the reservations for running NT had just evaporated. We’d gone from hiding among the sheep to full on eating them all. It was staggering how fast we were backing up NetWare volumes to only re-format the servers to NT, and get people converted to using them. Before NT 4, the consensus was that rolling out the client config was going to be a nightmare, and that being able to emulate NetWare was the way to go, as it would just work (see the MS-DOS VM talking to NT with an unmodified NetWare client). Instead we saw a massive drive to Windows 95, which ended up changing the client landscape and upending NetWare completly.

About the most difficult thing was user mappings, there was tools to do this kind of thing, and I believe we had something to even proxy passwords, but it was easier to make people just login to the NT side.

Of course this is ONE of the emulators, you might be asking, okay, what is the other?

Why, it’s WINDOWS 95.

YES.

I’m joining the NT domain for the full experence, but the NetWare emulation relies on NetWare servers for authentication. You could use an actual NetWare server, or of course a FPNW server.

Adding file and printer sharing for NetWare workgroups under Windows 95 is done by adding a Service to the network stack. It’s even on the floppy version.

To maximize the functionality and the pain, be sure to turn on SAP Advertising. This way it’ll appear in server lists.

SAP on!

So with all of this in place, yes you can map drives from the MS-DOS client to the Windows 95 workstation acting as a server.

Mapping a drive on 95, authenticated by the WOLF hiding as a SHEEP

And there we go, I can now see the Windows 95 workstation on the SLIST, and connect and map drives. My user account of course exists on the NT side.

While professionally I didn’t rely too much on this feature, but it was nice in that era where you still had MS-DOS/MacOS/OS2 desktops with NetWare clients to quickly share stuff. But in a large organisation this would lead to major issues.

The fundamental flaw in NetWare is that there is no directory service. Instead, all the servers have to broadcast that they exist, along with what services they provide.

On my tiny demo network this isn’t that much traffic. But on a larger network that spans continents this becomes a problem. With thousands of servers there can be an incredible amount of this SAP announcement traffic. Since there is no directory service, the other problem is that when a new client is booted up, it’ll do what is known as a GNS or Get Nearest Server request in order to find the closest server to attach to, in order to facilitate a login. And EVERY server will reply.

And as you can see some servers even will reply more than once. And this can have other effects where people reboot servers during the day, something that is very natural for a Windows 95 user, which could create issues for other users, even forcing them to reboot! And yes, anecdotally I ran into this so many times where people with laptops with this feature turned on, and they would screw up the local office building (impacting hundreds of people). Even when they weren’t winning the GNS elections.they are still generating extra traffic, and occasionally they will win. This was another problem we had with all these wolves hiding in sheep’s clothing.

In the end, NetWare was utterly removed from the data center’s by the end of 1997. Windows NT just scaled too well for SMP and large disks (I had one server with 1TB! It was using 4GB disks it was massive!), along with being able to easily install stuff like SQL Server & SNA Server, unlike NetWare where any NLM conflict will bring the entire thing down. Not having a name lookup server was a giant pain, but the final nail was also in 1997 with the rise of the internet, and normal people now getting involved the entire LAN/WAN was going TCP/IP, where it had only been a fringe protocol used for managing cisco routers, and tftp/ftp some files around, Windows NT’s ability to encapsulate named pipes, and NETBIOS over TCP/IP let them embrace this new world where the TCP/IP stack on NetWare 3.12/4.11 was only good for sending SNMP alerts.

But don’t cry for NetWare, they made so much money they were able to coast for decades before being bought out in 2010 by a Mainframe Terminal Emulation company of all things, The Attachmate Group, who was later in turn bought out by Micro Focus, a COBOL language company. I guess in the end, the Mainframes won?

Everyone seems to be losing their minds over the Windows XP Professional Key

algorithm being cracked.

But of course, how does that help me?

Unironically, I had purchased this for a whopping £4.68

No, really here’s the receipt. What a bargain!

Of course this is a legit copy with a legit key. But the online activation servers are all gone, and it looks like I’d have to call someone asking about my 22 year old copy of Windows, that I’ll load up and quickly forget.

Since I’m going to use QEMU, 0.90 with pcap support I thought I’d share the startup options:

set loopback=\Device\NPF_{3DF0EC5D-7FBE-46DF-ACF8-EF5D8679A473}
set vmnet1=\Device\NPF_{3BC364F4-5A15-405D-926C-C594383F0323}
qemu -m 512 -L pc-bios ^
-hda xphome.vmdk ^
-soundhw es1370 ^
-net nic,model=pcnet,macaddr=52:24:00:33:00:01 ^
-net pcap,devicename=%loopback% ^
%1 %2 %3 %4 %5 %6

I had high hopes for this thing. Clearly misplaced ambitions.

First up, it’s an upgrade version. So that means instead of installing XP I had to waste my time installing NT Workstation 3.51, then installing XP. Yuck. And of course it just want small FAT disks of the 2/4 gigabyte boundary type as it’s 1994. Not the bright future of 2002’s Windows XP.

I don’t know why Qemu 0.90 has issues with XP detecting the CD-ROM drive, but yeah that sucked. I wanted to load up some more insane SNA experiments, but there is no DLC / 802.2 driver for XP Home. wow.

At least once it’s satisfied, we can format the disk as one big happy partition, and we can get on with our lives.

Installation is rather uneventful, however we are instantly reminded that we have only 30 days to go. Since we have that nasty CD-ROM issue that means shutting down, and booting back up, but with this fun program on an ISO image, xp_activate.

I did try to make a call, to activate my Windows, but the connection was terrible and I’m not even sure if these numbers were right. No I mean I know they didn’t work.

So I did what all legit users end up doing, using the crack for my 21 year old copy of Windows.

And just a few clicks later, it was done.

Windows XP Home is activated.

I don’t know if it’s even really going to last, I didn’t try anything else, actually I already deleted it. And the XP folio is back on the bookshelf.

Not only is there no DLC, did you know you can’t uninstall TCP/IP? At least you can unbind it from your NIC. While it does have IPX/SPX there is no built in Netware client. When they said HOME they meant it!

Ghosts in the mainframe!

There is a LOT going on in this image, and I’ll try to explain it, but yeah “it’s complicated”.

SNA networking & Hercules has always been a goal for a lot of people, including me as we always wanted to setup some SNA server of some kind. Especially on RISC platforms, as there is only so much fun on SQL server.

Okay I know the practical among you will say, doesn’t it support telnet 3270? Isn’t that good enough? Yes for day to day mundane stuff, absolutely. But I’m not all that interested in that, I wan’t to have the whole ancient network, and I wan’t it self contained and on my desk! Or on a laptop, as I see fit.

What started this whole adventure was a simple image from 9track.net, showing that being able to connect physical devices to Hercules was indeed possible!

Image from https://www.9track.net/hercules/dlsw/

This is a physical IBM 3178 & 3179 terminals talking to TK4- , a MVS3.8j pre-configured system!

The magic that makes this all possible, is a cisco router, running enterprise IOS, with dlsw support.

My setup is going to be inspired by this setup, but not exactly 100% But this is what I’m going to use on Windows 10

  • Dynamips for the cisco router, running JS-M 12.2(25)S8
  • Qemu 0.90 with PCAP running Windows NT 3.51 Server along with SNA Server 2.1
  • Qemu 0.90 running Windows 3.1 and XVision
  • VMware Player
  • WireShark
  • Microsoft Loopback adapter
  • WSLv1

I had originally wanted to run the NT server on VMware but for some reason it just hangs trying to initialise the NT kernel. I didn’t bother trying to troubleshoot it, I just jumped to Qemu. Even service pack 5 didn’t help. VMware left me with the virtual network that will NAT if needed, and of course let me telnet to the Dynamips program. The SNA traffic is isolated to the MS Loopback adapter, which will let pcap programs talk to each other.

The first thing I did was run ‘hdwwiz’ on Windows 10, and added in the KM-TEST loopback adapter

We know what we want, so go to the manuall selection

Network adapters

And select the KM-TEST Loopback Adapter

Next I changed the protocols available on the loopback, as I don’t want my Windows 10 host interfering with the SNA network at all.

So the next thing to do is to get your network GUID’s. ethlist.exe from the Dynamips download will get you that:

C:\dynamips>ethlist.exe
Network devices:
  Number       NAME                                     (Description)
  0  \Device\NPF_{3DF0EC5D-7FBE-46DF-ACF8-EF5D8679A473} (loopback)
  1  \Device\NPF_{D9FBD118-B9DF-4C3C-BD9E-07A0E34D8F75} (Local Area Connection* 8)
  2  \Device\NPF_{F5057901-6A30-413A-80E4-4765DA794B7C} (Local Area Connection* 7)
  3  \Device\NPF_{E3D3EC8D-29C3-4B70-B01C-600D3F9ED1D6} (Local Area Connection* 6)
  4  \Device\NPF_{82EEDBC1-899D-416F-BD51-3DBE2287257F} (VMware Network Adapter VMnet8)
  5  \Device\NPF_{3BC364F4-5A15-405D-926C-C594383F0323} (VMware Network Adapter VMnet1)
  6  \Device\NPF_{DDF1FA94-7488-414F-A41A-EC88C1FB0DE4} (Ethernet)
  7  \Device\NPF_{E7CA8F40-4639-410D-B5CA-F402FE69AF5D} (Ethernet 2)

I want the cisco router to have two interfaces, one with TCP/IP for me to be able to telnet into it (maybe other management as well?!) and the other one for the SNA traffic.

Setting up Dynamips

As mentioned above I’m going to use the VMnet1 for TCP/IP to the router, and the loopback adapter for SNA traffic. To try to make things a little easier to read I setup a small batch file that let’s me plug in variables to Dynamips:

set loopback=\Device\NPF_{3DF0EC5D-7FBE-46DF-ACF8-EF5D8679A473}
set vmnet1=\Device\NPF_{3BC364F4-5A15-405D-926C-C594383F0323}
set IOS=c7200-js-mz.122-25.S8.bin
set NPE=npe-200
..\dynamips.exe -P 7200 %IOS%  ^
-t %NPE%  ^
-p 0:C7200-IO-FE ^
-s0:0:gen_eth:%vmnet1% ^
-p 1:PA-4E  ^
-s1:0:gen_eth:%loopback% ^
-p2:PA-4T+

The caret symbol will break up lines on NT, much like the ampersand will on Unix. And this let’s me use clear variables for the networks, IOS & NPE type so it’s nowhere near as complicated to edit.

This will create a cisco 7200 with an NPE-200, with the following cards:

The next thing is what ip address is bound to VMnet1? This is mine:

Ethernet adapter VMware Network Adapter VMnet1:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::c3d2:c891:b7e0:6797%5
   IPv4 Address. . . . . . . . . . . : 192.168.199.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

So all my TCP/IP in this example will be using 192.168.199.0/24

As mentioned on the 9track page, all the magic happens on the cisco router. I’ve made a few changes as I may want to try the SDLC in the future to perhaps some other experiment if I can find an emulator that’ll drive it over serial, but for now let’s just get to the config:

!
version 12.2
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname dlsw
!
boot-start-marker
boot-end-marker
!
enable password cisco
!
ip subnet-zero
!
!
no ip domain-lookup
!
ip cef
no mpls traffic-eng auto-bw timers frequency 0
call rsvp-sync
!
!
!
!
!
!
!
source-bridge ring-group 1
dlsw local-peer peer-id 192.168.199.10
dlsw remote-peer 0 tcp 192.168.199.1
dlsw mac-addr 4000.1020.0100 remote-peer ip-address 192.168.199.1
dlsw udp-disable
dlsw transparent switch-support
!
interface FastEthernet0/0
 ip address 192.168.199.10 255.255.255.0
 duplex half
 no clns route-cache
!
interface Ethernet1/0
 no ip address
 duplex half
 no clns route-cache
 dlsw transparent redundancy-enable 5555.5555.5000
 dlsw transparent map local-mac 4000.1020.0100  remote-mac 4000.0999.0100
!
interface Ethernet1/1
 no ip address
 shutdown
 duplex half
 no clns route-cache
!
interface Ethernet1/2
 no ip address
 shutdown
 duplex half
 no clns route-cache
!
interface Ethernet1/3
 no ip address
 shutdown
 duplex half
 no clns route-cache
!
interface Serial2/0
 no ip address
 encapsulation sdlc
 no keepalive
 serial restart-delay 0
 clockrate 64000
 no clns route-cache
 sdlc role primary
 sdlc vmac 4000.0999.0100
 sdlc address C1
 sdlc xid C1 01700019
 sdlc partner 4000.1020.1000 C1
 sdlc dlsw C1
!
interface Serial2/1
 no ip address
 shutdown
 serial restart-delay 0
 no clns route-cache
!
interface Serial2/2
 no ip address
 shutdown
 serial restart-delay 0
 no clns route-cache
!
interface Serial2/3
 no ip address
 shutdown
 serial restart-delay 0
 no clns route-cache
!
ip classless
!
no ip http server
!
!
!
!
!
!
control-plane
!
!
dial-peer cor custom
!
!
!
!
gatekeeper
 shutdown
!
!
line con 0
 session-timeout 35791
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 password cisco
 login
!
!
end

This sets up the router so I can telnet to it from my desktop at 192.168.199.10, and allows it to talk to the base Windows machine on 192.168.199.1

All the magical MAC addresses come from 9track.net, as he wrote the dlsw hooks, so I just copied that. There is probably a great deal that could be cleaned up, but once I saw the two talking I kind of froze what I was doing.

With that much in place I then jumped to WSL,and built the emulator from github. I cloned it, and renamed that to herc-dlsw. At least for me this was pretty straightforward. The Hercules fork will build with Visual Studio as well, but I knew I was going to need some kind of tn3270 emulator, and I wanted to use x3270, and I had just recently bought this discounted copy of XVision, so of course I wanted to use that.

Despite this catastrophic defect that wasn’t disclosed in the auction.

I downloaded and extracted the TK4- latest distro on WSL. I just created a ‘herc’ directory in my home to house the tk4- release. The next thing to do is overlay your dlsw enabled exe’s and libraries.

cd ~/herc-dlsw/.libs
mkdir x
cp * x
cd x
rm *.o *.lai
cp *.so $HOME/herc/hercules/linux/64/lib/hercules
cp *.la $HOME/herc/hercules/linux/64/lib/hercules
rm *.so *.la
cp * $HOME/herc/hercules/linux/64

Now with the binaries in place, I do need to setup the Xvision VM so I can receive the X11. Of course there is so many other ways to do this, but this is mine:

qemu.exe -L pc-bios -m 64 -hda xvision.vmdk -net nic,model=ne2k_isa -net user -redir tcp:6000::6000

The important thing is that tcp port 6000 is redirected inwards, and that I’m using the NE2000 card, which on my weird fork will print out the hardware config, so I know how to find the nic.

added SLIRP
adding a [GenuineIntelC♣] family 5 model 4 stepping 3 CPU
added 64 megabytes of RAM
trying to load video rom pc-bios/vgabios-cirrus.bin
added parallel port 0x378 7
added NE2000(isa) 0x320 10
pci_piix3_ide_init PIIX3 IDE
ide_init2 [0] s->cylinders 203 s->heads 16 s->sectors 63
ide_init2 [1] s->cylinders 0 s->heads 0 s->sectors 0
ide_init2 [0] s->cylinders 2 s->heads 16 s->sectors 63
ide_init2 [1] s->cylinders 0 s->heads 0 s->sectors 0
added PS/2 keyboard
ps2.c added PS/2 mouse handler
added Floppy Controller 0x3f0 irq 6 dma 2
installing PS/2 mouse in CMOS
  Bus  0, device   0, function 0:
    Host bridge: PCI device 8086:1237
  Bus  0, device   1, function 0:
    ISA bridge: PCI device 8086:7000
  Bus  0, device   1, function 1:
    IDE controller: PCI device 8086:7010
      BAR4: I/O at 0xffffffff [0x000e].
  Bus  0, device   1, function 3:
    Class 0680: PCI device 8086:7113
      IRQ 0.
  Bus  0, device   2, function 0:
    VGA controller: PCI device 1013:00b8
      BAR0: 32 bit memory at 0xffffffff [0x01fffffe].
      BAR1: 32 bit memory at 0xffffffff [0x00000ffe].

And in this case it’s 0x320 IRQ 10. XVision being it’s own level of disappointment, I’ll have to cover it further, and later but suffice to say it at least catches the x3270 so I can get onto the console.

Setting up Hercules

Editing conf/tk4-_default.cnf is pretty easy as it’s on Linux and you can use VI.

# NCP VTAM
#
0660 3705 lport=${N660PORT:=37051} locncpnm=N07 rmtncpnm=N08 …
          unitsz=252 ackspeed=1000
0661 3705 lport=${N661PORT:=37052} locncpnm=N10 rmtncpnm=N11 …
          idblk=017 idnum=00018 locsuba=10 rmtsuba=11 unitsz=252 …
          ackspeed=1000
0662 3705 lport=${N662PORT:=37053} debug=yes dlsw=yes locncpnm=N12 …
          rmtncpnm=N13 idblk=017 idnum=00019 locsuba=12 rmtsuba=13 …
          unitsz=252 ackspeed=1000
0663 3705 lport=${N663PORT:=37054} locncpnm=N14 rmtncpnm=N15 idblk=017 …
          idnum=0001a locsuba=14 rmtsuba=15 unitsz=252 ackspeed=1000

And it’s simple, just assign the dlsw to the 0662 3705 controller.

The real fun is in the VTAM configuration. Which had been stumping me for well over a year. But then I found this Bradrico Rigg article aptly titled : Run your own mainframe using Hercules mainframe emulator and MVS 3.8j tk4, and it gave me the confidence to get this DONE. Thanks Bradrico!

First get MVS up and running. You have to run the ‘console_mode’ script to see what is going on.

cd herc/unattended
./set_console_mode
cd ..
./mvs

It’s not all that difficult XVision is using SLiRP, so it’s listening on all my IP addresses so I just do a simple

export DISPLAY=192.168.1.72:0
nohup x3270 &

And the emulator will pop up in Qemu. Just connect to localhost:3270 and you’ll be greeted by the login pannel:

Credentials are HERC01 / CUL8TR

I would HIGHLY recommend following the tutorial to get used to submitting a simple COBOL program. It walks through the key concepts of locating a file, and viewing it on MVS. Something that up until yesterday was out of my league.

We need to edit the file S3705 on SYS1.VTAMLST

Basically it’s 1,3,4 from the main pannel:

or RFE, Utilities, DSLIST

Type in the Volume name, then tab over to the left of the volume and put in V to view

Now we will get a list of all the files. We want to edit S3705, so you can tab/arrow down, but sure to put an `E’ next to it, then hit enter so we can edit the file

F7/F8 will page down/page up as needed. As mentioned we are interested in Subarea 13, PU type 2.

The line we are changing is the MAXDATA or MTU size for this unit. Since we are doing dlsw, or an emulated serial link, we need to knock it down to 256. Notice all the plus signs on the right hand? THOSE ARE IMPORTANT! Not only do they need to exist, but they also have to be on the far right.

For those wondering the MTU sizes on the client side by media type are as follows: And notice that the host size is different, as this takes in account of packet headers.

Making sure to overtype the 3780, to a 256, and ensuring the + sign hasn’t moved you can hit enter, cursor to the top and type in SAVE.

We can then edit the N13 file, changing line 35 to have MAXLU=3

Hopefully this clears up editing VTAM files.

As mentioned the easiest way to regen the system is to delete the old object files. So hit f3 a few times and get back to the dataset list

This time we want the VTAMOBJ set. Go and ‘V’iew it like last time and we will get the list of files:

Now we are going to put a ‘d’ next to N13 and S3705. This will flag them for deletion. Hit enter!

The files are now gone! On the next boot they will be rebuilt.

I just hit F3 a bunch of times and it’ll drop to some TSO shell

From here you can shutdown the system. It’ll take a few minutes, but you can start it up again just the same way you brought it up. Remember to attach your console.

Setting up SNA Server

Just like Dynamips, I setup a batch file, as the default one is just far too long to read:

@echo you need to figure out your nic name..
@echo something like
@echo \Device\NPF_{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}
set loopback=\Device\NPF_{3DF0EC5D-7FBE-46DF-ACF8-EF5D8679A473}
set vmnet1=\Device\NPF_{3BC364F4-5A15-405D-926C-C594383F0323}
qemu -m 64 -L pc-bios ^
-hda SBS15.vmdk ^
-soundhw sb16,adlib ^
-net nic,model=pcnet,macaddr=52:24:00:22:00:01 ^
-net pcap,devicename=%loopback% ^
%1 %2 %3 %4 %5 %6

This will setup a small machine with 64MB of ram, a single AMD PCNet adapter on the loopback interface. I installed Windows NT 3.51 from the Small Business Server 1.5 setup. I don’t know why VMware + NT 3.51 didn’t get along, maybe it’s my Erying, Or maybe it just plain doesn’t work, I’m not sure, and far too impatient to troubleshoot it.

It’s very important that you do add the DLC Protocol during setup. It’s in the ‘Add Software’ part. I kept my NT very simple with only NetBEUI and DLC protocols. At the moment I’m not that interested in actually networking the NT, and if I was, I would add a second NIC, just like what I did for Dynamips.

Setting up NT isn’t that interesting, but SNA server is. I did use the 2.11 on the Back Office CD, but for completeness sake of testing I tried the oldest one I could find, and 2.1 beta from June, Build 2.1.0.216.

I left the network name & control point name blank as I just want terminal, I’m not even going to think that LU6.2 applications on such an ancient version of MVS was even possible.

This is pretty much default, the Link service basically sets itself up as we only have the one NIC.

Take note of the remote network address. 400010200100 which came from above the address we directly point to the dlsw. Also it’s form the 9track blog.

Insert a 3270 LU for us to try to talk to Hercules.

I’m pretty sure it was hard coded to be a model 2.

I turned off the ability for the model to be overwitten.

Create a pool, I called it swimming, because of ‘reasons’. I made it a type 2 pool and added the terminal to it.

Next I added the EVERYONE user, and gave them access to the SWIMMING pool

Finally we are ready to save the config, and do the hand holding and start up. If the stars aligned you will see them go ACTIVE/ACTIVE and the terminal will go Available.

Sadly the terminal won’t go live, it’s stuck in SSCP.

And this is as far as I can go. I have to think that with either something far older protocol wise for the PC, such as IBM Personal Communications/3270 for Windows V2.0 (v4 didnt work either), or a far newer Mainframe software version would support whatever it is SNA server wants to give us the crazy dream of running SNA self contained.

Running Wireshark on the loopback network I see this message:

UNSUPPORTED FUNCTION

Sadly this is as far as I can take you. I do want to give a special thanks to Vinatron & blackbit for trying to troubleshoot this with me. Best we can figure is that TK4- is just too old.

Troubleshooting

From the cisco router try dlsw commands like this:

dlsw>sho dlsw circuits
Index           local addr(lsap)    remote addr(dsap)  state          uptime
2281701660      4a24.0044.0080(04)  0200.9099.8000(04) CONNECTED      00:02:23
Total number of circuits connected: 1

This does show the connection. Notice that ‘show bridge’ will show nothing in this config.

Be sure to check peers as well:

dlsw>show dlsw peers
Peers:                state     pkts_rx   pkts_tx  type  drops ckts TCP   uptime
 TCP 192.168.199.1   CONNECT         10        13  conf      0    1   0 00:05:07
Total number of connected peers: 1
Total number of connections:     1

Make sure your interfaces are ‘up/up’ and passing traffic

FastEthernet0/0 is up, line protocol is up
  Hardware is DEC21140, address is ca00.48f4.0000 (bia ca00.48f4.0000)
  Internet address is 192.168.199.10/24
  MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Half-duplex, 100Mb/s, 100BaseTX/FX
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 1000 bits/sec, 2 packets/sec
  5 minute output rate 2000 bits/sec, 2 packets/sec
     12768 packets input, 1439279 bytes
     Received 3609 broadcasts (0 IP multicast)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog
     0 input packets with dribble condition detected
     9999 packets output, 1037736 bytes, 0 underruns
     0 output errors, 0 collisions, 1 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out
dlsw>show int eth1/0
Ethernet1/0 is up, line protocol is up
  Hardware is AmdP2, address is ca00.48f4.001c (bia ca00.48f4.001c)
  MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:02, output 00:00:02, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     52426 packets input, 5148287 bytes, 0 no buffer
     Received 12336 broadcasts (0 IP multicast)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 input packets with dribble condition detected
     36383 packets output, 2465490 bytes, 0 underruns
     0 output errors, 0 collisions, 3 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out
dlsw>

And of course check WireShark to see if there is any handshake:

And of course check the Hercules logs to make sure your VTAM rebuilt, look for ERROR or anything related to S3705 or N13.

Word & Excel for MIPS

Years ago when I’d bought Office 4.2 for Windows NT, it only included i386 & Alpha builds of Word and Excel in the box, and a coupon for MIPS and PowerPC.

About the only thing interesting is that it actually ran under Win32s.

But today looking at term24‘s uploads on archive.org, I saw two CD-ROM images:

I quickly fired up Qemu MIPS NT, and confirmed that both do in fact contain a MIPS version! Excel does have the PowerPC version as well.

As far as I know the only RISC platform to get apps from Office 97 was the Dec Alpha, but at least MIPS users can rejoice now, knowing that they too have been blessed with 32bit Office 4.2 apps!

One of the amazing things about NT & portable apps is that visually, they look identical. So other than me telling you that these are the MIPS native versions, there really is no way to tell.

Well, other than there is no ntvdm running. There is no WOW needed here!

100% native.

I guess the only other question is that since the Word is 1994, and Excel is from 1995, did they have earlier versions for Windows NT? It seems like everything was finally coming together for RISC NT, except the users. Would a release of 64bit Windows 2000 on Dec Alpha save the platform by bringing a strong 64bit platform with integrated JIT i386 WoW built in? (AXP64 Windows 2000 didn’t use !FX32). I guess we’ll never know.