Running the SWGEMU server on WSL

All alone

The topic had come up again about running Core3 aka the Star Wars Galaxies EMUlated server at home. It’s always been a somewhat involved setup, but with the advent of the Windows Subsystem for Linux, there exists the ability to export and import filesystems as if they were containers. With that in hand I went ahead and set one up.

I initially targeted the older WSL v1, as it supports far more hardware, and I didn’t want to leave so many behind. The added benefit of targeting v1, is that the end user can upgrade the filesystem to v2 if they so choose. The only real difference in this project between v1 & v2 is that v1 cannot run the needed MySQL instance. This isn’t a showstopper however as there exists MySQL & MariaDB native versions for Windows.

Obviously, you’ll need Windows 10 or 11 to do this. From what I can tell even the home version of Windows 10 has WSL capabilities, so this doesn’t pose any problems. Simply open the legacy control panel (Windows+R then type in ‘control’), and go to the Programs.

and then ‘Turn Windows features on or off’, scroll down and select ‘Windows Subsystem for Linux’.

Hit OK, and it’ll download what is needed from the internet.

This shouldn’t take too long, then it’ll want to reboot the computer.

Now you just have to wait for Windows to reboot.

Now you’ll need to download the filesystem. I’ve put it on archive.org for the time being right here: wslv1-SWGEMU_Ubuntu20.22 directory listing (archive.org)

At a minimum you’ll need the files:

  • swgemu-ubuntu.tar.gz
  • mariadb-10.10.2-winx64.msi

Keep in mind that the filesystem is 4.8GB. It will take some time to download. If the torrent servers are functioning I would highly recommend that.

Once it’s installed and it’s generated it’s own databases it’ll need just under 7GB of disk space. So figure that you need 15GB free for this fun experiment.

The first step is to install the database server. We will need to know the settings in order to configure the swgemu server.

You can use any MySQL, but Maria is fully GPL, and small, and more importantly doesn’t involve Oracle.

The install is pretty striaghtfoward.

GNU software requires it reproduce the license sooo here we are.

I would HIGHLY recommend use the default option of install everything.

It will ask for a root password. In this example I’m using ‘password’, of course you can use anything you want, as long as you remember it. You don’t have to allow remote machine access to the database as WSLv1 runs in the same userspace on Windows, so it’ll effectively be connecting to itself.

I use TCP port 10000 as I want the default port 3306 for another insance. Again it’s all up to you but you will have to remember what port you put it on. Ports below 1000 are highly NOT recommended as they need special permission. I left the buffer pool as default

With those options set we can let it do the install.

Now the database is installed, and we can go onto restoring the WSL image, and then configuring the core3 server

Firstly we need place to put it, On the machine I built it on it was the D: drive, but for this re-deploy on a new machine I’m using the C: drive. It’s all up to you but it MUST be a locally attached disk, no USB disks.

Simply create the target directory and run the import command something like this:

md c:\starwars
wsl --import swgemu c:\starwars Downloads\swgemu-ubuntu.tar.gz
I only have the one disk on this VM

This will take a while.

Once it’s done you can check the subsystem status:

C:\Users\Administrator>wsl --list -v
  NAME      STATE           VERSION
* swgemu    Stopped         1

You may have more than one installed, so we’ll refer to this one by it’s name ‘swgemu’.

You can start the instance with wsl -d swgemu:

C:\Users\Administrator>wsl -d swgemu
Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 4.4.0-19041-Microsoft x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Thu Dec  8 09:02:39 EST 2022

  System load:    0.5185546875   Processes:             7
  Usage of /home: unknown        Users logged in:       0
  Memory usage:   28%            IPv4 address for eth0: 198.46.174.160
  Swap usage:     0%


0 updates can be applied immediately.



This message is shown once a day. To disable it please create the
/root/.hushlogin file.

You will need to edit the script to create the databases. I like using vi, but you may not. The easiest way is to just browse to the location and open it up in notepad. The path to browse is:

C:\starwars\rootfs\root\starwars\sql

You can drag/drop re-init.sh on notepad, or make the file association, it’s all up to you.

I had set the SQLHOST to the LAN IP I have, you should set it to 127.0.0.1 . The PUBLIC address is the one that you’ll be connecting to. You can set it to your LAN address, or a public one if you know it (you’ll have to setup NAT), or if you are running it all on one machine for testing, it’s fine to be 127.0.0.1.

Save the file re-init.sh and now you have to run it.

[email protected]:~# cd starwars/sql/
[email protected]:~/starwars/sql# ./re-init.sh
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1008 (HY000) at line 1: Can't drop database 'swgemu'; database doesn't exist
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
[email protected]:~/starwars/sql#

The first SQL query will attempt to drop the swgemu database. In this case it doesn’t exist, so the first error is expected. Nothing to worry about. That gives us the database, now it’s time to configure the server.

You can use explorer to browse to:

C:\starwars\rootfs\root\starwars\conf

Now open/drop on notepad the file config.lua

Again change the DBHost, and MantisHost to 127.0.0.1, and update the password for root, if you have changed it. You may also want to change the DBSecret, but since this is all for fun, I’m leaving it stock.

Now it’s time to run the server.

[email protected]:~/starwars/sql# cd ..
[email protected]:~/starwars# ./core3
(0 s) [Core] Server start, pid: 105, time: Thu Dec  8 09:11:54 2022
(0 s) [ConfigManager] Did not find conf/config-local.lua
(0 s) [DatabaseManager] converting database objects to new format
(0 s) [DatabaseManager] finished converting data to new format

There will be a lot of messages going on, and this will take a few minutes. During the first run Windows will ask for network access permissions. Be sure to check both boxes allowing WAN and LAN access.

Check both boxes!

I have a Lenovo C20 and it takes about 2 minutes, depending on your config it may take longer or shorter.

One thing to point out is that I have configured Core3 for all the worlds, and this will need more than 8GB of ram to start up. So if you get something like this:

If this happens, you’ll have to try a minimal config, commenting out the areas from the config. On this test system, it only has 8GB of RAM, and that isn’t enough to have everything. However the minimal of the training & Tatooine fits no problem:

On the machine I originally set this up on, it has 32GB of RAM, so the processes needing just under 7GB of RAM wasn’t an issue. However an 8GB machine just isn’t enough as I found out the hard way re-producing my install.

Since we have previously allowed the Core3 application LAN/WAN access you should be able to connect. If you set the public address to something public, you will need to setup your firewall to forward the following ports to the machine hosting Core3

Otherwise you’ll need to open the following ports and protocols:

TCP 44419
TCP 44455
UDP 44453
UDP 44462
UDP 44463

Again making this a WSLv1 setup makes this easier as Windows runs core3 in the same memory space.

With any luck you should now be able to connect to the address you have specified as your PUBLIC address. If you want to change, you can either update the re-init.sh script and re-run it, destroying everything, or run the HeidiSQL tool, and just update it.

I had to set it to use libmysql.dll . As always remember your root password, and what Port you used to setup MariaDB. With the profile saved you can open it up.

The address field tells the clients what IP address they can find the server on, so it can the the LAN if you are staying at home, or it has to be a Publicly accessible one. Another alternative is something like Hamachi, which lets you create your own virtual network of machines. It’s great, but not all that cheap, but the added benefit is that your server isn’t a public resource, so you know exactly who is connecting.

If you want to be open but in control I’ve used this in the past to great success.

If you do change the IP you have to restart the core3 program.

Simply type in ‘exit’ at the console and it’ll start to gracefully shutdown.

Then simply start it again with a ./core3

If you want to enable the god mode, I assume you have read how what changes to make on the client side, and the table to edit is the accounts table, once you have logged in, it’ll create your record, and you can find it on HeidiSQL and change the admin_level field to 15. With this set the next character you create will be a ‘god’.

I’m not going to go into setting up the client, installing the SWGEMU stuff and how to add custom servers, as I assume you already know how to do so. This is more to let people run their own instances at home, or among friends.

On building core3 for yourself, be aware that by default it’ll use a ‘native’ config option that tunes the compiler to generate code as fast as possible for YOUR processor. So if you want to run it somewhere else, it has a good chance of not running. I had to edit the following files:

./CMakeLists.txt
./utils/engine3/MMOEngine/CMakeLists.txt

And change NATIVE to NO.

I also had issues with the SQL not having a default setting for timestamps, and it would fail to create accounts. Look for stuff like this:

TIMESTAMP NOT NULL,

and replace it with

TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,

And as of this current rev I had some issues with malloc hooks, which I just commented out:

$ diff -ruN AllocationHook.cpp_ utils/engine3/MMOEngine/src/system/mm/AllocationHook.cpp
--- AllocationHook.cpp_ 2022-12-06 07:59:42.160402900 +0000
+++ utils/engine3/MMOEngine/src/system/mm/AllocationHook.cpp 2022-12-06 08:00:22.510402900 +0000
@@ -50,6 +50,7 @@
        instance = this;

 #ifndef PLATFORM_MAC
+#if 0
        __saved_malloc_hook = __malloc_hook;
        __saved_free_hook = __free_hook;
        __saved_realloc_hook = __realloc_hook;
@@ -58,6 +59,7 @@
        __malloc_hook = mallocHook;
        __free_hook = freeHook;
        __realloc_hook = reallocHook;
+#endif

 #else
        malloc_zone_t *dz = malloc_default_zone();
@@ -91,9 +93,11 @@

 void AllocationHook::uninstall() {
 #ifndef PLATFORM_MAC
+#if 0
        __malloc_hook = __saved_malloc_hook;
        __free_hook = __saved_free_hook;
        __realloc_hook = __saved_realloc_hook;
+#endif
 #else
        malloc_zone_t *dz = malloc_default_zone();

$

Yes it’s terrible, but I was in a rush so.. yeah.

Also I have access to a Windows 2019 Server which is Windows 10 1809 based, and it doesn’t support the import-export of filesystems. So this won’t be a clean install, I guess it’d be easier to copy files over from Windows 10, and install Ubuntu 20.22 from the store.

Good Luck!

Windows scp to remote machines with spaces in the directories

Well one nice thing about Windows 10 is that it has a built in ssh/scp client! Although telnet is optional, I get that it’s insecure but jeez what is a retro user to do?

Anyways the subject at hand is copying files from somewhere that has spaces in the path. In this case I need a copy of OS X Snow Leopard from my Mac Pro cylinder to this junk Fujitsu Celsius. I’m still having USB issues, but I’d like to get my data off of an external disk formatted in HFS+. And for ‘reasons’ I wanted to use something “native” but I don’t feel like building a Hackintosh. While not a strict tutorial on getting Snow Leopard running, I did upload my old download of Empire EFI on archive.org as this kind of stuff is damned near impossible to find.

So back to the matter at hand, I have this VM setup on my Mac Pro, and I want it on this Windows machine. You’d think it would be something like this:

scp -C [email protected]:"/Users/neozeed/Virtual Machines.localized/OSX 10.6/*" .
Password:
scp: /Users/neozeed/Virtual: No such file or directory
scp: Machines.localized/OSX: No such file or directory
scp: 10.6/*: No such file or directory

Okay so double quotes didn’t work. How about a Unix style escape for spaces? I mean it *is* scp after all, maybe it doesn’t know it’s on Windows.

C:\osx>scp -C [email protected]:"/Users/neozeed/Virtual\ Machines.localized/OSX\ 10.6/*" .
Password:
scp: /Users/neozeed/Virtual: No such file or directory
scp: Machines.localized/OSX: No such file or directory
scp: 10.6/*: No such file or directory

Well maybe it parses it like C, so you need double backslash? NO that doesn’t work either. Talk about frustrating. So, in an act of insanity, I tried single quoting the interior spaces around double quotes, something idiotic like a bash variable:

C:\osx>scp -C [email protected]:"/Users/neozeed/Virtual' 'Machines.localized/OSX' '10.6/*" .
Password:
Mac OS Snow Leopard.vmdk                                                               69%   11GB  16.0MB/s   05:16 ETA

And yes, now it’s transferring. I’m just using a cheap 50zt 5 port 100Mbit dumb switch. It’s good enough and it’ll probably take some 30 minutes to transfer all the bits, but it’s working.

So there you go. You may not need it now, or tomorrow but it’ll save you the 20 minutes of frustration!

So I was missing Windows 10 Phone, so I fired up the emulator

First you can download it here:

https://www.microsoft.com/en-us/download/details.aspx?id=53424

In my “C:\Program Files (x86)\Windows Kits\10\Emulation\Mobile\10.0.14393.0” directory I have a modest 2GB file called flash.vhd which contains the phone image. I copy it to where I run my VM’s and run it with the XDE emulator:

"C:\Program Files (x86)\Microsoft XDE\10.0.14393.0\"xde /memsize 3072 /vhd FLASH.vhd

And I’m running in no time, I login, load some apps, then I notice the storage:

What is this, a phone for ants?

What?! the disk image is a paultry 10GB. I guess the idea is that you wouldn’t actually try to load up the emulator like it’s your daily driver, rather you load YOUR app and only YOUR app, and just pretend that this isn’t some weird offshoot nostalgia machine.

Well needless to say something needs to be done about this storage situation.

I look and find this package, vhdutils. I had to go to some sketchy site, but it did include source. I should put this somewhere more legit to take away from all those weird squatters.

Update from the future I put VHD-Utilities over on GitHub, along with downloadable EXE’s. I can’t get the original site to load in wayback to give credit to whomever wrote it.

30/09/2021 09:09 pm 6,995,017,728 FLASH.vhd

So with stuff installed onto my phone I’m almost at 7GB physical 7.6 virtual space. I could go all crazy with 128 or 256GB but it’d largely be stuff I bought… which of course thanks to the magical world of DRM won’t play.

Yeah I guess you are welcome that I bought all those movies, and stuff but sure I wasn’t going to watch them on this phone… emulator. Thanks. thanks again.

So the resize vhd is quick. brutal. and efficient. I go with 64GB, because, why not? I could probably just grow it again if I needed to.

resizevhd.exe v:\Windows10-Phone\FLASH.vhd 68719476736

Now for the fun part. We need to attach the vhd, and resize the volume. I hope you like diskpart.

In the MMC I attach the disk image.. it’ll pop a few folders as it’s got a bunch of drive letters. I’ve never explored a phone, I don’t know if the ARM images are just as weird.

Even more strange, it’s MBR!

So if you were thinking, lots of partitions, and a clear win for GPT, sadly this isn’t it.

Sadly there is no free partitions (although one hiding could be deleted…?) And the UI doesn’t support expanding a logical drive (the green container). But diskpart does.

As indicated above the emulator’s vhd is disk3. You can see it’s the 64GB disk. Select it.

Next list the volumes. The Data disk (J:) is what we want so select Volume 10.

Literally just ‘select volume 10’ and ‘extend’. Don’t tell me this is difficult.

Listing the volumes again will show a 59GB Data partition. Congratulations we did it!

Back in the MMC, you’ll see it as well how the Data partition, along with the green extended partition is now taking up the entire disk. So we can now Detach the VHD, and run the emulator again!

Nice!

And just like that we now have plenty of free space on the emulator.

I downloaded some games, and some music. It’s nice to be back home.

It’s not an extensive list as I didn’t game much on my phone but here is what I know works:

  • Final Fantasy 1
  • Heroes of Larkwood
  • Skulls of the Shogun
  • Sonic CD
  • FL Studio
  • Candy Crush Saga
  • Pixel Dungeon +

Halo Spartan Asslt just closes, and Asphalt 8: Airborne doesn’t get the screen size right so it’s impossible to click enough buttons.

Sonic CD on Windows Phone 10

It’s nice that sonic runs, (haha) although using a mouse makes it impossible to control.

Now one fun thing is that the emulator is x86, nor ARM based so I converted the VHD to a VMDK, ran it under VMware, and YES it RUNS… sort of.

Perhaps a format that never was to be, the Phone/Tablet but it boots quickly and is so responsive. Windows without most of the .. Windows bits. I guess the real experiment will have to be will it run on a Surface?

It’s all 32bit anyways, and such an evolutionary dead end. Pitty.

Enable Hyper-V on Windows 10 Home

So you are in a hurry and need to build a network in a box. It was a bit of a surprise, and you have no time. On site there is ONE computer, it’s a NUC. A tiny one. And you cannot replace the base OS for “reasons”… No problem, you say, just add in Hyper-V and you can build an ‘older’ but useful domain controller, exchange server, VPN & utility servers and then yeah you find out the killer:

Ugh

Windows 10 Home.

Well it turns out that you actually *CAN* install Windows 10 with a little command line shake and bake:

pushd "%~dp0"
dir /b %SystemRoot%\servicing\Packages\*Hyper-V*.mum >hyper-v.txt
for /f %%i in ('findstr /i . hyper-v.txt 2^>nul') do dism /online /norestart /add-package:"%SystemRoot%\servicing\Packages\%%i"
del hyper-v.txt
Dism /online /enable-feature /featurename:Microsoft-Hyper-V -All /LimitAccess /ALL
pause

I picked up this tip from TheWindowsClub. And yep, it works!

Super cool!

Of course this also means you can turn your unsuspecting parent’s home machine into a remote server….

Windows Dev VM

(This is a guest post from Antoni Sawicki aka Tenox)

I have been living under a rock for several years now when it comes to Windows development. Recently wanting to do some maintenance on couple of my projects I needed to download Visual Studio and Windows SDK. Poking around the download page I have discovered that Microsoft now provides a fully pre-installed VMs with Visual Studio, SDK etc. for VMware, Hyper-V, VirtualBox and Parallels. That’s actually super cool and handy. Thank you Microsoft!

Looks like this has been available for 3 or 4 years now. Oh well.

Windows 10 on the Raspberry Pi 4

As far as computers go, Raspberry Pi’s are cheap. The latest (and vastly incompatible) Pi4 is no real exception. Now you’d think Microsoft would want to get WoA (Windows on ARM) into as many hands as possible to get people to port apps to the new cpu architecture. But that is not the case.

As of this moment there is no real desktop machines, the only route to go is with the laptops, which are the few models from Lenovo, HP, Acer and Microsoft themselves. Brand new these things are not cheap, and of course people find out quickly enough that the emulation just isn’t quite there (not all that surprising) and of course the lack of native apps doesn’t help. It’s that chicken/egg problem that can only be solved by getting hardware into people’s hands.

Developers!

So getting back to to the Pi, on ETA Prime’s channel I saw this video, which quickly went over how to get Windows 10 up and running in no time flat.

On the Discord look for the #download-links and look for build 0.2.1 After you extract it, the image should be about 10GB

10,100,932,608 build 0.2.1.img

With a MD5 checksum of: aad51a0e02ba947d24d543ff8ed612b0

Use etcher to write the image to the SD. It took me about 5 minutes to do so. No bigge. I unplugged the SD/IDE/USB adapter thing I’m using, plugged it back in, and used Windows disk manager to expand the partition to take up the rest of the disk. It’s not terribly complicated to setup.

SSD Flashed
Right click and select Extend
The default option will consume the entire disk

After that slap in the SD to your Pi4 and away you go. Or so I first thought.

Realistically you also need:

  • A USB Hub
  • A USB Ethernet adapter (I have some cheapo no name realtek)
  • A USB audio card
  • A mini HDMI to regular human sized HDMI cables/adapters
  • a 5Amp USB charger for extra power!

As I found out rather quickly that the only peripherals that are working is the USB ports. However the USB controller has some DMA bug where it can’t xfer higher than 3GB which caps the current memory ceiling to 3GB.

Otherwise the Pi will think and reboot a few times, and about 15-30 minutes later (I didn’t time it, I walked out) you’ll be up and running Windows 10 on ARM!

It’s heavily customized in that when Windows boots up it is only consuming about 1GB of RAM. So that gives us just under 2GB for user programs. GREAT! Included is the setup program for the latest beta of Microsoft Edge (with the chromium engine) so at least you can actually hit web sites. However Google doesn’t like it, so if you are going to try to watch anything with DRM it will not work.

While many people complain about STEAM, game compatibility, really what on earth were you expecting? Naturally people will want to know how fast it is, and well… It’s not. Although it does have 4 cores, running at 1.5Ghz, there is barely any cache (well compared to an i7/Xeon), and it’s clearly not a power house of a box. The only real test of a machine like this is going to be native stuff. And speaking of, it’s nice that my previous builds for ARM still work! The sales guy that borrwed the ASUS should be back soon so I can do some side by side comparisons of how slow they are.

In addition to DOSBox, Neko98, and frontvm, I managed to get MAME 0.36 cross compiled and I had to disable the DirectX input and output, as although they do compile they have issues on the Pi4. So it’s GDI all the way. That said, it does run:

This isn’t the port you want, or the platform to play it on. I’m using the command line Visual C++ tools to build this, and MAME 0.37 drifted to being more of a MinGW thing, and I just don’t feel like fighting the build process.

Another point of fun, is that this processor & OS does have x86 compatibility you can take things to the extreme with OTVDM, and run Win16 based programs on Windows 10 for ARM! Not that I would know why you want Excel 3.0, but rest assured, it works fine.

Hit:1 http://ports.ubuntu.com/ubuntu-ports bionic InRelease
Get:2 http://ports.ubuntu.com/ubuntu-ports bionic-updates InRelease [88.7 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports bionic-backports InRelease [74.6 kB]
Err:1 http://ports.ubuntu.com/ubuntu-ports bionic InRelease
Unknown error executing apt-key

The platform supports WSL, so I went ahead and installed Ubuntu 18 & 20… and both have one apparently known catastrophic failure on the Pi’s. When trying to update packages the updater crashes. The fault is apparently in dirmngr, or the usual Linux bandaid of switching distros. There doesn’t appear to be any ‘fix’ to this, so if anyone knows what to do, I’m all ears. Also don’t enable WSL2, it’ll hang at the bootloader. I ended up having to reflash the disk.

In my quick conclusion, is this the RISC Windows workstation of the future? No, not really. It’s more the $50(+peripherals) tyre kicker edition. It’s a cheap way into the platform, to see what the fuss is all about. This machine feels like a low end i3, the CPU just isn’t there, it’s only a BCM2711 Cortex-A72, so there isn’t all that much to be expected. On the other hand it’s FAR FAR FAR cheaper than something like the Surface X. If you have the hardware it’s worth checking out if you are interested in non x86 Windows. Otherwise you aren’t going to miss much.

UML Linux on WSL2

Since WSL2 runs an actual Linux kernel that means you can run 32bit binaries! And that now means you can run stuff like ancient unmodified UML kernels on Windows! Isn’t that great!

I followed my old posts here & here, to get running or the short version being:

dpkg --add-architecture i386
apt-get update
apt-get upgrade
apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386 slirp

Download a kernel, linux-2.6.24-rc7.bz2 and a root filesystem: Debian-4.0-x86-root_fs.bz2 and get running right away:

./linux-2.6.24-rc7 ubd0=test_fs eth0=slirp mem=512m

And it just worked!

Upgrading to WSL2

I finally got the update to version 2004 of Windows 10 (OS Build 19041.329) which now includes support for running a Linux kernel inside a VM, using paravirtual drivers letting it hook into Windows 10. It reminds me of WinOS/2 where, OS/2 would run Windows 3.1 in a virtual machine, but using special drivers it could redirect it on the native filesystem, and paint the windows on the OS/2 desktop.

So the first thing is make sure you have 2004, winver should look like this:

You will need to go to the old control panel, and enable the Virtual Machine Platform. For those of you using VMware, you’ll need at least version 15.5.5 (15.5.6 is current as of the time of me writing this). I’d suggest you do that before turning on this, but some people like to live dangerously. Also fully shut down all your VM’s otherwise things will end up giving you errors (as you’ll see later).

After the Virtual Machine Platform is installed, your machine will need to reboot. After a reboot, you’ll need to download the Linux Update to the system, available at:

https://aka.ms/wsl2-install

It’s a simple install, and only takes a few seconds. I didn’t have to reboot.

now for the fun parts, you’ll need to list and shut down (terminate) your running WSL’s if you have any running.

C:\Users\neozeed>wsl --list --verbose
NAME STATE VERSION
Debian Running 1

As you can see I only have one, and it’s called Debian, and it is currently running something. Let’s kill it!

C:\Users\neozeed>wsl -t Debian
C:\Users\neozeed>wsl --list --verbose
NAME STATE VERSION
Debian Stopped 1

And now it’s stopped. The command to upgrade is pretty simple:

C:\Users\neozeed>wsl --set-version Debian 2
Conversion in progress, this may take a few minutes…
For information on key differences with WSL 2 please visit https://aka.ms/wsl2

And yes, you can also stop the WSL instance, and move it back to version 1, if you find 2 not doing what you want. However on first impressions, WSL2 is significantly faster.

If you are running VMware, and you didn’t shut down your VM’s you’ll get this lovely message when you try to resume:

The save state will be deleted, but rest assured it’ll launch okay (at least mine did!)

10.6 the last great OS X

One sad consequence of using the Microsoft Virtual Machine framework is that nesting is no longer available.

which is kind of a bummer, as that was the killer feature.

Sandboxie went GPL3!

I’ve been using Sandboxie for a long while to run all those questionable downloads on Windows. It’s a great light weight sandbox (as the name implies) for running random downloads, or even going to questionable websites as Sandboxie does a great job of isolating processes, and their filesystem access.

It’s not been all that cheap, but I felt it was worth it. I went to check to see how much it is as the conversation had come up on discord, and it turns out that Sophos had bought Sandboxie, and opened up the source code!

The downside is that there will be no further official development of the product. So I guess at some point I’ll have to break down and get a signing cert and re-build it if I want to keep using it.

Last version is locally mirrored here, as I understand it’ll be deleted soon enough.

SandboxieInstall-533-3.exe

Examining Windows 1.0 HELLO.C

The following is a guest post by NCommander of SoylentNews fame!

For those who’ve been long-time readers of SoylentNews, it’s not exactly a secret that I have a personal interest in retro computing and documenting the history and evolution of the Personal Computer. About three years ago, I ran a series of articles about restoring Xenix 2.2.3c, and I’m far overdue on writing a new one. For those who do programming work of any sort, you’ll also be familiar with “Hello World”, the first program most, if not all, programmers write in their careers.

A sample hello world program might look like the following:

#include <stdio.h>


int main() {
 printf("Hello world\n");
 return 0;
}

Recently, I was inspired to investigate the original HELLO.C for Windows 1.0, a 125 line behemoth that was talked about in hush tones. To that end, I recorded a video on YouTube that provides a look into the world of programming for Windows 1.0, and then testing the backward compatibility of Windows through to Windows 10.

For those less inclined to watch a video, my write-up of the experience is past the fold and an annotated version of the file is available on GitHub (https://github.com/NCommander/win1-hello-world-annotations)


Bring Out Your Dinosaurs – DOS 3.3

Before we even get into the topic of HELLO.C though, there’s a fair bit to be said about these ancient versions of Windows. Windows 1.0, like all pre-95 versions, required DOS to be pre-installed. One quirk however with this specific version of Windows is that it blows up when run on anything later than DOS 3.3. Part of this is due to an internal version check which can be worked around with SETVER. However, even if this version check is bypassed, there are supposedly known issues with running COMMAND.COM. To reduce the number of potential headaches, I decided to simply install PC-DOS 3.3, and give Windows what it wants.

You might notice I didn’t say Microsoft DOS 3.3. The reason is that DOS didn’t exist as a standalone product at the time. Instead, system builders would license the DOS OEM Adaptation Kit and create their own DOS such as Compaq DOS 3.3. Given that PC-DOS was built for IBM’s own line of PCs, it’s generally considered the most “generic” version of the pre-DOS 5.0 versions, and this version was chosen for our base. However, due to its age, it has some quirks that would disappear with the later and more common DOS versions.

PC DOS 3.3 loaded just fine in VirtualBox and — with the single 720 KiB floppy being bootable — immediately dropped me to a command prompt. Likewise, FDISK and FORMAT were available to partition the hard drive for installation. Each individual partition is limited, however, to 32 MiB. Even at the time, this was somewhat constrained and Compaq DOS was the first (to the best of my knowledge) to remove this limitation. Running FORMAT C: /S created a bootable drive, but something oft-forgotten was that IBM actually provided an installation utility known as SELECT.

SELECT’s obscurity primarily lies in its non-obvious name or usage, nor the fact that it’s actually needed to install DOS; it’s sufficient to simply copy the files to the hard disk. However, SELECT does create CONFIG.SYS and AUTOEXEC.BAT so it’s handy to use. Compared to the later DOS setup, SELECT requires a relatively arcane invocation with the target installation folder, keyboard layout, and country-code entered as arguments and simply errors out if these are incorrect. Once the correct runes are typed, SELECT formats the target drive, copies DOS, and finishes installation.

Without much fanfare, the first hurdle was crossed, and we’re off to installing Windows.

Windows 1.0 Installation/Mouse Woes

With DOS installed, it was on to Windows. Compared to the minimalist SELECT command, Windows 1.0 comes with a dedicated installer and a simple text-based interface. This bit of polish was likely due to the fact that most users would be expected to install Windows themselves instead of having it pre-installed.

Another interesting quirk was that Windows could be installed to a second floppy disk due to the rarity of hard drives of the era, something that we would see later with Microsoft C 4.0. Installation went (mostly) smoothly, although it took me two tries to get a working install due to a typo. Typing WIN brought me to the rather spartan interface of Windows 1.0.

Although functional, what was missing was mouse support. Due to its age, Windows predates the mouse as a standard piece of equipment and predates the PS/2 mouse protocol; only serial and bus mice were supported out of the box. There are two ways to solve this problem:

The first, which is what I used, involves copying MOUSE.DRV from Windows 2.0 to the Windows 1.0 installation media, and then reinstalling, selecting the “Microsoft Mouse” option from the menu. Re-installation is required because WIN.COM is statically linked as part of installation with only the necessary drivers included; there is no option to change settings afterward. The SDK documentation details the static linking process, and how to run Windows in “slow mode” for driver development, but the end result is the same. If you want to reconfigure, you need to re-install.

The second option, which I was unaware of until after producing my video is to use the PS/2 release of Windows 1.0. Like DOS of the era, Windows was licensed to OEMs who could adapt it to their individual hardware. IBM did in fact do so for their then-new PS/2 line of computers, adding in PS/2 mouse support at the time. Despite being for the PS/2 line, this version of Windows is known to run on AT-compatible machines.

Regardless, the second hurdle had been passed, and I had a working mouse. This made exploring Windows 1.0 much easier.

The Windows 1.0 Experience

If you’re interested in trying Windows 1.0, I’d recommend heading over to PCjs.org and using their browser-based emulator to play with it as it already has working mouse support and doesn’t require acquiring 35 year old software. Likewise, there are numerous write-ups about this version, but I’d be remiss if I didn’t spend at least a little time talking about it, at least from a technical level.

Compared to even the slightly later Windows 2.0, Windows 1.0 is much closer to DOSSHELL than any other version of Windows, and is essentially a graphical bolt-on to DOS although through deep magic, it is capable of cooperative multitasking. This was done entirely with software trickery as Windows pre-dates the 80286, and ran on the original 8086. COMMAND.COM could be run as a text-based application, however, most DOS applications would launch a full-screen session and take control of the UI.

This is likely why Windows 1.0 has issues on later versions of DOS as it’s likely taking control of internal structures within DOS to perform borderline magic on a processor that had no concept of memory protection.

Another oddity is that this version of Windows doesn’t actually have “windows” per say. Instead applications are tiled, with only dialogue boxes appearing as free-floating Windows. Overlapping Windows would appear in 2.0, but it’s clear from the API that they were at least planned for at some point. Most notable, the CreateWindow() function call has arguments for x and y coordinates.

My best guess is Microsoft wished to avoid the wrath of Apple who had gone on a legal warpath of any company that too-closely copied the UI of the then-new Apple Macintosh. Compared to later versions, there are also almost no included applications. The most notable applications that were included are: NOTEPAD, PAINT, WRITE, and CARDFILE.

While NOTEPAD is essentially unchanged from its modern version, Write could be best considered a stripped-down version of Word, and would remain a mainstay until Windows 95 where it was replaced with Wordpad. CARDFILE likewise was a digital Rolodex. CARDFILE remained part of the default install until Windows 3.1, and remained on the CD-ROM for 95, 98, and ME before disappearing entirely.

PAINT, on the other hand, is entirely different from the Paintbrush application that would become a mainstay. Specifically, it’s limited to monochrome graphics, and files are saved in MSP format. Part of this is due to limitations of the Windows API of the era: for drawing bitmaps to the screen, Windows provided Display Independent Bitmaps or DIBs. These had no concept of a palette and were limited to the 8 colors that Windows uses as part of the EGA palette. Color support appears to have been a late addition to Windows, and seemingly wasn’t fully realized until Windows 3.0.

Paintbrush (and the later and confusingly-named Paint) was actually a third party application created by ZSoft which had DOS and Windows 1.0 versions. ZSoft Paintbrush was very similar to what shipped in Windows 3.0 and used a bit of technical trickery to take advantage of the full EGA palette.

With that quick look completed, let’s go back to actually getting to HELLO.C, and that involved getting the SDK installed.

The Windows SDK and Microsoft C 4.0

Getting the Windows SDK setup is something of an experience. Most of Microsoft’s documentation for this era has been lost, but the OS/2 Museum has scanned copies of some of the reference binders, and the second disk in the SDK has both a README file and an installation batch file that managed to have most of the necessary information needed.

Unlike later SDK versions, it was the responsibility of the programmer to provide a compiler. Officially, Microsoft supported the following tools:

  • Microsoft Macro Assembler (MASM) 4
  • Microsoft C 4.0 (not to be confused with MSC++4, or Visual C++)
  • Microsoft Pascal 3.3

Unofficially (and unconfirmed), there were versions of Borland C that could also be used, although this was untested, and appeared to not have been documented beyond some notes on USENET. More interestingly, all the above tools were compilers for DOS, and didn’t have any specific support for Windows. Instead, a replacement linker was shipped in the SDK that could create Windows 1.0 “NE” New Executables, an executable format that would also be used on early OS/2 before being replaced by Portable (PE) and Linear Executables (LX) respectively.

For the purposes of compiling HELLO.C, Microsoft C 4.0 was installed. Like Windows, MSC could be run from floppy disk, albeit it with a lot of disk swapping. No installer is provided, instead, the surviving PDFs have several pages of COPY commands combined with edits to AUTOEXEC.BAT and CONFIG.SYS for hard drive installation. It was also at this point I installed SLED, a full screen editor as DOS 3.3 only shipped with EDLIN. EDIT wouldn’t appear until DOS 5.0

After much disk feeding and some troubleshooting, I managed to compile a quick and dirty Hello World program for DOS. One other interesting quirk of MSC 4.0 was it did not include a standalone assembler; MASM was a separate retail product at the time. With the compiler sorted, it was time for the SDK.

Fortunately, an installation script is provided. Like SELECT, it required listing out a bunch of folders, but otherwise was simple enough to use. For reasons that probably only made sense in 1985, both the script and the README file was on Disk 2, and not Disk 1. This was confirmed not to be a labeling error as the script immediately asks for Disk 1 to be inserted.

The install script copies files from four of the seven disks before returning to a command line. Disk 5 contains the debug build of Windows, which are roughly equivalent to checked builds of modern Windows. Disk 6 and 7 have sample code, including HELLO.C.

With the final hurdle passed, it wasn’t too hard to get to compiled HELLO.EXE.

Dissecting HELLO.C

I’m going to go through these at a high level, my annotated hello.c goes into much more detail on all these points.

General Notes

Now that we can build it, it’s time to take a look at what actually makes up the nuts and bolts of a 16-bit Windows application. The first major difference, simply due to age is that HELLO.C uses K&R C simply on the basis of pre-dating the ANSI C function. It’s also clear that certain conventions weren’t commonplace yet: for example, windows.h lacks inclusion guards.

NEAR and FAR pointers

long FAR PASCAL HelloWndProc(HWND, unsigned, WORD, LONG);

Oh boy, the bane of anyone coding in real mode, near and far pointers are a “feature” that many would simply like to forget. The difference is seemingly simple, a near pointer is nearly identical to a standard pointer in C, except it refers to memory within a known segment, and a far pointer is a pointer that includes the segment selector. Clear right?

Yeah, I didn’t think so. To actually understand what these are, we need to segue into the 8086’s 20-bit memory map. Internally, the 8086 was a 16-bit processor, and thus could directly address 2^16 bits of memory at a time, or 64 kilobytes in total. Various tricks were done to break the 16-bit memory barrier such as bank switching, or in the case of the 8086, segmentation.

Instead of making all 20-bits directly accessible, memory pointers are divided into a selector which forms the base of a given pointer, and an offset from that base, allowing the full address space to be mapped. In effect, the 8086 gave four independent windows into system memory through the use of the Code Segment (CS), Data Segment (DS), Stack Segment (SS), and the Extra Segment (ES).

Near pointers thus are used in cases where data or a function call is in the same segment and only contain the offset; they’re functionally identical to normal C pointers within a given segment. Far pointers include both segment and offset, and the 8086 had special opcodes for using these. Of note is the far call, which automatically pushed and popped the code segment for jumping between locations in memory. This will be relevant later.

HelloWndProc is a forward declaration for the Hello Window callback, a standard feature of Windows programming. Callback functions always had to be declared FAR as Windows would need to load the correct segment when jumping into application code from the task manager. Hence the far declaration. Windows 1.0 and 2.0, in addition, had other rules we’ll look at below.

WinMain Decleration:

int PASCAL WinMain( hInstance, hPrevInstance, lpszCmdLine, cmdShow )
HANDLE hInstance, hPrevInstance;
LPSTR lpszCmdLine;
int cmdShow;

PASCAL Calling Convention

Windows API functions are all declared as PASCAL calling convention, also known as STDCALL on modern Windows. Under normal circumstances, the C programming language has a nominal calling convention (known as CDECL) which primarily relates to how the stack is cleaned up after a function call. In CDECL-declared functions, its the responsibility of the calling function to clean the stack. This is necessary for vardiac functions (aka, functions that take a variable number of arguments) to work as the callee won’t know how many were pushed onto the stack.

The downside to CDECL is that it requires additional prologue and epilogue instructions for each and every function call, thereby slowing down execution speed and increasing disk space requirements. Conversely, PASCAL calling convention left cleanup to be performed by the called function and usually only needed a single opcode to clean the stack at function end. It was likely due to execution and disk space concerns that Windows standardized on this convention (and in fact still uses it on 32-bit Windows.

hPrevInstance

if (!hPrevInstance) {
/* Call initialization procedure if this is the first instance */
if (!HelloInit( hInstance ))
return FALSE;
} else {
/* Copy data from previous instance */
GetInstanceData( hPrevInstance, (PSTR)szAppName, 10 );
GetInstanceData( hPrevInstance, (PSTR)szAbout, 10 );
GetInstanceData( hPrevInstance, (PSTR)szMessage, 15 );
GetInstanceData( hPrevInstance, (PSTR)&MessageLength, sizeof(int) );
}

hPrevInstance has been a vestigial organ in modern Windows for decades. It’s set to NULL on program start, and has no purpose in Win32. Of course, that doesn’t mean it was always meaningless. Applications on 16-bit Windows existed in a general soup of shared address space. Furthermore, Windows didn’t immediately reclaim memory that was marked unused. Applications thus could have pieces of themselves remain resident beyond the lifespan of the application.

hPrevInstance was a pointer to these previous instances. If an application still happened to have its resources registered to the Windows Resource Manager, it could reclaim them instead of having to load them fresh from disk. hPrevInstance was set to NULL if no previous instance was loaded, thereby instructing the application to reload everything it needs. Resources are registered with a global key so trying to register the same resource twice would lead to an initialization failure.

I’ve also gotten the impression that resources could be shared across applications although I haven’t explicitly confirmed this.

Local/Global Memory Allocations

NOTE: Mostly cribbled off Raymond Chen’s blog, a great read for why Windows works the way it does.

pHelloClass = (PWNDCLASS)LocalAlloc( LPTR, sizeof(WNDCLASS) );
LocalFree( (HANDLE)pHelloClass );

Another concept that’s essentially gone is that memory allocations were classified as either local to an application or global. Due to the segmented architecture, applications have multiple heaps: a local heap that is initialized with the program and exists in the local data segment, and a global heap which requires a far pointer to make access to and from.

Every executable and DLL got their own local heaps, but global heaps could be shared across process boundaries, and as best I can tell, weren’t automatically deallocated when a process ended. HEAPWALK could be used to see who allocated what and find leaks in the address space. It could also be combined with SHAKER which rearranged blocks of memories in an attempt to shake loose bugs. This is similar to more modern-day tools like valgrind on Linux, or Microsoft’s Application Testing tools.

MakeProcInstance

lpprocAbout = MakeProcInstance( (FARPROC)About, hInstance );

Oh boy, this is a real stinker and an entirely unnecessary one at that. MakeProcInstance didn’t even make it to Windows 3.1 and its entire existence is because Microsoft forgot details of their own operating environment. To explain, we’re going to need to dig a bit deeper into segmented mode programming.

MakeProcInstance’s purpose was to register a function suitable as a callback. Only functions that have been marked with MPI or declared as an EXPORT in the module file can be safely called across process boundaries. The reason for this is that Windows needs to register the Code Segment and Data Segment to a global store to make function calls safely. Remember, each application had its own local heap which lived in its own selector in DS.

In real mode, doing a CALL FAR to jump to a far pointer automatically push and popped the code segment as needed, but the data segment was left unchanged. As such, a mechanism was required to store the additional information needed to find the local heap. So far, this is sounding relatively reasonable.

The problem is that 16-bit Windows has this as an invariant: DS = SS …

If you’re a real mode programmer, that might make it clear where I’m going with this. The Stack Segment selector is used to denote where in memory the stack is living. SS also got pushed to the stack during a function call across process boundaries along with the previous SP. You might begin to see why MakeProcInstance becomes entirely unnecessary.

Instead of needing a global registration system for function calls, an application could just look at the stack base pointer (bp) and retrieve the previous SS from there. Since SS = DS, the previous data segment was in fact saved and no registration is required, just a change to how Windows handles function epilogs and prologs. This was actually found by a third party, and a tool FixDS was released by Michael Geary that rewrote function code to do what I just described. Microsoft eventually incorporated his fix directly into Windows, and MakeProcInstance disappeared as a necessity.

Other Oddities

From Raymond Chen’s blog and other sources, one interesting aspect of 16-bit Windows was it was actually designed with the possibility that applications would have their own address space, and there was talk that Windows would be ported to run on top of XENIX, Microsoft’s UNIX-based operating system. It’s unclear if OS/2’s Presentation Manager shared code with 16-bit Windows although several design aspects and API names were closely linked together.

From the design of 16-bit Windows and playing with it, what’s clear is this was actually future-proofing for Protected Mode on the 80286, sometimes known as segmented protection mode. On 286’s Protected Mode, while the processor was 32-bit, the memory address space was still segmented into 64-kilobyte windows. The primary difference was that the segment selectors became logical instead of physical addresses.

Had the 80286 actually succeeded, 32-bit Windows would have been essentially identical to 16-bit Windows due to how this processor worked. In truth, separate address spaces would have to wait for the 80386 and Windows NT to see the light of day, and this potential ability was never used. The 80386 both removed the 64-kilobyte limit and introduced a flat address space through paging which brought the x86 processor more inline with other architectures.

Backwards Compatibility on Windows 3.1

While Microsoft’s backward compatibility is a thing of legend, in truth, it didn’t actually start existing until Windows 3.1 and later. Since Windows 1.0 and 2.0 applications ran in real mode, they could directly manipulate the hardware and perform operations that would crash under Protected Mode.

Microsoft originally released Windows 286, and 386 to add support for the 80286 and 80386, functionality that would be merged together in Windows 3.0 as Standard Mode, and 386 Enhanced Mode along with legacy “Real Mode” support. Due to running parts of the operating system in Protected Mode, many of the tricks applications could perform would cause a General Protection Fault and simply fail. This wasn’t seen as a problem as early versions of Windows were not popular, and Microsoft actually dropped support for 1.x and 2.x applications in Windows 95.

Windows for Workgroups was installed in a fresh virtual machine, and HELLO.EXE, plus two more example applications, CARDFILE and FONTTEST were copied with it. Upon loading, Windows did not disappoint throwing up a compatibility warning right at the get-go.

Accepting the warning showing that all three applications ran fine, albeit it with a broken resolution due to 0,0 being passed into CreateWindow().

However, there’s a bit more to explore here. The Windows 3.1 SDK included a utility known as MARK. MARK was used, as the name suggests, to mark legacy applications as being OK to run under Protected Mode. It also could enable the use of TrueType fonts, a feature introduced back in Windows 3.0.

The effect is clear, HELLO.EXE now renders in TrueType fonts. The reason TrueType fonts are not immediately enabled can be see in FONTTEST, where the system typeface now overruns several dialog fields.

The question now was, can we go further?

35 Years Later …

As previously noted, Windows 95 dropped support for 1.x and 2.x binaries. The same however was not true for Windows NT, which modern versions of Windows are based upon. However, running 16-bit applications is complicated by the fact that NTVDM is not available on 64-bit installations. As such, a fresh copy of Windows 10 32-bit was installed.

Some pain was suffered convincing Windows that I didn’t want to use a Microsoft account to sign in. Inserting the same floppy disk as used in the previous test, I double-clicked HELLO and Feature Installer popped up asking to install NTVDM. After letting NTVDM install, a second attempt shows, yes, it is possible to run Windows 1.x applications on Windows 10.

FONTTEST also worked without issue, although the TrueType fonts from Windows 3.1 had disappeared. CARDFILE loaded but immediately died with an initialization error. I did try debugging the issue and found WinDbg at least has partial support for working with these ancient binaries, although the story of why CARDFILE dies will have to wait for another day.

In Closing …

I do hope you enjoyed this look at ancient Windows and HELLO.C. I’m happy to answer questions, and the next topic I’m likely going to cover is a more in-depth look at the differences between Windows 3.1 and Windows for Workgroups combined with demonstrating how networking worked in those versions.

Any feedback on either the article, or the video is welcome to help me improve my content in the future.

Until next time,

73 de NCommander