Running the SWGEMU server on WSL

All alone

The topic had come up again about running Core3 aka the Star Wars Galaxies EMUlated server at home. It’s always been a somewhat involved setup, but with the advent of the Windows Subsystem for Linux, there exists the ability to export and import filesystems as if they were containers. With that in hand I went ahead and set one up.

I initially targeted the older WSL v1, as it supports far more hardware, and I didn’t want to leave so many behind. The added benefit of targeting v1, is that the end user can upgrade the filesystem to v2 if they so choose. The only real difference in this project between v1 & v2 is that v1 cannot run the needed MySQL instance. This isn’t a showstopper however as there exists MySQL & MariaDB native versions for Windows.

Obviously, you’ll need Windows 10 or 11 to do this. From what I can tell even the home version of Windows 10 has WSL capabilities, so this doesn’t pose any problems. Simply open the legacy control panel (Windows+R then type in ‘control’), and go to the Programs.

and then ‘Turn Windows features on or off’, scroll down and select ‘Windows Subsystem for Linux’.

Hit OK, and it’ll download what is needed from the internet.

This shouldn’t take too long, then it’ll want to reboot the computer.

Now you just have to wait for Windows to reboot.

Now you’ll need to download the filesystem. I’ve put it on archive.org for the time being right here: wslv1-SWGEMU_Ubuntu20.22 directory listing (archive.org)

At a minimum you’ll need the files:

  • swgemu-ubuntu.tar.gz
  • mariadb-10.10.2-winx64.msi

Keep in mind that the filesystem is 4.8GB. It will take some time to download. If the torrent servers are functioning I would highly recommend that.

Once it’s installed and it’s generated it’s own databases it’ll need just under 7GB of disk space. So figure that you need 15GB free for this fun experiment.

The first step is to install the database server. We will need to know the settings in order to configure the swgemu server.

You can use any MySQL, but Maria is fully GPL, and small, and more importantly doesn’t involve Oracle.

The install is pretty striaghtfoward.

GNU software requires it reproduce the license sooo here we are.

I would HIGHLY recommend use the default option of install everything.

It will ask for a root password. In this example I’m using ‘password’, of course you can use anything you want, as long as you remember it. You don’t have to allow remote machine access to the database as WSLv1 runs in the same userspace on Windows, so it’ll effectively be connecting to itself.

I use TCP port 10000 as I want the default port 3306 for another insance. Again it’s all up to you but you will have to remember what port you put it on. Ports below 1000 are highly NOT recommended as they need special permission. I left the buffer pool as default

With those options set we can let it do the install.

Now the database is installed, and we can go onto restoring the WSL image, and then configuring the core3 server

Firstly we need place to put it, On the machine I built it on it was the D: drive, but for this re-deploy on a new machine I’m using the C: drive. It’s all up to you but it MUST be a locally attached disk, no USB disks.

Simply create the target directory and run the import command something like this:

md c:\starwars
wsl --import swgemu c:\starwars Downloads\swgemu-ubuntu.tar.gz
I only have the one disk on this VM

This will take a while.

Once it’s done you can check the subsystem status:

C:\Users\Administrator>wsl --list -v
  NAME      STATE           VERSION
* swgemu    Stopped         1

You may have more than one installed, so we’ll refer to this one by it’s name ‘swgemu’.

You can start the instance with wsl -d swgemu:

C:\Users\Administrator>wsl -d swgemu
Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 4.4.0-19041-Microsoft x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Thu Dec  8 09:02:39 EST 2022

  System load:    0.5185546875   Processes:             7
  Usage of /home: unknown        Users logged in:       0
  Memory usage:   28%            IPv4 address for eth0: 198.46.174.160
  Swap usage:     0%


0 updates can be applied immediately.



This message is shown once a day. To disable it please create the
/root/.hushlogin file.

You will need to edit the script to create the databases. I like using vi, but you may not. The easiest way is to just browse to the location and open it up in notepad. The path to browse is:

C:\starwars\rootfs\root\starwars\sql

You can drag/drop re-init.sh on notepad, or make the file association, it’s all up to you.

I had set the SQLHOST to the LAN IP I have, you should set it to 127.0.0.1 . The PUBLIC address is the one that you’ll be connecting to. You can set it to your LAN address, or a public one if you know it (you’ll have to setup NAT), or if you are running it all on one machine for testing, it’s fine to be 127.0.0.1.

Save the file re-init.sh and now you have to run it.

[email protected]:~# cd starwars/sql/
[email protected]:~/starwars/sql# ./re-init.sh
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1008 (HY000) at line 1: Can't drop database 'swgemu'; database doesn't exist
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
[email protected]:~/starwars/sql#

The first SQL query will attempt to drop the swgemu database. In this case it doesn’t exist, so the first error is expected. Nothing to worry about. That gives us the database, now it’s time to configure the server.

You can use explorer to browse to:

C:\starwars\rootfs\root\starwars\conf

Now open/drop on notepad the file config.lua

Again change the DBHost, and MantisHost to 127.0.0.1, and update the password for root, if you have changed it. You may also want to change the DBSecret, but since this is all for fun, I’m leaving it stock.

Now it’s time to run the server.

[email protected]:~/starwars/sql# cd ..
[email protected]:~/starwars# ./core3
(0 s) [Core] Server start, pid: 105, time: Thu Dec  8 09:11:54 2022
(0 s) [ConfigManager] Did not find conf/config-local.lua
(0 s) [DatabaseManager] converting database objects to new format
(0 s) [DatabaseManager] finished converting data to new format

There will be a lot of messages going on, and this will take a few minutes. During the first run Windows will ask for network access permissions. Be sure to check both boxes allowing WAN and LAN access.

Check both boxes!

I have a Lenovo C20 and it takes about 2 minutes, depending on your config it may take longer or shorter.

One thing to point out is that I have configured Core3 for all the worlds, and this will need more than 8GB of ram to start up. So if you get something like this:

If this happens, you’ll have to try a minimal config, commenting out the areas from the config. On this test system, it only has 8GB of RAM, and that isn’t enough to have everything. However the minimal of the training & Tatooine fits no problem:

On the machine I originally set this up on, it has 32GB of RAM, so the processes needing just under 7GB of RAM wasn’t an issue. However an 8GB machine just isn’t enough as I found out the hard way re-producing my install.

Since we have previously allowed the Core3 application LAN/WAN access you should be able to connect. If you set the public address to something public, you will need to setup your firewall to forward the following ports to the machine hosting Core3

Otherwise you’ll need to open the following ports and protocols:

TCP 44419
TCP 44455
UDP 44453
UDP 44462
UDP 44463

Again making this a WSLv1 setup makes this easier as Windows runs core3 in the same memory space.

With any luck you should now be able to connect to the address you have specified as your PUBLIC address. If you want to change, you can either update the re-init.sh script and re-run it, destroying everything, or run the HeidiSQL tool, and just update it.

I had to set it to use libmysql.dll . As always remember your root password, and what Port you used to setup MariaDB. With the profile saved you can open it up.

The address field tells the clients what IP address they can find the server on, so it can the the LAN if you are staying at home, or it has to be a Publicly accessible one. Another alternative is something like Hamachi, which lets you create your own virtual network of machines. It’s great, but not all that cheap, but the added benefit is that your server isn’t a public resource, so you know exactly who is connecting.

If you want to be open but in control I’ve used this in the past to great success.

If you do change the IP you have to restart the core3 program.

Simply type in ‘exit’ at the console and it’ll start to gracefully shutdown.

Then simply start it again with a ./core3

If you want to enable the god mode, I assume you have read how what changes to make on the client side, and the table to edit is the accounts table, once you have logged in, it’ll create your record, and you can find it on HeidiSQL and change the admin_level field to 15. With this set the next character you create will be a ‘god’.

I’m not going to go into setting up the client, installing the SWGEMU stuff and how to add custom servers, as I assume you already know how to do so. This is more to let people run their own instances at home, or among friends.

On building core3 for yourself, be aware that by default it’ll use a ‘native’ config option that tunes the compiler to generate code as fast as possible for YOUR processor. So if you want to run it somewhere else, it has a good chance of not running. I had to edit the following files:

./CMakeLists.txt
./utils/engine3/MMOEngine/CMakeLists.txt

And change NATIVE to NO.

I also had issues with the SQL not having a default setting for timestamps, and it would fail to create accounts. Look for stuff like this:

TIMESTAMP NOT NULL,

and replace it with

TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,

And as of this current rev I had some issues with malloc hooks, which I just commented out:

$ diff -ruN AllocationHook.cpp_ utils/engine3/MMOEngine/src/system/mm/AllocationHook.cpp
--- AllocationHook.cpp_ 2022-12-06 07:59:42.160402900 +0000
+++ utils/engine3/MMOEngine/src/system/mm/AllocationHook.cpp 2022-12-06 08:00:22.510402900 +0000
@@ -50,6 +50,7 @@
        instance = this;

 #ifndef PLATFORM_MAC
+#if 0
        __saved_malloc_hook = __malloc_hook;
        __saved_free_hook = __free_hook;
        __saved_realloc_hook = __realloc_hook;
@@ -58,6 +59,7 @@
        __malloc_hook = mallocHook;
        __free_hook = freeHook;
        __realloc_hook = reallocHook;
+#endif

 #else
        malloc_zone_t *dz = malloc_default_zone();
@@ -91,9 +93,11 @@

 void AllocationHook::uninstall() {
 #ifndef PLATFORM_MAC
+#if 0
        __malloc_hook = __saved_malloc_hook;
        __free_hook = __saved_free_hook;
        __realloc_hook = __saved_realloc_hook;
+#endif
 #else
        malloc_zone_t *dz = malloc_default_zone();

$

Yes it’s terrible, but I was in a rush so.. yeah.

Also I have access to a Windows 2019 Server which is Windows 10 1809 based, and it doesn’t support the import-export of filesystems. So this won’t be a clean install, I guess it’d be easier to copy files over from Windows 10, and install Ubuntu 20.22 from the store.

Good Luck!

pptp always ignoring localip on Ubuntu

I don’t know how the other various linux distros handle this but I found this by accident:

Nov 17 12:04:25 ukweb pppd[4943]: Using interface ppp0
Nov 17 12:04:25 ukweb pppd[4943]: Connect: ppp0 <--> /dev/pts/0
Nov 17 12:04:25 ukweb pptpd[4942]: GRE: Bad checksum from pppd.
Nov 17 12:04:25 ukweb systemd-udevd[4944]: Using default interface naming scheme 'v249'.
Nov 17 12:04:25 ukweb pppd[4943]: peer from calling number 1.1.1.1.1 authorized
Nov 17 12:04:25 ukweb pppd[4943]: MPPE 128-bit stateless compression enabled
Nov 17 12:04:27 ukweb systemd-networkd[592]: ppp0: Link UP
Nov 17 12:04:27 ukweb systemd-networkd[592]: ppp0: Gained carrier
Nov 17 12:04:27 ukweb pppd[4943]: found interface br0 for proxy arp
Nov 17 12:04:27 ukweb pppd[4943]: local  IP address 192.168.0.1
Nov 17 12:04:27 ukweb pppd[4943]: remote IP address 192.168.23.10
Nov 17 12:05:28 ukweb systemd[1]: Stopping PoPToP Point to Point Tunneling Server...
Nov 17 12:05:28 ukweb pppd[4943]: Terminating on signal 15
Nov 17 12:05:28 ukweb pppd[4943]: Connect time 1.1 minutes.
Nov 17 12:05:28 ukweb pppd[4943]: Sent 0 bytes, received 6937 bytes.
Nov 17 12:05:28 ukweb systemd-networkd[592]: ppp0: Link DOWN
Nov 17 12:05:28 ukweb systemd-networkd[592]: ppp0: Lost carrier

With the emphasis on “local IP address 192.168.0.1”. Which is *NOT* in my config. I went as far as adding a bridge to satisfy the proxy arp! Netplan is some yaml thing and yeah not a big fan.

    ethernets:
        eth0:
            addresses:
            - PUBLICIP/24
            gateway4: GATEWAY
            match:
                macaddress: AA:BB:CC:00:00:01
            nameservers:
                addresses:
                - 1.1.1.1
                - 8.8.8.8
    bridges:
      br0:
        dhcp4: no
        addresses: [192.168.23.1/24]
    version: 2

my /etc/ppp/pptpd.conf had the options set, but no matter what it *ALWAYS* went to 192.168.0.1

option /etc/ppp/pptpd-options
logwtmp
localip 192.168.24.1
remoteip 192.168.23.30-250

And then I found it after doing what i should have done, and grep around to find out that pptpd.conf should actually live in /etc

Yeah that’s right, there is 2 of them although they should be the same. A symlink and a restart later, and now I get this:

Nov 17 12:19:56 ukweb kernel: [  112.718861] PPP MPPE Compression module registered
Nov 17 12:19:56 ukweb pppd[1002]: MPPE 128-bit stateless compression enabled
Nov 17 12:19:58 ukweb systemd-networkd[599]: ppp0: Link UP
Nov 17 12:19:58 ukweb systemd-networkd[599]: ppp0: Gained carrier
Nov 17 12:19:58 ukweb pppd[1002]: found interface br0 for proxy arp
Nov 17 12:19:58 ukweb pppd[1002]: local  IP address 192.168.23.1
Nov 17 12:19:58 ukweb pppd[1002]: remote IP address 192.168.23.10

MUCH much better. I don’t know if this is anything worth wriging about, but if I can save someone else an hour of wondering why the config isn’t working and why their pptp is always defaulting to 192.168.0.1 and why it’s wreaking havoc with any default home router, where here it is.

Networking on AIX 4.3

Well oslevel says 4.3.3.0, but you get the idea.

You’ll need to have the ethernet driver handy, or better loaded. Since I had disabled the NIC on install it’s not loaded. And since I’m still using a cellphone for internet I extracted the file somewhere else and copied in some patches. I’ve managed to reproduce this twice now, so I guess it’s good to go. Apparently, this just works in later versions, but this is very touchy.

To start how I’m running qemu:

./qemu-build/ppc-softmmu/qemu-system-ppc -M 40p -bios q40pofw-serial.rom -serial telnet::4441,server -hda disk0.vmdk-post-install -vga none -nographic -net none -device pcnet,netdev=ne -netdev user,id=ne,hostfwd=tcp::42323-:23 -cdrom /mnt/c/temp/pcnet-aix.iso

With aix booted, extract the tar file from the cdrom:

mount /cdrom
mkdir /pcnet
cd pcnet
tar -xvf /cdrom/pci.tar

Fix your terminal up… if needed (it probably is)

export TERM=vt100
stty erase ^?
export LIBPATH=$LIBPATH:/usr/lib
export PATH=/usr/local/bin:$PATH

Now run smitty -> devices -> after ipl

Change the directory to /pcnet , and let it run It will give errors but thats okay. All being well it won’t crash AIX, otherwise you’ll want to restore your hardisk. You did make a backup beforehand right?!

I don’t think it matters but I run this afterwards:

odmchange -o CuAt -q "name=ent0 and attribute=busio" /cdrom/lance_ch.asc
odmget -q "name=ent0 and attribute=busio" CuAt
shutdown -h now
halting does take forever

As tempting as it is to kill the emulator, wait for it to complete. Otherwise you may have to do the whole thing agian.

For me the value attribute was never preserved, so we get to do it again on reboot/restart:

odmget -q "name=ent0 and attribute=busio" CuAt
mount /cdrom
odmchange -o CuAt -q "name=ent0 and attribute=busio" /cdrom/lance_ch.asc
rmdev -l ent0
mkdev -l ent0
ifconfig en0 10.0.2.15
ping -c 1 10.0.2.2

If everything went well this time you should get a ping reply! Great! Now to configure the system for real.

smitty -> communication -> tcpip -> minimum -> en0

simple slirp

As always I configure my system for slirp. We’re almost there! Now to pad the DNS records for slirp:

cat >> /etc/hosts
10.0.2.2 slirp
10.0.2.3 slirpdns
^D

And now you can reboot!

If everything goes well, you will have a patched up pcnet driver that works (well mine does)

It works!

The big test is to of course reboot. Then you’ll know for sure.

I have tried this a few times, and yeah it can crash when adding the drivers, so I had to restore a few times. I would say 1/3 times worked flawlessly. So be patient. And backup!

Windows scp to remote machines with spaces in the directories

Well one nice thing about Windows 10 is that it has a built in ssh/scp client! Although telnet is optional, I get that it’s insecure but jeez what is a retro user to do?

Anyways the subject at hand is copying files from somewhere that has spaces in the path. In this case I need a copy of OS X Snow Leopard from my Mac Pro cylinder to this junk Fujitsu Celsius. I’m still having USB issues, but I’d like to get my data off of an external disk formatted in HFS+. And for ‘reasons’ I wanted to use something “native” but I don’t feel like building a Hackintosh. While not a strict tutorial on getting Snow Leopard running, I did upload my old download of Empire EFI on archive.org as this kind of stuff is damned near impossible to find.

So back to the matter at hand, I have this VM setup on my Mac Pro, and I want it on this Windows machine. You’d think it would be something like this:

scp -C [email protected]:"/Users/neozeed/Virtual Machines.localized/OSX 10.6/*" .
Password:
scp: /Users/neozeed/Virtual: No such file or directory
scp: Machines.localized/OSX: No such file or directory
scp: 10.6/*: No such file or directory

Okay so double quotes didn’t work. How about a Unix style escape for spaces? I mean it *is* scp after all, maybe it doesn’t know it’s on Windows.

C:\osx>scp -C [email protected]:"/Users/neozeed/Virtual\ Machines.localized/OSX\ 10.6/*" .
Password:
scp: /Users/neozeed/Virtual: No such file or directory
scp: Machines.localized/OSX: No such file or directory
scp: 10.6/*: No such file or directory

Well maybe it parses it like C, so you need double backslash? NO that doesn’t work either. Talk about frustrating. So, in an act of insanity, I tried single quoting the interior spaces around double quotes, something idiotic like a bash variable:

C:\osx>scp -C [email protected]:"/Users/neozeed/Virtual' 'Machines.localized/OSX' '10.6/*" .
Password:
Mac OS Snow Leopard.vmdk                                                               69%   11GB  16.0MB/s   05:16 ETA

And yes, now it’s transferring. I’m just using a cheap 50zt 5 port 100Mbit dumb switch. It’s good enough and it’ll probably take some 30 minutes to transfer all the bits, but it’s working.

So there you go. You may not need it now, or tomorrow but it’ll save you the 20 minutes of frustration!

Web Rendering Proxy – Full Page Scrolling

(This is a guest post by Antoni Sawicki aka Tenox)

Due to a popular demand I have added an option of generating full page height screenshot and allowing client browser to do the scrolling.

This makes the browsing experience much smoother, you have resources for it. Beware, a full page screenshot can be several MB in size encoded as gif/png and much more as a decoded raw bitmap on the client. I managed to crash Mosaic and OmniWeb a few times. Fortunately typical Wikipedia page is under 1 MB so for most part is should be fine. To activate just put 0 in page Height.

I have drafted a pre-release on github for testing. Please let me know any feedback. I’m also thinking whether enable this by default, or not.

WRP 4.0 Preview

(This is a guest post from Antoni Sawicki aka Tenox)

Welcome a completely new and absolutely insane mode of Web Rendering Proxy. ISMAP on steroids!

While v3.0 was largely just a port from Python/Webkit to GoLang/Chromedp, the new version is a whole new game. Previously WRP worked by walking the DOM and making a clickable imagemap out of <A HREF> nodes. Version 4.0 works by using x,y coordinates obtained from ISMAP to perform a simulated mouse click in Chrome browser. This way you can click on any element of the page. From annoying cookie warnings, to various drop down menus and even play some online games. Also pagination has been replaced with a clickable scroll bar.

Enough talking, you can watch this video:

Or download the new version and try it yourself!

Please report bugs on github.com. Thank you!

WRP 3.0 Beta ready for testing

(This is a guest post from Antoni Sawicki aka Tenox)

I have released WRP 3.0 for testing. It’s currently a browser-in-browser server rather than a true proxy, but that’s in the works. Please try it out and let me know. Usage instructions are on the main github project page.

Today using trickery I was able to login to my reddit account from Mosaic:

Update: just added the missing image quantizer so that the color number input box actually does something useful. Now you can browse porn even with 16 colors:

WRP Runs on Windows

(This is a guest post by Antoni Sawicki aka Tenox)

Thats right, the new beta version of Web Rendering Proxy runs natively on Windows. Single EXE, no libraries or dependencies required. Only Chrome Browser.

I took a Internet Explorer 1.5 for a spin today while WRP was running on my Windows 10 PC. Worked just fine.

I have added Prev/Next buttons so that you can easily “scroll” through long pages.

ISMAP support has been added, proof:

You can download a preview build on github.

Web Rendering Proxy – Overdue Status Update

(This is a guest post from Antoni Sawicki aka Tenox)

There hasn’t been a major update to WRP (Web Rendering Proxy) in 5 years or so. Some new features have been added thanks to efforts of Claunia but the whole project was mostly impeded with mass migration of the whole Internet to SSL/TLS/https. It does semi work somehow thanks to sslstrip but the whole stack is an unmaintainable pile of crap which I’m not going to update any more.

A new rewrite from scratch is well under way. This time written in GoLang and using Chrome DevTools Protocol. Things should be much more stable and future proof.

Far from complete but I have a fully functional prototype now working in just under 100 lines of code:

UPDATE 1: You can play with it if you want. Please do not submit any bug reports just yet, as this is just a development version. Note that WRP is currently not a true HTTP proxy but rather browser-in-browser. Proxy may be supported later.

UPDATE 2: As of today online setting of size, scaling and scrolling is supported. I’m specifically happy about the scrolling feature albeit it probably needs a better user input, like prev/next page.

Windows version still doesn’t work due to an upstream bug, which is probably easy to fix.

ISMAP is currently in development.