Re-visiting Gopher on A/UX

Rather unintentionally some 7 years ago (to the day!) I was playing with an early gopher server on Linux, musing that one day it’d be cool to run it fully on A/UX (what is it anyways?!). And thanks to Qemu’s 68040 support the time is at hand.

First off I need to run this on Linux so I’ll need to build the appropriate branch myself. Thankfully Cat_7 has boiled it down to a really simple formula:

git clone -b q800.upstream https://www.gitlab.com/mcayland/qemu q800-upstream
cd q800-upstream
./configure --target-list=m68k-softmmu --enable-gtk --enable-sdl
make

In my case I remove the gtk and sdl as I’m running this headless.

Now onto the OS itself. While I had numerous images built over the years for Shoebill there was one major issue when compared to Qemu, and that is Shoebill loads the kernel directly while Qemu emulates the hardware so it will boot MacOS 7 directly. While on the surface this is mundane that does mean however that none of my images will actually work on Qemu as they don’t include a blessed copy of System 7. Not that I care that much I could always do a simple dump/restore [ dump.bsd 0f – /dev/rdsk/c0d0s0 | (cd /mnt; restore xf -) ] of my A/UX stuff that I care about anyways. Luckily since I had added that SCSI file support to Cockatrice I could still partition out some disks and install from there.

Now for the further bit of bad news for me is that I found that the 68020 based Shoebill ran 3.0.0 far more stable than 3.0.1 or 3.1. So I’d built everything around 3.0.0. And of course trying to boot 3.0.0 on a Quadra 800 just gives you a hard lock up. I don’t have the setup disk for 3.0.0 but mounting the CD-ROM gives you access to the disk tool (the 3.0.0 version doesn’t check for the Apple string on SCSI ROMS so you can partition with that as well). Anyways too much time thinking I’d done something wrong until this had to be pointed out to me:

Compatibility matrix from penelope

That’s right, 3.0.0 doesn’t run on the Quadra 800. Much longer ago I had a Quadra 950, fantastic beast of a machine, and yes it ran 3.0.0 just great. So shockingly running the right versions got me up to a working system just fine.

Now of course back in the Shoebill days I got ‘3.0.1’ kind of working by cheating. The /mac programs didn’t work on Shoebill however I could copy them over from 3.0.0 to get a working system. Could I substitute a 3.0.1 kernel & /mac directory onto a 3.0.0 system?

So first up the System 7 install from A/UX 3.0.0 is too old for a Quadra 800. Obviously just use the one from 3.0.1. Great.

This lead to a problem where the root filesystem always needs to be checked in single user mode. Something that is shockingly hard to do when your Quadra runs so fast as you have less than a second to hit the ‘top’ button to halt the autoload.

Naturally the standalone runs fine, with no errors.

Thinking that it’s the start-up scripts I remove all the fsck’s and then get this message:

Great a kernel panic. ialloc: dup alloc. Thinking that maybe it’s confusing the UFS, I go ahead and format the disk in SYSV and restore the image onto that.

This gets me another kernel panic, this time no root filesystem. Surprise the SYSV filesystem was made optional in a default install. I run ‘newconfig sysv’ from 3.0.1 and copy that kernel back, and for good measure the shared libraries from 3.0.1. Now I get a different error:

Interesting, I try to hit restart, and instead I get dumped into text mode!

Victory!

So here we are a 3.0.1 kernel with a 3.0.0 userland! I’m going to use this as a server anyways so I don’t really care about the Mac UI. Naturally so many twists and turns I’ll just skip to the end. Networking didn’t work correctly. Maybe I should have copied all the network stuff from 3.0.1 over but at this point it’s basically a 3.0.1 system so why even bother?

So the next thing of course is just to setup Qemu to listen on a loopback and add some disks. A lot of disks.

./qemu-system-m68k \
-L pc-bios \
-m 256 \
-M q800 \
-vnc 10.11.0.1:35 \
-serial stdio \
-bios Quadra800.rom \
-net nic,model=dp83932,netdev=ne -netdev user,id=ne,hostfwd=tcp:10.11.0.1:42323-:23,hostfwd=tcp:10.11.0.1:40070-:70,hostfwd=tcp:10.11.0.1:40080-:80 \
-drive file=pram-aux.img,format=raw,if=mtd \
-device scsi-hd,scsi-id=0,drive=hd0,vendor="SEAGATE",product="ST225N",ver="1.0" \
-drive file=scsi0.vmdk,media=disk,format=vmdk,if=none,id=hd0 \
-device scsi-hd,scsi-id=1,drive=hd1,vendor="SEAGATE",product="ST225N",ver="1.1" \
-drive file=scsi1.vmdk,media=disk,format=vmdk,if=none,id=hd1 \
-device scsi-hd,scsi-id=2,drive=hd2,vendor="SEAGATE",product="ST225N",ver="1.2" \
-drive file=scsi2.vmdk,media=disk,format=vmdk,if=none,id=hd2 \
-device scsi-hd,scsi-id=3,drive=hd3,vendor="SEAGATE",product="ST225N",ver="1.3" \
-drive file=scsi3.vmdk,media=disk,format=vmdk,if=none,id=hd3 \
-device scsi-hd,scsi-id=4,drive=hd4,vendor="SEAGATE",product="ST225N",ver="1.4" \
-drive file=scsi4.vmdk,media=disk,format=vmdk,if=none,id=hd4 \
-device scsi-hd,scsi-id=5,drive=hd5,vendor="SEAGATE",product="ST225N",ver="1.5" \
-drive file=scsi5.vmdk,media=disk,format=vmdk,if=none,id=hd5 \
-device scsi-hd,scsi-id=6,drive=hd6,vendor="SEAGATE",product="ST225N",ver="1.6" \
-drive file=scsi6.vmdk,media=disk,format=vmdk,if=none,id=hd6
Yeah well… great!?

One nice thing is that since we are on Qemu I don’t have to use raw disk images, I can zero stuff out and use VMDK’s. Nice. I guess I could bridge the VM later, but for now NAT is fine enough as all I need is telnet & gopher. So I grab gopher2_3.1.tar.gz, rebuild and move over my gopher site from Linux into A/UX and I’m up and running in no time. It was shockingly easy. I update a few things to reflect it running on A/UX now.

Currently 2 days of uptime!

And just like that I took my semi popular gopher site, and moved it to A/UX seven years after thinking that this would be a ‘good idea(tm)’. I’m sure it won’t backfire spectacularly.

I don’t know if any of this is useful or interesting but it was to me. It’s been nice that Qemu has been able to keep uptime in several days, I had 3 days of uptime before I took it down to max out the storage so I could possibly do more with it.

Naturally it’s still available as gopher://gopher.superglobalmegacorp.com

Apparently I’m notable on Gopherspace

I found this map by accident while telling someone what the internet was like before the advent of HTTP. Behold Matt Owen’s handiwork!

And for those who can’t see it…

I have to suspect it’s due to the name, and how long it’s been up. As you can see from the diagram not a lot actually links to there. I think it may also seem like a more ‘significant’ site than it really is, as I put the MSPL documents on there.

And to add to even more Gopher madness, he’s written an excellent client! You can find it here: http://www.jaruzel.com/gopher/gopher-client-browser-for-windows

I have to say that first impressions are pretty great. Although looking at the page makes me realize just how much is broken, and cruft. Kind of a sad state I’ve ended up in, since I moved my BBS to VMware, where the disk subsequently corrupted beyond hope. I don’t know if many ever used it, but I guess out of spite of my NT 3.1 VM still being operational, I ought to. Just as I probably should move my blog back to a machine I own to get it fully under my control as the outsourcing thing has really been volatile this year.

Speaking of gopher, there is even a gopher app on the MS Store, aptly titled Simple Gopher Client, by Shipwreck Software.

Although it follows in the tradition of every click getting it’s own pane type client, which does make going through your history easier ,but it ends up looking a bit cluttered. I preferred the ‘web browser’ type paradigm myself, but by the time that became common place, that HTTP thing took off, seeing it was not only entirely free, but the fine folks over at Mosaic had added inline images.

Gopher kills the LC

Macintosh LC

The LC isn’t a strong Macintosh.  It is after all, a low cost model.  And what I’m doing isn’t even slightly fair to it.

Since it has a mere 68020 running at a blazing 16Mhz with no 68881 nor any MMU running something like A/UX is simply out of the question.  However MMU less Mac’s can run MachTen.

Although I did make a backup of the disk to find out that this thing had been in Harvard of all places, apparently once belonging to Mark Saroyan.

Although there was nothing even slightly academic or useful on the disk.  I wonder if the software was even pirated as the last owner sure enjoyed all the various SIM games (city/earth/life/ant) it seems more than anything else.

I formatted the massive 50MB SCSI disk, put on a fresh copy of MacOS 7.0.1 along with the network driver and MachTen 2.2.

System 7.0.1

And as far as LC’s go, this one isn’t too bad, it’s loaded up with the maximum 10MB of RAM, although it seems the VRAM is pretty sparse as it’ll only go to 16 colours.  But since we are playing UNIX here, I didn’t see any need for that, and set it to mono.

I thought it’d be fun to install a gopherd server onto this machine, and that is where the fun started.

Granted it’s been a long time since I used a machine with no real L2 cache, let alone running at a whopping 16Mhz, and using a compiler like GCC is just incredibly slow.

So I thought I could just ‘cheat’ the system by taking the source code to GCC-1.42 and tweaking the SUN3-Mach configuration into a SUN2-Mach configuration but keeping it targeting a BSD like OS, along with setting it to compile to a 68020 without a 68881.  Oddly enough getting a cross compiler wasn’t so difficult, but the assembler on the LC, a modified GAS wouldn’t assembler the files. So I went ahead and built a68 from GAS 1.38 and now I can cross assemble from Windows. However I couldn’t get the linker ld from binutils-1.9 working.  I guess it was an endian issue somewhere, but my attempt at byte swapping files it was reading just led to further confusion.  And I figured linking on the target host wouldn’t be the end of the world, as compiling sure feels like it is.

I can’t see like anyone would care, but here it is: 
MachTen-crossgcc-1.42-nolinker.7z

So fighting the source and in a matter of a 30 minutes of on/off work I had it compiled.  All I needed to do then was FTP the objects to the machine, link and run.   Surprisingly this proved to be pretty simple.

gopherd running!

I managed to get a few pages out of it, and suddenly my telnet sessions dropped.  Looking over at the console and MacOS was busy being MacOS.

error of type 3

And that was that.

I tried another program to cross compile and upload phoon!

phoon cross compiled, natively linked.

It took a while to set the clock to the right year, as my minimal System 7 install doesn’t have the time control panel, and advancing 1 year at a time from 1999 takes time, by advancing the date to New Years Eve every minute 19 times to get us to 2018 with the old date syntax:

date 12312359

Lessons learned?

Obviously if I want to do something like this, I’m going to need a better Macintosh.  Or just not do things like this….

I’m kind of on the fence as to whither 68k Unix is really all that useful in the age of Ghz x86.  

GopherVista

Continuing with our gopher madness, next up we have GopherVista.  And this is everything I had hoped it would be when I first learned about this project.  I had joked to another friend that it’d be cool to crawl and feed the indexer data in a manner that could basically bring AltaVista back to life.  And we laughed, and I had my utzoo search and that was that.

Except it wasn’t.

However, across the internet, Ben didn’t hear any naysay about limitations or anything to get in the way, and went ahead and wrote a crawler in go, kept the results in a sane name/db order for later sanitisation in and out of AltaVista, and after an aggressive gopher port scan of the internet, he created GopherVista, an index of the gopher-verse, running on Windows 98.

No, really, you read that right, GopherVista backends on Windows 98!

Read all about the creation of GopherVista over a Ben’s blog blog.benjojo.co.uk.

GopherVista

Keep in mind that this is a search engine, not a proxy, so it’s best to use something like Internet Explorer 4, or an ancient Netscape that supports both HTTP 1.1 & Gopher.

I have to also say, that something like this is far more cooler, and better thought out than my utzoo hack, and I’m just happy to have inspired him, but now I really want to re-think my setup, and of course index all the things….

Peter Godiwn’s Gopher Server

Thanks to reader Peter Godiwn, he built a .net Gopher Proxy that can read several providers via API, and to get the ball rolling, he built a WordPress connector, and yes it works great with my site!

The added bonus is that now I am automatically updating my site via gopher.  It lives as a root, so instead of taking my existing gopher offline, I added a new DNS record, and brought up:

gopher://gopherblog.superglobalmegacorp.com

Naturally that URL won’t work for 99.99% of the people out there since their browsers don’t support gopher, but the Floodgap Gopher proxy, will do the trick!

One cool feature is that it can take modern images, and convert them to GIF for older browsers.  Images even work on Netscape!

Auto converted PNG to GIF

You can find source & binaries on his github page.

Super thanks to Peter, and what a fitting 10+1 post!

Playing around with Gopher

virtuallyfun gopher

My Gopher Site, via proxy

Back in the day, if you were ‘hip’ and ‘cool’ and had a UNIX shell account back in 1993 there was this cool way of getting around various computer systems around the world called gopher.  What was really cool, is that it offered search services, indexing and even gateways into various libraries (where they kept physical books) where you could search their card catalogs for various tomes you were looking for.

Some colleges even had various services that you could connect to, offering things like news, weather and whatnot.  It was pretty neat, however there was one stumbling block, which is gopher was a VERY controlled environment, where most universities locked their client to only starting at one particular gopher server, and to get anywhere else you had to memorize an insane number of keystrokes that would make 1800 operators go crazy.  Also there was nothing like virtual hosting, so the idea of having your own gophersite was most likely out of the question.  The other issue is that the University of Minnesota, where boombox resided (the master gopherserver) saw they had something good going, and unlike UCB’s CSRG which gave BSD away for free, they were going to license the server for $100 for a educational institution, and $500 for a commercial institution.

Needless to say, this CERN thing called HTTP which they were trying to distance themselves from, which wanted no money for became the next big thing, and with the freedom and ease of setting up website, gopher became a ghost of the past.

But as the world was starting to build web clients, many understood gopher, including Microsoft’s Internet Explorer.  Although starting with version 7 (gopher was also disabled with some updates in IE6), gopher has since been removed.  But thanks to the Utilu IE Collection, and this quick registry setting you too can surf gopher space with IE 4.0 (or 5,5.5..) like it’s the mid 1990s.

My gopher site via IE 4.0

My gopher site via IE 4.0

So needless to say with a client in hand, I wanted to setup my own server.  And keeping with it being old, I decided to use the old 2.3.1 gopher server.  I also compiled it with freeWAIS support, although I haven’t quite worked out how to get that fully working right now.  Compiling this stuff on 32bit i386 Linux was trivial to say the least, but if you need binaries or anything they are here.

the next thing was to get both wais, and gopherd running from xinetd, which was easy once I knew how.  These are the service files I created:

service gopher
{
socket_type = stream
protocol = tcp
wait = no
user = gopher
server = /usr/local/etc/gopherd
server_args = -I -l /var/log/gopherd.log -u gopher /gopher-data 70
instances = 20
}

And for wais:

service wais
{
socket_type = stream
protocol = tcp
wait = no
user = root
server = /usr/local/bin/waisserver
server_args = -d /gopher-data/wais /gopher-data/wais.log -l 10 -p 210
instances = 20
}

Easy, right?

Gopher likes to be a named service, so I went with my virtuallyfun.com domain, as it is easier to type.  I guess I could have gone with gopher.superglobalmegacorp.com but that is… LONG. Another cool thing is that there are several gopher proxies out there for HTTP only people, so you can also get to my gopher page here (via gopher.floodgap.com).

The harder part was figuring out how the directory mapping works, but luckily there was enough in the test directory to get something working, changing this

ls

About decode-n-scripts install mspl tmp
bin ftp-horrors lib pids

Into this:

The old UNIX gopher client

The old UNIX gopher client

The secret is all in the .names and .Links files.  The .names file will map a directory name to something more pleasing, such as changing mspl to the “Microsoft Programmer’s Libary”.

Path=./mspl
Name=Microsoft Programmer’s Library

Easy, right?

And the .Links file creates links to various content, from a telnet example (to my bbs….)

Type=8
Name=QemuOS/2 BBS (My BBS!)
Host=bbs.superglobalmegacorp.com
Port=23
Numb=100
Abstract=My Synchronet BBS running on OS/2 inside of Qemu! #100
Path=

To another gopher system

Name=My lame SDF.org site
Type=1
Host=sdf.org
Port=70
Path=/users/jsteve
Abstract=My personal SDF gopherspace (itsucks) #-11
Numb=-11

To even doing some crude ascii art!

Name= # # # ##### ##### # # ## # # # #
Type=3
Path=
Numb=5
#
Name= # # # # # # # # # # # # # #
Type=3
Path=
Numb=6

.. and so on.

So yes, to be difficult, all the links in this post are gopher:// links. I don’t know if that’ll deter the likes of any self proclaimed virus scanner king, but I’d like to think that his poorly constructed automated tests will be unable to connect to gopher resources.

Now if I can figure out how to setup my own jughead or veronica to search my own wais of information, that’d be excellent.

If I had the virtual space I’d host the whole thing on a virtual VAX…. or something equally insane.  Word is Shoebill just got ethernet support, so running my gopher space on A/UX would be cool.