386BSD 0.0 on sourceforge

I didn’t realize that I never uploaded this over there. After a discussion on the passing anniversary on the TUHS mailing list I had to dig out my installed copy.

I had forgotten just how rough around the edges this was, as it’s missing quite a few utilities from the Net/2 tape, and isn’t complete enough to come up in multiuser mode, but it is capable of booting up.

Although 386BSD itself was really short lived with its effective short death in the subsequent release it paved the way for an internet only release of a BSD Unix by just 2 people. And it closed up the glaring hole of the lack of a free i386 port of Net/2.

The natural competition was Mach386, which was based around the older 4.3BSD Tahoe, and the up and coming BSDI, which had many former CSRG people which were also racing to deliver their own i386 binary / source release for sale.

One thing about this era is that you had SUN apparently forced out of the BSD business instead to work with the USL on making SYSV usable, leaving NeXT as the next big seller of BSD. The commercial world was going SYSV in a big way, and the only place that was to have a market was on the micros. And for those of us who wanted something open and free 386BSD paved the way realizing the dream of the Net/2 release. A free Unix for the common person, the true democratization of computing by letting common people use, develop and distribute it independently of any larger organization.

It’s almost a shame that GNU had stuck with the unrealized dream of a hierarchy of daemons, instead of adopting the BSD kernel with a GNU userland, on top of that tendy micro kernel Mach.

The landscape radically changed with the infamous ad proudly proclaiming “It’s UNIX”.

While USL was happy to fight both BSDI and the CSRG they never persued Bill Jolitz. And after the internet flame and lawsuit dragged on, neither of the splinter groups NetBSD or FreeBSD caught up, although both did reset upon the release of the 4.4BSD Lite 2 code.

I zipped up Bochs along with the disk here: 386BSD-0.0-with-bochs.7z

Mach ’86 on the SIMH VAX

Kernel booting from 1986

While the kernel may boot on SIMH there is certainly something going on with the SIMH emulation of the hardware that threw me for a few loops. I had a pre-installed version of 4.3BSD which was on a RA81 disk but shortly after loading the kernel various binaries wouldn’t load, filesystems wouldn’t mount and of course the inevitable file corruption.

This led me to the fun of loading up 4.3 onto RP06 disks as they are smaller and I was hoping less prone to errors. During this fun, I found this page on plover.net, which as a fun filled tangent shows how to use the Quasijarus console floppy image to run the standalone programs. With the latest version of SIMH, I can run format and it initializes the disks, so I almost think it may be possible to do some kind of ‘native install’ on the VAX-11/780 SIMH, although It’s not what I was in the mood for.

So finally with an install over several RP06 disks, I was lucky enough to figure out how to build the Mach’86 kernel, and get it to boot, and then the corruption happened again. Luckily for me I had snapshotted the disks before experimenting and noticed how even those had issues booting up. It’s after a bit of searching I found that other people may have issues with SIMH’s Async I/O code, and the best thing to do, is just to disable it with a “set noasync”.

Now my disks could boot under the Mach kernel, and I could self host!

self hosted!

Setting up the build involved copying files from the ‘cs’ directory to their respective homes, along with the ‘mach/bin/m*’ commands to the /bin directory. Configuring the kernel is very much like a standard BSD kernel config, however the directory needs to exist beforehand, and instead of the in path config command run the config command in the local directory.

While maybe not perfect, keep in mind I haven’t found any real instructions as of yet, so this is more of a ‘wow it booted’ kind of thing at the moment.

While this kernel does have mentions of multi processor support I haven’t quite figured out what models (if any) are supported On the VAX, and if SIMH emulates them. While oboguev.net has a very interesting looking multiprocessor VAX emulation, VAX_MP it’s a fictional model based on the microvax, which I’m pretty sure 4.3BSD/Mach’86 is far too old for.

And for those who want to play along, here is a zip file of the disk images, emulator and config file I’m using, Mach86.zip

WRP 4.0 Preview

(This is a guest post from Antoni Sawicki aka Tenox)

Welcome a completely new and absolutely insane mode of Web Rendering Proxy. ISMAP on steroids!

While v3.0 was largely just a port from Python/Webkit to GoLang/Chromedp, the new version is a whole new game. Previously WRP worked by walking the DOM and making a clickable imagemap out of <A HREF> nodes. Version 4.0 works by using x,y coordinates obtained from ISMAP to perform a simulated mouse click in Chrome browser. This way you can click on any element of the page. From annoying cookie warnings, to various drop down menus and even play some online games. Also pagination has been replaced with a clickable scroll bar.

Enough talking, you can watch this video:

Or download the new version and try it yourself!

Please report bugs on github.com. Thank you!

Mach 2.5 Independence day

With apologies to a very 90’s movie

Ever since I got my hands on the Mt Xinu disk images, I’ve been working to see if the old Mach kernels on the CSRG CD-ROM set are actually buildable and runnable. And the TL;DR is that yes, they are.

The CD has 3 Mach kernels, the MK35 kernel, a kernel that appears to be something called X147, and a release of Mach 3.0. While X147 has hardware support for the SUN-3 and most of the files for the VAX, only MK35 has hardware support for the i386. The MK35 kernel has incomplete Makefiles and other dependencies, while X147 lacks i386 support. The good news is that it’s possible to use portions of the missing config & Makefiles from X147 to fill in for MK35, as it’s possible to copy the platform code from MK35 along with the i386 specific config into X147, yielding 2 working kernels.

Now this leads to the next few issues. The hardware support appears to be code ‘donated’ from various OEMs from Intel, Olivetti, Toshiba, OSF, and the CSRG. Dates vary from 1987 to 1991.

I started with the MK35 kernel as it was smaller, and since it was tagged as an ‘Intel only release’ of Mach, I figured that this one had the best chance of actually working.

boot:
442336+46792+115216[+38940+39072]

And this is as far as it got on it’s first attempted boot. The Qemu VM would immediately reboot. Since I had installed Mt Xinu on VMware I went ahead and tried it there, and it said that there was a critical CPU exception and that it was shutting down. Bochs did the same thing, as did PCem. Since nothing was being printed to the screen it must be failing in the locore.s which is split into several assembly modules. I put in a hlt at various points and kept rebuilding and rebooting to see if it would halt or if it’d reboot. Thankfully VM’s are cheap and plentiful, I can’t imagine how tedious this would be on actual hardware. Eventually I found out that right after the paging bit in CR0 was flipped the VM would reboot. Now I had something.

        / turn PG on
        mov     %cr0, %eax
        or      $PAGEBIT, %eax
        mov     %eax, %cr0

        mov   %edx, %cr3 

I had tried not flipping the page bit, not flipping cr3, no matter what I tried it would triple fault and reboot.

I had to break down and beg for help, and as luck would have it, someone who knows a heck of lot more about the i386 than I could ever hope to know took a glace at the above code and immediately noted:

I looked at start.s. And it immediately jumped out at me as being very fishy. What they do is enable protected mode *and* paging, but only then load CR3. That’s something which may well work on some CPUs, but it’s against the rules. You could try just swapping the instructions around, first load CR3, then CR0. The next question is then if that code executes out of an identity-mapped page; if yes then just swapping the instructions should do the trick, if not then there is a bigger issue.


 Background: Old CPUs, especially 386/486, will decode and pipeline several instructions past the protected mode switch (mov cr0, eax). The jmp instruction is there to flush that pipeline and make sure all further instructions are executed with the new addressing mode in effect. But old CPUs did not enforce that and it was possible to execute the jmp from a non-identity-mapped page, and I guess it was also possible to execute instructions between the move to cr0 and the jmp, at least most of the time. That tends to break on modern CPUs (probably P6 and later) and definitely in emulation/virtualization. The move to cr0 effectively flushes the pipeline and if the next instruction is not in the page tables, poof, there goes the OS.

Could it really be that simple?

        mov     %edx, %cr3

        / turn PG on
        mov     %cr0, %eax
        or      $PAGEBIT, %eax
        mov     %eax, %cr0

        / mov   %edx, %cr3 

I commented out the cr3 line and just pasted above the cr0 pagebit flip.

The first boot of Mach 2.5 MK35

Amazingly the kernel booted. Behold the first boot of Mach/4.3 which very well could be the first boot independent of the CMU and I’d venture the first boot from the source on the CSRG CD-ROM set. I tried to tell Mach to use the disk as prepared by Mt Xinu, but naturally it’s incompatible.

The next thing to do was create a root diskette, which thankfully the CMU folks left the needed files in the standi386at directory. I was able to build the disk, and using VMware I could boot into single user mode. I went through the ‘unpublished’ documentation I was able to mirror, and was able to get lucky enough to have Mach prepare the hard disk, format the partitions, and I used tar to transfer the root diskette onto the hard disk. I thought it ought to be possible to boot from the boot disk, have it mount the hard disk, and re-mount the boot disk, and copy the kernel. Sounds reasonable right?

This is where the incredibly stale platform code showed it’s head once more again as the floppy driver in MK35 is amazingly useless. It seems that the emulated hardware is too fast? But all reads from the floppy using the hard disk as root failed. Instead I removed a bunch of files from the disk, and copied over gzip & a compressed copy of the kernel to disk, along with the boot.hd program, and was able to copy them to hard disk using that modified root diskette. Luckily Mach has support for a.out binaries, and this stuff being so old it’s all statically linked. My Mt Xinu build of gzip runs fine on the Mach kernel, so I could decompress the kernel and install the bootblocks.

This is where the next weird issue would happen, which is that Mach was quite insistent on mounting everything under this /RFS directory. It appears that RFS was CMU’s answer to NFS… Which needless to say didn’t ignite the world on fire. I was later able to find that I could disable the RFS code, re-configure, rebuild and re-transfer a kernel and with a bit of fighting with mount I was able to mount hd0d/hd0e. Sadly during the install process there was no visible option to specify slice sizes so I’m stuck with a 10MB root.

With this much luck in hand I thought it may be interesting to see if Mt Xinu could mount the Mach disk. Turns out that it can without any issues. So I went ahead and wiped the Mach disk, and transfered Mt Xinu over to the Mach disk, and rebooted with that. And it “works”! Although of course there is some caveats.

The first being the aforementioned floppy support is broken. The next one being that the serial support also suffers from basically losing interrupts and leaving the system waiting. The kernel debugger still works, and you can see it in the idle loop, along with the other threads waiting. This means my favorite method of using uuencode and pasting to the terminal won’t work, MK35 locks up after 35kb, and X147 made it as far as 150kb. Keep in mind that they are using the same i386/i386at platform directories.

So I’m quite sure that there is other issues hiding in the code, maybe obvious ones like the cr3/cr0 thing. On the other front I’ve been starting at looking at doing some porting of the Tahoe/Quasijarus userland with varying success. I have already started to rebuild some binaries with a substitute crt0.o as there is no source for anything included in the Mt Xinu distribution outside of the Mach 3.0 kernel.

For those who want to play along I have uploaded VMDK’s and the source tarballs.

For people using Qemu I find that a serial terminal is FAR nicer to use than the console. Also I’m unsure of how hard the 16MB ISA DMA window is being hit, but X147 seems okay with 64MB of ram, while M35 really needs to be 50MB or less..

qemu.exe -L pc-bios -hda Mach25-MK35.vmdk -serial telnet:127.0.0.1:42323,server,nowait -m 16

This puts the serial port into TCP server mode, so you can simply telnet into the serial port. As always change the memory are your own discretion.

Mach386: MACH and BERKELEY UNIX for i386

I’ve been looking for this, since I first found out about it a few years ago. It’s a port of 4.3BSD Tahoe to the i386, utilizing the Mach kernel. This is the biggest gap of the era, which is bringing mini-computer BSD to an affordable platform, the AT386.

Sadly like many others after Mach386, it did not find widespread commercial success and MtXinu wound down operations of the product, and eventually the company itself. It’s a shame too that both Mt Xinu & BSDi eventually exited the BSD market, while the open 386 alternatives flourished and grew.

One thing is for sure, it wasn’t cheap! At least on the perpetually starving college student budget the base license was $995! And that included no source code at all. Although the Mach 3.0 Add-on does include source code, however because of the then new AT&T USL vs. BSDI/UCB lawsuit CMU got cold feet over it’s BSDSS/BNR2SS for Mach 3.0 and pulled it, leaving you with a micro kernel with no personality. Although years later the rights would flow from AT&T to Novel who then let Caldera acquire them, and then give the infamous 32v giveaway (pdf) essentially setting BSD free. Although I was one of the people who shelled out the $100 for the oldSCO SYSIII license back in the day.

Mach386 lived from around 1991 until 1993. Needless to say the Juggernaut called Linux appeared at the right time and the right place, when all of the BSD’s had faltered because of that lawsuit. Sometimes in life, timing is absolutely everything.

Sadly by the time I could afford expensive OS’s it was 1999 and I’d bought OS X Server 1.0, with all it’s 4.4BSD + Mach 2.5 fun.

Anyways fast forward a few decades and I have been looking for a mythical 4.3BSD on i386 for far too long, and I came across a post on betaarchive mentioning retrosys.net, and all of Scott’s adventures with Mach386. So I was able to contact him, and get a copy of Mach386!

Installation:

Well the disk set is from 1992, and going back to that era means you are going be locked into the old disk geometry where an IDE disk under 500MB is the best way to go. The floppy controller is programmed in a weird way that the only thing I could get it working with was VMWare. It wasn’t so bad going through the disks, and I quickly had a system up and running. Once the install is done it’ll run under QEMU for instance just fine.

Mach386 on Qemu

Currently there is no ‘modern’ ish networking support, aka no NE2000. So I’ve been using serial terminals to use uuencode/uudecode to get files in & out of the VM.

So what’s in the box? Well I didn’t install the X11 stuff as I’m just not in the mood to fight it, but it’s a 4.3BSD system! Sadly adventure/zork is absent, however rogue and all the other BSD type fun is there. gcc version 1.37.1 & GNU assembler version 1.36 among others are also includes, although without any diff’s or source. Although the networking headers & tools are on separate disks, there is no nonsensical link kit type thing like Xenix, meaning that TCP/IP is fully integrated to the kernel. While there is SLIP support apparently I haven’t messed with it at all yet.

What is really interesting is the other disk set, MtXinu-Mach386-M3921131020-Mach_3.0+DUI.7z which is that Mach 3.0 kernel version MK78 which I believe is the first widespread & public release of Mach 3.0.

Being that this a Mach based system it builds the 3.0 kernel with ease. It even includes a 4.3BSD (sadly binary only) ported kernel to the 3.0 Mach which you can run. It’s defiantly not as fast as the default kernel, but seems to work well enough.

The kernel in question is what they term Mach 2.6 which is the 2.5 plus lots of enhancements. Among others is a different disk layout/partitioning scheme so you can dualboot. Although in the era of cheap VMs it’s kind of pointless.

So it may not look like much, but it’s a really fun thing to play around with. At the same time 386BSD had been pushed out into the world, and Linux was also a thing. It’s not surprising that Mt Xinu & BSDi would eventually fail in the marketplace, and Linux would go on to decimate the UNIX landscape. But it’s cool to run a direct VAX based OS on the PC.

I’m not dead

It’s just been really busy with this move, unpacking and the usual losing things, finding things and breaking things.

In the middle of it all, I found something online, that I want to at least do some proper article thing about… As it’s been really exciting, and goes back to the first month I started this blog.

Outside of NeXTSTEP, the other i386 commercial version of Mach 2.5 surfaced, the Mt Xinu version!

Even better, a few years ago, I had stumbled onto the source code for 2.5 buried on the 4th disc of the CSRG set, and with a LOT of luck and persistence I can confirm that the sources are complete enough to build.

loading vmunix.sys
rearranging symbols
text	data	bss	dec	hex
389088	45564	101364	536016	82dd0
ln vmunix.sys vmunix; ln vmunix vmunix.I386x.STD+WS-afs-nfs

However, as luck always has it, start.s in the i386 code does something weird at the 3GB mark causing a triple fault on any kind of modern emulation/virtualization setup.

    / Fix up the 1st, 3 giga and last entries in the page directory
    mov     $EXT(kpde), %ebx
    and     $MASK, %ebx

    mov     $EXT(kpte), %eax
    and     $0xffff000, %eax
    or      $0x1, %eax

    mov     %eax, (%ebx)
    mov     %eax, 3072(%ebx)        / 3 giga -- C0000000

    mov     $EXT(kpde), %edx
    and     $MASK, %edx

Not all that sure why, but at least on Bochs, I can see the triple fault.

00036527018d[CPU0  ] page walk for address 0x0000000000101122
00036527018d[CPU0  ] page walk for address 0x00000000e0000011
00036527018d[CPU0  ] PDE: entry not present
00036527018d[CPU0  ] page fault for address 00000000e0000011 @ 0000000000101124
00036527018d[CPU0  ] exception(0x0e): error_code=0002
00036527018d[CPU0  ] interrupt(): vector = 0e, TYPE = 3, EXT = 1
00036527018d[CPU0  ] page walk for address 0x00000000c0161370
00036527018d[CPU0  ] PDE: entry not present
00036527018d[CPU0  ] page fault for address 00000000c0161370 @ 0000000000101122
00036527018d[CPU0  ] exception(0x0e): error_code=0000
00036527018d[CPU0  ] exception(0x08): error_code=0000
00036527018d[CPU0  ] interrupt(): vector = 08, TYPE = 3, EXT = 1
00036527018d[CPU0  ] page walk for address 0x00000000c0161340
00036527018d[CPU0  ] PDE: entry not present
00036527018d[CPU0  ] page fault for address 00000000c0161340 @ 0000000000101122
00036527018d[CPU0  ] exception(0x0e): error_code=0000
00036527018i[CPU0  ] CPU is in protected mode (active)
00036527018i[CPU0  ] CS.mode = 32 bit
00036527018i[CPU0  ] SS.mode = 32 bit
00036527018i[CPU0  ] EFER   = 0x00000000
00036527018i[CPU0  ] | EAX=e0000011  EBX=0015f000  ECX=00161dc1  EDX=0015f000
00036527018i[CPU0  ] | ESP=0000efbc  EBP=0000efbc  ESI=00193fb8  EDI=00009d84
00036527018i[CPU0  ] | IOPL=0 id vip vif ac vm RF nt of df if tf SF zf af PF cf
00036527018i[CPU0  ] | SEG sltr(index|ti|rpl)     base    limit G D
00036527018i[CPU0  ] |  CS:0028( 0005| 0|  0) 00000000 ffffffff 1 1
00036527018i[CPU0  ] |  DS:0020( 0004| 0|  0) 00000000 ffffffff 1 1
00036527018i[CPU0  ] |  SS:0010( 0002| 0|  0) 00001000 0000ffff 0 1
00036527018i[CPU0  ] |  ES:0020( 0004| 0|  0) 00000000 ffffffff 1 1
00036527018i[CPU0  ] |  FS:0000( 0005| 0|  0) 00000000 0000ffff 0 0
00036527018i[CPU0  ] |  GS:0000( 0005| 0|  0) 00000000 0000ffff 0 0
00036527018i[CPU0  ] | EIP=00101122 (00101122)
00036527018i[CPU0  ] | CR0=0xe0000011 CR2=0xc0161340
00036527018i[CPU0  ] | CR3=0x00000000 CR4=0x00000000
00036527018i[CPU0  ] 0x0000000000101122>> add byte ptr ds:[eax], al : 0000
00036527018d[SIM   ] searching for component 'cpu' in list 'bochs'
00036527018d[SIM   ] searching for component 'reset_on_triple_fault' in list 'cpu'
00036527018e[CPU0  ] exception(): 3rd (14) exception with no resolution, shutdown status is 00h, resetting

Mach 3.0 doesn’t do this, so I’ll have to dig far deeper into start.s which is kind of really beyond me.

Building a boot disk … is involved. 😐

rm -rf /usr/src/mach25-i386/obj
mkdir /usr/src/mach25-i386/obj
cd /usr/src/mach25-i386/standi386at/boot
make fdboot
/home/user/mkfs /dev/rfloppy 2880 18 2 4096 512 32 1
dd if=/usr/src/mach25-i386/obj/standi386at/boot/boot.fd of=/dev/rfd0d
/home/user/fsck -y /dev/rfloppy
cd /usr/src/mach25-i386/
make
mount /dev/floppy /mnt
cp /usr/src/mach25-i386/obj/STD+WS-afs-nfs/vmunix /mnt
sync
umount /mnt
/home/user/fsck -y /dev/rfloppy

So, I’m not all that dead. For anyone super impatient, you can download my VMDK here, which runs on Qemu & VMware, it includes a serial terminal on COM1 so you can use a real terminal, and if you are like me, uuencode/uudecode files in & out of the system. As always read the 404 page for the current username/password.

Moving offices again

Things are going well, and I’ve outgrown the old place. So time to move.

I’m super lucky, there is no denying that. So to push my luck I’m giving myself the corner office.

Unfortunately the prior tenant believes that masking tape X’s in the windows makes them stronger and will prevent them from breaking during a typhoon. I cannot believe how many people try to tell me that paper tape is somehow going to catch shards of glass being propelled upwards of 200km/hr.

It’s all exciting to me as the success is not only with my company, but its not an IT company either. It’s such an interesting thing being thrown into a different field although many of the challenges oddly enough remain the same.

Anyway all the hosted stuff is obviously offline. I think I’m getting a different public address, to further complicate things.

So yes, I’ve been busy

WRP 3.0 Beta ready for testing

(This is a guest post from Antoni Sawicki aka Tenox)

I have released WRP 3.0 for testing. It’s currently a browser-in-browser server rather than a true proxy, but that’s in the works. Please try it out and let me know. Usage instructions are on the main github project page.

Today using trickery I was able to login to my reddit account from Mosaic:

Update: just added the missing image quantizer so that the color number input box actually does something useful. Now you can browse porn even with 16 colors:

Apple releases the Cheese grater MK II!

As someone who’s owned a few G5’s over the years, and 2 intel ‘cheese grater/Mac Pros’ this is like exciting news! Although I don’t see why this machine took YEARS to churn out after the trashcan fire, but here we go!

Mmmm Cheese

Somehow the aesthetic is even more cheese grater than the prior G5/Pro’s. Almost a desperate call back to pros saying you missed the grater, so here it is again! Now with more grating action, and like the iPhone now with rounded corners!

Single Processor!

One thing I’ve heard time and time again is that XNU really struggles with multiprocessor setups. And I guess we’ve hit that peak as that 2013 Mac Pro was single processor, and the new Mac Pro continues in this trend with a single processor, a Xeon from the ‘W’ or workstation lineup. Which I guess isn’t all that surprising.

The real great thing is expandibility is back! SLOTS SLOTS SLOTS! Although there is no front 3.5mm RCA audio (lol remember that?!) there is 2 USB-C on the top of the case for somewhat accessible ports. Still not too bad.

Another quick to open and upgrade machine. Just like the good old days of the cheese grater!

Although many were hoping for an end to the NVIDIA embargo, bringing CUDA to the table, there was no such luck. Instead the whole ‘dual GPU’ thing was doubled down on.

Radeon Pro Vega II Duo

Bundled is the Radeon Pro Vega II Duo card, featuring dual GPU’s on the same card. Although the case is now large enough for two of these cards giving you 4 GPUs in the box.

So far, so good right?!

And then there is the expected MSRP. $5,999 USD. For the cheapest ‘base’ model featuring a bare 8 core, 16 threaded processor clocking in at 3.5Ghz.

However this does mean for people who want to collect old Mac stuff, the trashcans are no doubt going to crash in price. If you enjoy having a stack of external peripherals, and wires and cables everywhere. Kind of like the old Power Mac G4 Cube.