Back in 1995 I bought this rather expensive, and ambitious book simply called: Developing Your Own 32-Bit Operating
And while it is a LONG read, it really is the embodiment of Apple pie from scratch. Rather than rely on open and available tools, the author Richard Burgess instead goes on to write his own assembler, compiler, and then onward to a simple message passing RTOS.
No doubt the price he paid for eschewing popular GNU tools, and having a non BSD/GPL license for the time is that it was quickly relegated to history as the inevitable rise of Linux took place.
For those wishing to look, not only is the source code and a few patches available on the site ipdatacorp.com, but so is a PDF of the 1st edition of the book.
While MMURTL may not have caught on in the marketplace of ideas, it’s still astounding to look at the volume of work produced, that even though open source tools and starting points were available (The book easily could have been using CMU Mach 3.0) instead it’s all written from scratch by a single person.
I didn’t realize that I never uploaded this over there. After a discussion on the passing anniversary on the TUHS mailing list I had to dig out my installed copy.
I had forgotten just how rough around the edges this was, as it’s missing quite a few utilities from the Net/2 tape, and isn’t complete enough to come up in multiuser mode, but it is capable of booting up.
Although 386BSD itself was really short lived with its effective short death in the subsequent release it paved the way for an internet only release of a BSD Unix by just 2 people. And it closed up the glaring hole of the lack of a free i386 port of Net/2.
The natural competition was Mach386, which was based around the older 4.3BSD Tahoe, and the up and coming BSDI, which had many former CSRG people which were also racing to deliver their own i386 binary / source release for sale.
One thing about this era is that you had SUN apparently forced out of the BSD business instead to work with the USL on making SYSV usable, leaving NeXT as the next big seller of BSD. The commercial world was going SYSV in a big way, and the only place that was to have a market was on the micros. And for those of us who wanted something open and free 386BSD paved the way realizing the dream of the Net/2 release. A free Unix for the common person, the true democratization of computing by letting common people use, develop and distribute it independently of any larger organization.
It’s almost a shame that GNU had stuck with the unrealized dream of a hierarchy of daemons, instead of adopting the BSD kernel with a GNU userland, on top of that tendy micro kernel Mach.
The landscape radically changed with the infamous ad proudly proclaiming “It’s UNIX”.
While USL was happy to fight both BSDI and the CSRG they never persued Bill Jolitz. And after the internet flame and lawsuit dragged on, neither of the splinter groups NetBSD or FreeBSD caught up, although both did reset upon the release of the 4.4BSD Lite 2 code.
Ever since I got my hands on the Mt Xinu disk images, I’ve been working to see if the old Mach kernels on the CSRG CD-ROM set are actually buildable and runnable. And the TL;DR is that yes, they are.
The CD has 3 Mach kernels, the MK35 kernel, a kernel that appears to be something called X147, and a release of Mach 3.0. While X147 has hardware support for the SUN-3 and most of the files for the VAX, only MK35 has hardware support for the i386. The MK35 kernel has incomplete Makefiles and other dependencies, while X147 lacks i386 support. The good news is that it’s possible to use portions of the missing config & Makefiles from X147 to fill in for MK35, as it’s possible to copy the platform code from MK35 along with the i386 specific config into X147, yielding 2 working kernels.
Now this leads to the next few issues. The hardware support appears to be code ‘donated’ from various OEMs from Intel, Olivetti, Toshiba, OSF, and the CSRG. Dates vary from 1987 to 1991.
I started with the MK35 kernel as it was smaller, and since it was tagged as an ‘Intel only release’ of Mach, I figured that this one had the best chance of actually working.
And this is as far as it got on it’s first attempted boot. The Qemu VM would immediately reboot. Since I had installed Mt Xinu on VMware I went ahead and tried it there, and it said that there was a critical CPU exception and that it was shutting down. Bochs did the same thing, as did PCem. Since nothing was being printed to the screen it must be failing in the locore.s which is split into several assembly modules. I put in a hlt at various points and kept rebuilding and rebooting to see if it would halt or if it’d reboot. Thankfully VM’s are cheap and plentiful, I can’t imagine how tedious this would be on actual hardware. Eventually I found out that right after the paging bit in CR0 was flipped the VM would reboot. Now I had something.
/ turn PG on
mov %cr0, %eax
or $PAGEBIT, %eax
mov %eax, %cr0
mov %edx, %cr3
I had tried not flipping the page bit, not flipping cr3, no matter what I tried it would triple fault and reboot.
I had to break down and beg for help, and as luck would have it, someone who knows a heck of lot more about the i386 than I could ever hope to know took a glace at the above code and immediately noted:
I looked at start.s. And it immediately jumped out at me as being very fishy. What they do is enable protected mode *and* paging, but only then load CR3. That’s something which may well work on some CPUs, but it’s against the rules. You could try just swapping the instructions around, first load CR3, then CR0. The next question is then if that code executes out of an identity-mapped page; if yes then just swapping the instructions should do the trick, if not then there is a bigger issue.
Background: Old CPUs, especially 386/486, will decode and pipeline several instructions past the protected mode switch (mov cr0, eax). The jmp instruction is there to flush that pipeline and make sure all further instructions are executed with the new addressing mode in effect. But old CPUs did not enforce that and it was possible to execute the jmp from a non-identity-mapped page, and I guess it was also possible to execute instructions between the move to cr0 and the jmp, at least most of the time. That tends to break on modern CPUs (probably P6 and later) and definitely in emulation/virtualization. The move to cr0 effectively flushes the pipeline and if the next instruction is not in the page tables, poof, there goes the OS.
Could it really be that simple?
mov %edx, %cr3
/ turn PG on
mov %cr0, %eax
or $PAGEBIT, %eax
mov %eax, %cr0
/ mov %edx, %cr3
I commented out the cr3 line and just pasted above the cr0 pagebit flip.
Amazingly the kernel booted. Behold the first boot of Mach/4.3 which very well could be the first boot independent of the CMU and I’d venture the first boot from the source on the CSRG CD-ROM set. I tried to tell Mach to use the disk as prepared by Mt Xinu, but naturally it’s incompatible.
The next thing to do was create a root diskette, which thankfully the CMU folks left the needed files in the standi386at directory. I was able to build the disk, and using VMware I could boot into single user mode. I went through the ‘unpublished’ documentation I was able to mirror, and was able to get lucky enough to have Mach prepare the hard disk, format the partitions, and I used tar to transfer the root diskette onto the hard disk. I thought it ought to be possible to boot from the boot disk, have it mount the hard disk, and re-mount the boot disk, and copy the kernel. Sounds reasonable right?
This is where the incredibly stale platform code showed it’s head once more again as the floppy driver in MK35 is amazingly useless. It seems that the emulated hardware is too fast? But all reads from the floppy using the hard disk as root failed. Instead I removed a bunch of files from the disk, and copied over gzip & a compressed copy of the kernel to disk, along with the boot.hd program, and was able to copy them to hard disk using that modified root diskette. Luckily Mach has support for a.out binaries, and this stuff being so old it’s all statically linked. My Mt Xinu build of gzip runs fine on the Mach kernel, so I could decompress the kernel and install the bootblocks.
This is where the next weird issue would happen, which is that Mach was quite insistent on mounting everything under this /RFS directory. It appears that RFS was CMU’s answer to NFS… Which needless to say didn’t ignite the world on fire. I was later able to find that I could disable the RFS code, re-configure, rebuild and re-transfer a kernel and with a bit of fighting with mount I was able to mount hd0d/hd0e. Sadly during the install process there was no visible option to specify slice sizes so I’m stuck with a 10MB root.
With this much luck in hand I thought it may be interesting to see if Mt Xinu could mount the Mach disk. Turns out that it can without any issues. So I went ahead and wiped the Mach disk, and transfered Mt Xinu over to the Mach disk, and rebooted with that. And it “works”! Although of course there is some caveats.
The first being the aforementioned floppy support is broken. The next one being that the serial support also suffers from basically losing interrupts and leaving the system waiting. The kernel debugger still works, and you can see it in the idle loop, along with the other threads waiting. This means my favorite method of using uuencode and pasting to the terminal won’t work, MK35 locks up after 35kb, and X147 made it as far as 150kb. Keep in mind that they are using the same i386/i386at platform directories.
So I’m quite sure that there is other issues hiding in the code, maybe obvious ones like the cr3/cr0 thing. On the other front I’ve been starting at looking at doing some porting of the Tahoe/Quasijarus userland with varying success. I have already started to rebuild some binaries with a substitute crt0.o as there is no source for anything included in the Mt Xinu distribution outside of the Mach 3.0 kernel.
For those who want to play along I have uploaded VMDK’s and the source tarballs.
For people using Qemu I find that a serial terminal is FAR nicer to use than the console. Also I’m unsure of how hard the 16MB ISA DMA window is being hit, but X147 seems okay with 64MB of ram, while M35 really needs to be 50MB or less..
I’ve been looking for this, since I first found out about it a few years ago. It’s a port of 4.3BSD Tahoe to the i386, utilizing the Mach kernel. This is the biggest gap of the era, which is bringing mini-computer BSD to an affordable platform, the AT386.
Sadly like many others after Mach386, it did not find widespread commercial success and MtXinu wound down operations of the product, and eventually the company itself. It’s a shame too that both Mt Xinu & BSDi eventually exited the BSD market, while the open 386 alternatives flourished and grew.
One thing is for sure, it wasn’t cheap! At least on the perpetually starving college student budget the base license was $995! And that included no source code at all. Although the Mach 3.0 Add-on does include source code, however because of the then new AT&T USL vs. BSDI/UCB lawsuit CMU got cold feet over it’s BSDSS/BNR2SS for Mach 3.0 and pulled it, leaving you with a micro kernel with no personality. Although years later the rights would flow from AT&T to Novel who then let Caldera acquire them, and then give the infamous 32v giveaway (pdf) essentially setting BSD free. Although I was one of the people who shelled out the $100 for the oldSCO SYSIII license back in the day.
Mach386 lived from around 1991 until 1993. Needless to say the Juggernaut called Linux appeared at the right time and the right place, when all of the BSD’s had faltered because of that lawsuit. Sometimes in life, timing is absolutely everything.
Anyways fast forward a few decades and I have been looking for a mythical 4.3BSD on i386 for far too long, and I came across a post on betaarchive mentioning retrosys.net, and all of Scott’s adventures with Mach386. So I was able to contact him, and get a copy of Mach386!
Well the disk set is from 1992, and going back to that era means you are going be locked into the old disk geometry where an IDE disk under 500MB is the best way to go. The floppy controller is programmed in a weird way that the only thing I could get it working with was VMWare. It wasn’t so bad going through the disks, and I quickly had a system up and running. Once the install is done it’ll run under QEMU for instance just fine.
Currently there is no ‘modern’ ish networking support, aka no NE2000. So I’ve been using serial terminals to use uuencode/uudecode to get files in & out of the VM.
So what’s in the box? Well I didn’t install the X11 stuff as I’m just not in the mood to fight it, but it’s a 4.3BSD system! Sadly adventure/zork is absent, however rogue and all the other BSD type fun is there. gcc version 1.37.1 & GNU assembler version 1.36 among others are also includes, although without any diff’s or source. Although the networking headers & tools are on separate disks, there is no nonsensical link kit type thing like Xenix, meaning that TCP/IP is fully integrated to the kernel. While there is SLIP support apparently I haven’t messed with it at all yet.
Being that this a Mach based system it builds the 3.0 kernel with ease. It even includes a 4.3BSD (sadly binary only) ported kernel to the 3.0 Mach which you can run. It’s defiantly not as fast as the default kernel, but seems to work well enough.
The kernel in question is what they term Mach 2.6 which is the 2.5 plus lots of enhancements. Among others is a different disk layout/partitioning scheme so you can dualboot. Although in the era of cheap VMs it’s kind of pointless.
So it may not look like much, but it’s a really fun thing to play around with. At the same time 386BSD had been pushed out into the world, and Linux was also a thing. It’s not surprising that Mt Xinu & BSDi would eventually fail in the marketplace, and Linux would go on to decimate the UNIX landscape. But it’s cool to run a direct VAX based OS on the PC.
text data bss dec hex
389088 45564 101364 536016 82dd0
ln vmunix.sys vmunix; ln vmunix vmunix.I386x.STD+WS-afs-nfs
However, as luck always has it, start.s in the i386 code does something weird at the 3GB mark causing a triple fault on any kind of modern emulation/virtualization setup.
/ Fix up the 1st, 3 giga and last entries in the page directory
mov $EXT(kpde), %ebx
and $MASK, %ebx
mov $EXT(kpte), %eax
and $0xffff000, %eax
or $0x1, %eax
mov %eax, (%ebx)
mov %eax, 3072(%ebx) / 3 giga -- C0000000
mov $EXT(kpde), %edx
and $MASK, %edx
Not all that sure why, but at least on Bochs, I can see the triple fault.
00036527018d[CPU0 ] page walk for address 0x0000000000101122
00036527018d[CPU0 ] page walk for address 0x00000000e0000011
00036527018d[CPU0 ] PDE: entry not present
00036527018d[CPU0 ] page fault for address 00000000e0000011 @ 0000000000101124
00036527018d[CPU0 ] exception(0x0e): error_code=0002
00036527018d[CPU0 ] interrupt(): vector = 0e, TYPE = 3, EXT = 1
00036527018d[CPU0 ] page walk for address 0x00000000c0161370
00036527018d[CPU0 ] PDE: entry not present
00036527018d[CPU0 ] page fault for address 00000000c0161370 @ 0000000000101122
00036527018d[CPU0 ] exception(0x0e): error_code=0000
00036527018d[CPU0 ] exception(0x08): error_code=0000
00036527018d[CPU0 ] interrupt(): vector = 08, TYPE = 3, EXT = 1
00036527018d[CPU0 ] page walk for address 0x00000000c0161340
00036527018d[CPU0 ] PDE: entry not present
00036527018d[CPU0 ] page fault for address 00000000c0161340 @ 0000000000101122
00036527018d[CPU0 ] exception(0x0e): error_code=0000
00036527018i[CPU0 ] CPU is in protected mode (active)
00036527018i[CPU0 ] CS.mode = 32 bit
00036527018i[CPU0 ] SS.mode = 32 bit
00036527018i[CPU0 ] EFER = 0x00000000
00036527018i[CPU0 ] | EAX=e0000011 EBX=0015f000 ECX=00161dc1 EDX=0015f000
00036527018i[CPU0 ] | ESP=0000efbc EBP=0000efbc ESI=00193fb8 EDI=00009d84
00036527018i[CPU0 ] | IOPL=0 id vip vif ac vm RF nt of df if tf SF zf af PF cf
00036527018i[CPU0 ] | SEG sltr(index|ti|rpl) base limit G D
00036527018i[CPU0 ] | CS:0028( 0005| 0| 0) 00000000 ffffffff 1 1
00036527018i[CPU0 ] | DS:0020( 0004| 0| 0) 00000000 ffffffff 1 1
00036527018i[CPU0 ] | SS:0010( 0002| 0| 0) 00001000 0000ffff 0 1
00036527018i[CPU0 ] | ES:0020( 0004| 0| 0) 00000000 ffffffff 1 1
00036527018i[CPU0 ] | FS:0000( 0005| 0| 0) 00000000 0000ffff 0 0
00036527018i[CPU0 ] | GS:0000( 0005| 0| 0) 00000000 0000ffff 0 0
00036527018i[CPU0 ] | EIP=00101122 (00101122)
00036527018i[CPU0 ] | CR0=0xe0000011 CR2=0xc0161340
00036527018i[CPU0 ] | CR3=0x00000000 CR4=0x00000000
00036527018i[CPU0 ] 0x0000000000101122>> add byte ptr ds:[eax], al : 0000
00036527018d[SIM ] searching for component 'cpu' in list 'bochs'
00036527018d[SIM ] searching for component 'reset_on_triple_fault' in list 'cpu'
00036527018e[CPU0 ] exception(): 3rd (14) exception with no resolution, shutdown status is 00h, resetting
Mach 3.0 doesn’t do this, so I’ll have to dig far deeper into start.s which is kind of really beyond me.
Building a boot disk … is involved. 😐
rm -rf /usr/src/mach25-i386/obj
/home/user/mkfs /dev/rfloppy 2880 18 2 4096 512 32 1
dd if=/usr/src/mach25-i386/obj/standi386at/boot/boot.fd of=/dev/rfd0d
/home/user/fsck -y /dev/rfloppy
mount /dev/floppy /mnt
cp /usr/src/mach25-i386/obj/STD+WS-afs-nfs/vmunix /mnt
/home/user/fsck -y /dev/rfloppy
So, I’m not all that dead. For anyone super impatient, you can download my VMDK here, which runs on Qemu & VMware, it includes a serial terminal on COM1 so you can use a real terminal, and if you are like me, uuencode/uudecode files in & out of the system. As always read the 404 page for the current username/password.
Linux supports ELF binaries for ~25 years now. a.out coredumping has bitrotten quite significantly and would need some fixing to get it into shape again but considering how even the toolchains cannot create a.out executables in its default configuration, let’s deprecate a.out support and remove it a couple of releases later, instead.
I can’t say I’m all that surprised, maintaining backwards compatibility has not really been a Linux thing, as most people are incapable of doing any troubleshooting in the myriad of hundreds to thousands of independent packages, and instead find it far easier to switch to a different distro entirely.
At the same time, the vast majority of Linux packages are available in source code, so re-building things as ELF most likely has happened in the last 25 years.
During the great ELF migration, it was a gigantic PITA having basically 2 copies of all the libraries as things were converted over, and a.out stuff quickly evaporated. For me, the beauty of a.out was for later kernels to be able to mount and run older stuff. But as we are in the era of both ‘cheap’ user mode kernels along with virtualization will the old executable format truly be missed?
Linux has survived the removal of native support for the 80386, and even the detection logic for the NexGen processors (yes they were real, and yes they did ship), so no doubt this further amputation won’t matter to the vast majority.
I have to wonder how long until the i386 32bit target is removed? Distros like Debian have long since removed support for 80486/80586 classed processors to bring the minimum up to requiring SSE-2 based instructions, and I can’t imagine anyone who is running a 32bit OS for their main OS in this day and era.
And Alan Cox points out that the a.out binary loader _could_ be done in user space if somebody wants to, but we might keep just the loader in the kernel if somebody really wants it, since the loader isn’t that big and has no really odd special cases like the core dumping does.
I know that on the surface it may not seem like much, but considering the last update was nearly 3 years ago…
Time flies, it’s flying fast, and it’s been two years till now. The latest Chinese revision, the Fifth revision of the book:”A Heavily Commemted Linux Kernel Source Code” is now HERE, and, more importantly, the English version of the book is also given HERE. After nearly one year of translation, the English version of this book has finally been completed. Of course, there must be translation errors and tipos in the book. I hope you may point them out for me. Finally, thanks Trent Jarvi who helped me a lot, thank you friends in the Linux communities, and thank you all, and Happy Chinese Spring Festival!
Yes, that’s right the book is now available in English. And in PDF form! It’s an extensive dive into the 0.12 source, complete with diagrams, notes and annotated source code.
It’s a whopper though, 1109 pages, and weighing in at 11 MB! This is not a light read!
So retrohun is doing their blog thing on github of all things, and the latest entry, is of course Xenix tales. As mentioned in comments on this blog & other places they found another driver for Xenix TCP/IP!
Going back years ago, the tiny NIC driver support for the elderly Microsoft/SCO Xenix 386 v2 included 3COMA/B/C and SLIP. However it’s been recently unearthed that D-Link had drivers for their DE-100 & DE-200 models, and as it happens the DE-200 is a NE-2000 compatible card!
That means that Qemu can install/run Xenix, and it can get onto the internet* (there is a catch, there is always a catch).
You can download the driver either from github or my password protected mirror. Simply untar the floppy under Xenix (tar -xvf /dev/fd0) and do the install via ‘mkdev dlnk’
Setting up the driver is… tedious. Much like the system itself. I found Qemu 0.90 works great, and is crazy fast (in part to GCC 3) even though Qemu 0.9’s floppy emulation isn’t good enough to install or read disks. With all the updates to Qemu 3.1 use that, it’ll read the disks, and allow for networking.
To give some idea of speed I ran the age old Dhrystone test, compiled by GCC 1.37.1 and scored the following:
Dhrystone(1.1) time for 5000000 passes = 8 This machine benchmarks at 625000 dhrystones/second
When compared to the SGI Indy’s 133Mhz R4600SC score of 194,000 @ 50000 loops that makes my Xeon W3565 322 times faster, under Qemu 0.90! And that’s under Windows!
Setting up the commandline/launching is pretty much this:
qemu.exe -L pc-bios -m 16 -net nic,model=ne2k_isa -net user -redir tcp:42323::23 -hda ..\xenix.vmdk added SLIRP adding a [GenuineIntelC] family 5 model 4 stepping 3 CPU added 16 megabytes of RAM trying to load video rom pc-bios/vgabios-cirrus.bin added parallel port 0x378 7 added NE2000(isa) 0x320 10 pci_piix3_ide_init PIIX3 IDE ide_init2  s->cylinders 203 s->heads 16 s->sectors 63 ide_init2  s->cylinders 0 s->heads 0 s->sectors 0 ide_init2  s->cylinders 2 s->heads 16 s->sectors 63 ide_init2  s->cylinders 0 s->heads 0 s->sectors 0 added PS/2 keyboard added PS/2 mouse added Floppy Controller 0x3f0 irq 6 dma 2 Bus 0, device 0, function 0: Host bridge: PCI device 8086:1237 Bus 0, device 1, function 0: ISA bridge: PCI device 8086:7000 Bus 0, device 1, function 1: IDE controller: PCI device 8086:7010 BAR4: I/O at 0xffffffff [0x000e]. Bus 0, device 1, function 3: Class 0680: PCI device 8086:7113 IRQ 0. Bus 0, device 2, function 0: VGA controller: PCI device 1013:00b8 BAR0: 32 bit memory at 0xffffffff [0x01fffffe]. BAR1: 32 bit memory at 0xffffffff [0x00000ffe].
In the file /etc/tcp the default installation does a terrible job of setting up the NIC. I changed the ifconfig line to this:
So there you go, all 20 Xenix fans out there! Not only a way to get back online, but to do it in SPEED!
Thanks to Mark for pointing out that there has been tremendous progress with version 3.1 of Qemu, and it’s TCG user speed is up to the 0.90 levels of speed (at least with dhrystone/Xenix), and it just takes a little (lot) of massaging to get up and running with Xenix with the right flags:
I was kindly sent these a while ago from an avid reader, and I tried to get them to boot up into anything useful and didn’t get anywhere. I’m sure emulators of today are probably up to task, be it Bochs/PCem/86Box or even Qemu.
As you can read right now It’s running a simple OpenWatcom 16bit hello world based program. The 16bit OS/2 and 32bit OS/2 API’s ended up having different calling sizes, among other issues which had complicated the bridge program. However Ryan’s newer use of scripts to generate the required glue for the API’s at least mean that adding the 16bit/32bit calling conventions & required bridges/glue is at least now automated.
This is super cool, as this will eventually open the door to Watcom C/Fortran, Zortec C, Microsoft Basic/C/Cobol/Fortran and of course many other languages that burst out into the initial OS/2 scene before the eventual weight of the SDK & associated costs doomed OS/2 to failure.
Seriously, for those among us who love OS/2 and have like $5 to spare, send some encouragement to Ryan… 🙂