Using Qemu in 2020

So I wanted to dive back into old BSD, and I wanted to use the new native Qemu on OS X. That means having to “learn” the new crazy syntax that has drifted a LOT over the last 10+ years.

I want to run NetBSD 1.0 and tear into the GCC 2.x patches required back in the day

In the old days, I was using Qemu 0.13 and had it working using the following:

qemu.exe -L pc-bios -hda netbsd-1.0.vmdk -net nic,model=ne2k_isa -net user -redir tcp:42323::23 -m 64 -no-reboot  -rtc base=localtime -k en-us

Thankfully I was using VMDK containers, probably as I’m sure I installed it under VMware, with it’s better floppy support, but pulled it back to Qemu to have NE2000 support. Now let’s look at the dmesg to where the source of the problem is, the network card:

NetBSD 1.0 (QEMU) #0: Thu Sep 22 06:45:37 PDT 2016
    [email protected]:/usr/src/sys/arch/i386/compile/QEMU
CPU: Pentium (GenuineIntel 586-class CPU)
real mem  = 67764224
avail mem = 62312448
using 852 buffers containing 3489792 bytes of memory
pc0 at isa0 port 0x60-0x6f irq 1: color
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
lpt0 at isa0 port 0x378-0x37f irq 7
wdc0 at isa0 port 0x1f0-0x1f7 irq 14
wd0 at wdc0 drive 0: 1023MB 2080 cyl, 16 head, 63 sec <QEMU HARDDISK>
fdc0 at isa0 port 0x3f0-0x3f7 irq 6 drq 2
fd0 at fdc0 drive 1: density unknown
pci0 at isa0 port 0x0-0x665: configuration mode 1
pci0 bus 0 device 0: identifier 12378086 class 06000002 not configured
pci0 bus 0 device 1: identifier 70008086 class 06010000 not configured
pci0 bus 0 device 2: identifier 11111234 class 03000002 not configured
ed0 at isa0 port 0x320-0x33f irq 10: address 52:54:00:12:34:56, type NE2000 (16-bit)
npx0 at isa0 port 0xf0-0xff: using exception 16
biomask 4040 netmask 412 ttymask 12
changing root device to wd0a

So the important thing here is that the NE2000 is the ed0 device using an IO of 0x320 and IRQ 10. I’m pretty sure back then I cheated, and just recompiled Qemu to remove the default definition that doesn’t work properly with trying to share IRQ 9.

Keeping this in mind, let’s build this for the new Qemu:

qemu-system-i386 -net none -device ne2k_isa,iobase=0x320,irq=10,netdev=ne -netdev user,id=ne,hostfwd=tcp::42323-:23 -drive file=netbsd-1.0.vmdk,if=ide,index=0,media=disk,cache=writeback,format=vmdk -rtc base=localtime -k en-us

So the first important thing is to disable any/all defualt NIC’s with the ‘net none’ flag. Next to add in the NE2K_ISA type device manually so I can specify the iobase & irq manually. Note the netdev as this is used to tie in the emulated device, to the backend that will process the packets. The next part is the redirect to have the host listen on port 42323 and redirect them into port 23 of the VM. Again the syntax to redirect ports has drifted significantly. The hostfwd now is bound directly to the netdev user, in this new syntax it allows for multiple NIC’s to be bound to multiple user NAT’s if needed.

-device ne2k_isa,iobase=0x320,irq=10,netdev=ne -netdev user,id=ne,hostfwd=tcp::42323-:23

The next ting is the hard disk, before ‘hda’ worked fine, although like everything else that is subject to change in the future, so dealing with it now, use the drive flag:

-drive file=netbsd-1.0.vmdk,if=ide,index=0,media=disk,cache=writeback,format=vmdk

It could be my personal opinion but the drive syntax, although much longer is easier to work with. The key parts being what interface to use (IDE), and what port/index to plug the device in (0), along with the media type, the cache strategy, along with the encoded format, which is more so important when dealign with RAW devices, as unidentified media will be tagged as raw, however it’ll be mounted read-only unless it is directly specified as above.

For additional fun, the serial port can be set to a MS Mouse port, although I haven’t tested it as of yet.

-serial msmouse

Have fun QEMU’ing!

Qemu 5 was recently released

It’s been jumping numbers like crazy, and I’m still holding onto 0.9 because I’m weird. Anyways there is something amazing hiding in all those release notes and stuff:

68040!

The m68k emulation is good enough to run Linux! Granted the target machine is the Macintosh 800, although the Mac ROM doesn’t boot enough to do anything Apple enough, using a serial console however does get us into the system. On my ancient Mac Pro I get emulation in the speed range of a 1Ghz G4!

root@qemu68k:~# cat /proc/cpuinfo 
CPU: 68040
MMU: 68040
FPU: 68040
Clocking: 1304.9MHz
BogoMips: 869.99
Calibration: 4349952 loops

Absolutely amazing!

Installation is a bit tricky as there is no true bootrom / boot process, so I had to load the installer with a dummy ‘raw’ disk, and tar the kernel & initrd to that raw disk so I could later extract it on the native OS to boot into the disk. I followed mostly the instructions here.

And what is that? NeXT CUBE emulation?

The peripherals are nowhere near complete enough to boot, HOWEVER it does boot the PROM, complete with keyboard support.

qemu-system-m68k -M next-cube -bios Rev_2.5_v66.BIN

It’s fun enough to play with. And thanks to Qemu’s fast emulation, perhaps this is a speedy way to run stuff in the future?

Now isn’t that cool?

How not to store optical media

Or how I finally broke down and bought that MkLinux book after all these decades. When I did own a PowerPC Mac as my daily driver it was an iMac back in 1999 and I ran OS X Server. I later bought a G4 to only find out that OS X didn’t support the G4. Linux had issues too and I ended up running OpenBSD on the G4. Which was fun, although for the ‘work at home’ bit, I ended up needing Windows NT 4.0, so I ran that in OS 8 on SoftPC. Yay.

I didn’t have any luck with Linux on Power as MkLinux wanted the beige hardware, and by the time I felt like digging in again to Linux, OS X had finally been ported to the G4 Sawtooth’s so it really didn’t matter.

What secrets lie inside?

I’d seen this book in a store but it was pretty expensive, and geared to such a tiny market. Although Mach does compile on the i386, why they didn’t include it was well to push Mach/Linux as a platform well that’s beyond me. Then again looking at the stunning success of Darwin on i386/x86_64 I guess the reality is, why bother.

I ordered this on Ebay, for the usual $5 plus $10 to ship, and it just showed up today! What mysteries lie on the CD-ROMs? I know others have posted stuff, but I wanted to hold them in my hands myself.

Uh-oh

I didn’t know if the CD-ROM’s were included, and I first thought I got lucky: not only were they included, neither had been opened up before! These CD’s had been packed way like this for the last 22.5 years! Now for the bad part.

Ink transfer

See that stupid leaflet in the back? Yeah well it turns out that it was a really stupid idea. No doubt this thing sat in the bottom of a stack for decades where the ink had been pressed for so long against the disk that it has transferred to the surface.

HOW ANNOYING.

Gag me with a spoon…

So yes, they actually advertise the book, namely the one that I had bought with some crap ink leaflet in the CD-ROM pouch and it’s transferred to the disc.

Sigh.

I tried rubbing alcohol but that had no effect. I tried rubbing with a credit card, and it got a little off, but I fear I’m just going to damage the surface more.

I can only imagine what other CD-ROM’s out there that haven’t been archived are sitting under hundreds or thousands of pounds of book weight having nonsense imprinted onto them.

At least the second CD-ROM doesn’t suffer this defect and I’ll be uploading it later.

No book review yet, I’m just sitting here with this impacted CD.

UPDATE

Thanks to Shawn Novak for uploading the R3 images so I can at least pull up the compatible machines:

The latest Qemu can pull it the image fine, however trying to boot up looks like the Mach kernel just isn’t compatible enough with the emulated Mac99 machine (which isn’t surprising).

I’ll need to mess with stuff to see if the G3beige can boot Linux on Qemu, and if the BootX (I think it’s bootx?) can load the mach kernel.

Installing AIX on Qemu!

YES it’s real!

I’m using the Linux subystem on Windows, as it’s easier to build this Qemu tree from source. I’m using Debian, but these steps will work on other systems that use Debian as a base.

First thing first, you need to get your system with the needed pre-requisites to compile:

apt-get update;apt-get upgrade apt-get install build-essential pkg-config libz-dev libglib2.0-dev libpixman-1-dev libfdt-dev

Great with those in place, now clone Artyom Tarasenko’s source repository

git clone --branch 40p-20190406-aix-boots --single-branch https://github.com/artyom-tarasenko/qemu.git

*NOTE from the future (2022) you may want to jump here: to check out building on newer systems. Also don’t forget about networking!

Since the frame buffer apparently isn’t quite working just yet, I configure for something more like a text mode build.

././configure --target-list=ppc-softmmu --disable-sdl --disable-vnc --disable-gtk --disable-gnutls --disable-nettle --disable-gcrypt --disable-spice --disable-numa --disable-libxml2 --disable-werror

Now for me, GCC 7 didn’t build the source cleanly. I had to make a change to the file config-host.mak and remove all references to -Werror. Also I removed the sound hooks, as we won’t need them. remove the following lines:

CONFIG_AUDIO_DRIVERS=oss CONFIG_AUDIO_OSS=m

Now you can build Qemu. it’ll happily build in parallel so feel free to build using the -j parameter with how many cores you have. I have 32, so I use

make -j32

Okay, all being well you now have a Qemu. Now following the steps from
Artyom Tarasenko’s blog post, we can get started on the install!

First we create a 8GB disk

qemu-img create -f qcow2 aix-hdd.qcow2 8G

Next we need the custom BIOS with serial as the console.

wget https://github.com/artyom-tarasenko/openfirmware/releases/download/40p-20190413/q40pofw-serial.rom

You’ll need some AIX. I tried a 3.2.5 CD-ROM and it didn’t pick up, but AIX 4.3.3 did.

Now with all those bits in place, it’s time to run Qemu.

./ppc-softmmu/qemu-system-ppc -M 40p -bios q40pofw-serial.rom -serial telnet::4441,server -hda aix-hdd.qcow2 -vga none -nographic -net none -cdrom Volume_1.iso

Now telnet to your localhost on port 4441 and you will see the console doing it’s BIOS initialize and eventually drop to the OK prompt.

One trick I’ve found is that from the Open Firmware prompt you can find out what partitions are recognized from the firmware. If it see’s partitions then there is some hope that the image you have is valid enough to boot. In the last few days I’ve found quite a few AIX images, which are lacking the partition table, and unable to boot.

.partitions cdrom

simply type in boot cdrom:2 to kick off the installer. It may take a minute or so for the installer to kick off.

If all goes well, you’ll see the BIOS reload itself, then after a minute you’ll be prompted to press 1 to select the console

It doesn’t echo, don’t panic!

Next select your language. I’m doing English.

Next it’ll ask about installation type. Default ought to be fine.

Because this will destroy the contents of the disk (which doesn’t matter as it’s blank) it’ll prompt for confirmation.

After this it’ll begin the installation. Depending on how fast your disk & CPU is this will take a while.


For me, the installation took about 11 minutes. This is using my Xeon E5-2667 v2. It took 17 minutes on my 2006 Mac Pro, with X5365’s it .

After it’s done, right around the 96% time it’ll reboot back to the BIOS

Once you are back at the OK prompt, you can now boot disk:

it’ll look like it’s hung for a minute, then it’ll start booting from disk!

Once the OS is booted up, you select the terminal type. I’m using putty but I’ll select the vt100. Of note the function keys are selected by hitting escape and then the number key. So F3 is ESC 3.

I’m just going to finish the install, as we can always run smitty to mess with the system more, but right now I’m just interested in a base install of the BOS (Base Operating System, and IBM ISM).

A few moments later, you’ll get dumped to the login prompt.

By default there is no password, so just login as root, and there you go, your very own virtual AIX 4.3 system.

# uname -a AIX localhost 3 4 000000004C00

So there you go! All thanks to Artyom’s hard work!

Added UDP to Qemu0.90

Yes, I know it’s kinda pointless as it’s horribly out of date, but I’m still having fun. Anyways now it’ll integrate with dynamips!

So yeah, in this example, instead of the usual “-net user” and the built in SLiRP stack, this uses UDP to talk to a stand-alone SLiRP stack. This will let you plug your Qemu into virtual Ethernet switches/hubs and interfaces of virtual routers. Even mixing and matching connections to different emulators that use UDP to exchange packets.

In this case I have a VMDK loaded with a NE2K-PCI driver, and QuakeWorld for MS-DOS. I also made the MPU401 external although it’s hanging on a secondary flag…

qemu.exe -L pc-bios -m 64 -soundhw sb16,adlib -mpu401 xx -net nic,model=ne2k_pci -net socket,udp=127.0.0.1:4000,remote=127.0.0.1:4001 -hda qworld.vmdk

This has the Qemu instance listen on UDP port 4000, and connect to the same physical host on port 4001.

The recipical for the SLiRP stack from my bashed Dynamips is:

slirp_rdr.exe 4001 127.0.0.1 4000

Which in this case listens on port 4001, and connects back on port 4000.

Hopefully it makes some sense to someone.

Download it, if you want!

Download Qemu090b

Degrading qemu performance in DooM Part II

ticks for demo (fewer is better)

VersionASM FixedDiv/FixedMulC
0.9.0259268
0.10.5299300
0.11.0316317
0.12.0289281
0.12.5290282
0.14.0282274
0.14.1280269
20180430192195
20190218187187

So after the last round, I went ahead and dug out my crap version, where I had just recently found a nice abs() fix for a FixedDiv issue that the old iD code suffers from, and re-built a version of DooM that both used the assembly fixed division, and another with the C version. To compile I used my old GCC 2.7.2.3 to build with the flags:

-m486 -msoft-float -ffast-math -O2 -fforce-addr -fomit-frame-pointer

So here we go using the versions of Qemu that I can build quickly with GCC 3.4.5 MinGW, along with the last two pre-built Win64 builds.

It’s kind of interesting just how close the performance is between the two versions.

Naturally the real test is to run it on actual hardware, and to try a few versions of Watcom C.

Maybe the real takeaway is that Qemu runs GCC built code better…?

Adding DOSBox’s MPU401 to Qemu 0.90

I thought this may be something cool, if not kind of pointless. Anyways the MPU401 UART can be run like a traditional serial port with an IRQ, in intelligent mode, or just as a ‘dumb’ device you can just bit bang to talk to MIDI devices. So while playing with DOSBox I thought it’d be fun to take it’s emulation and plug it into Qemu.

And this is the end result.

It’s far from perfect, when it works it does tend to work well, although it fails to work with things like Return to Zork, but it does work with DMX’s sound code in DooM and the MPU401 driver for Windows 3.1

While doing this I was originally struggling with mapping the IO ports. Qemu has some functions to map in the memory model to assign a function that will trap read/write space. In this case base is 0x330 the base of the MPU401 device.

register_ioport_write(base, 8, 1, mpu401_ioport_write, s); register_ioport_read(base, 8, 1, mpu401_ioport_read, s);

I was thinking that the port 0x331 needed to be mapped in the same way, but it turns out after looking through more of the source, it’s actually a word aligned access. So in that case you can use a switch to see which port is actually being accessed.

static uint32_t mpu401_ioport_read(void *opaque, uint32_t addr) { switch(addr&0xf) { case 0: return(MPU401_ReadData(addr,0)); break; case 1: return(MPU401_ReadStatus(addr,0)); break; default: return(0xff); break; } return(0xff); }

Pretty simple, right?

And from there it’s a matter of mapping the DOSBox MPU code, along with the Windows interface code. Since I’m not using intelligent or IRQ mode, I just amputated the code where applicable.

If anyone wants to look at what I did to merge into anything else (and probably do a better job!) it’s on sourceforge as mpu401.c.

Otherwise the binary is available on sourceforge:

Download Qemu090b

Degrading qemu performance in DooM

Quick table… its late.

I’m using MS-DOS 5 & this benchmark suite loaded into a VMDK, and ran a few tests to check performance numbers.

version2 3d bench3 chris bencha doom ticksc quake demo
0.9.0182257.426346.4
0.10.5204247.649141.7
0.12.5305.1296.747546.8
0.13.0305.3284.351145.7
0.14.1289.6285.350945.2
0.15.0213.4176.952015.3
1197.9152.154615.5
2.4.0.1392.1340.674324.2
3.1.50405462.3149733.3

I snagged 3.1.50 from https://qemu.weilnetz.de/w64/

better performance than v2, sure, but for interactive stuff.. not so much.

So what is really going on here? Why is 0.90 so much faster when it comes to doom, and how is it possible that it’s the slowest in raw CPU performance. And fastest at IO? It appears that the crux of the issue is simply how it handles its IO, heavily favoring device performance VS CPU.

I’ll have to follow up with more builds and reading release notes to see what changed between releases. And what was it exactly that broke between gcc 3 and 4, and why the rip had to be.

I still like 0.90, if anything for it’s ability to run NeXTSTEP and NetWare.

QEMU Advent Calendar 2018

It’s that time of the year again!

Every day there will be a ‘tiny’ OS to run on Qemu!

The QEMU Advent Calendar 2018 features a QEMU disk image each day of December until Christmas. Each day a new package becomes available for download.

Every download contains a little ‘run’ shell script that starts the QEMU emulator with the recommended parameters for the disk image. Disk images are either contained directly in the download or are downloaded by the ‘run’ script (you need to have installed ‘curl’ or ‘wget’ in that case).

The disk images contain interesting operating systems and software that run under the QEMU emulator. Some of them are well-known or not-so-well-known operating systems, old and new, others are custom demos and neat algorithms.

The ‘run’ scripts (and disk images if included in the download) were created by volunteers from the QEMU community to showcase cool software that QEMU can run.

https://www.qemu-advent-calendar.org/2018/

Animated GIF’s from Qemu

I found this one recently… So the first thing is you need Qemu 0.10 or higher (probably not a problem), as it’ll save in ppm format no issues.  Then the fun expect program (Yay Linux subsystem), and of course Imagemagik.

Run Qemu so you can telnet to the command monitor:

i386-softmmu\qemu.exe -L pc-bios -hda c:\temp\127disk.img -monitor telnet:127.0.0.1:23,server,nowait -hdb fat:\temp\dosb

I used this small program

#!/usr/bin/expect
set timeout -60
set capture 1
spawn telnet localhost
expect “(qemu)”
send “brake 1000\r”;
expect “(qemu)”
while { 1 == 1 } {
set fstring [format %04s $capture]
send “screendump /temp/$fstring.ppm\r”;
expect “(qemu)”
incr capture
sleep 3
}

and then to convert it into an animated gif:

d:\ImageMagick-7.0.7-18-Q16>convert -loop 0 -delay 100 \temp\*.ppm \temp\GHZ.gif

and behold:

Isn’t that great?