Compiling rsync for VMWare ESXi 6.5

So what had started this little ‘adventure’ is that years ago there was this great sale over at Joe’s Datacenter, where I had picked up a physial server for the mere price of $20 USD a month!  What a deal!  No more quotas, CPU sharing issues and generally having to share.  Awesome!

So I have them install Debian, and load up the KVM modules, and away I go and life is good.  So foolishly years later, I jumped onto the whole container thing, installed docker, and that is where everything went south.

Every few seconds while doing a tcpdump on the 100% virtual bridge I’d see a massive influx of arp traffic.  I tried static arp’s on the host & the guest and it was ‘better’ but now the network traffic would hang.  Things like TCP would break after a minute and stuff like UDP game state would break so bad it’d end up unplayable.  This basically broke maraakate.org and hosting stuff like Quake I/ Quake II/ Daikatana and other iD based games.

My existing virtual machines now had a major issue where they would stop responding to traffic.  I never could find a fix, and it ended up with me moving my blog out to sloppy.io to keep running as a container based service until it magically stopped working and I gave up and did a full dump/reload on a hosted WordPress over at ChicagoVPS.  What a nightmare!

But what about all those virtual machines?

Well even after apt-get purge of everything docker, upgrading and downgrading the kernel nothing helped.  The VMs still dropped off the network periodically.  So with some spare time I decided to just go ahead and backup the box, and wipe the machine.

Since the physical network was working fine I was able to rsync the 300GB+ worth of data over the span of a few days.  That was fine, and considering how crap the server had been, I figured some straight downtime wouldn’t hurt anything too much.  While looking for an OS to install onto the machine, I saw that Joe offers VMWare ESXi 6.5, so I thought I’d just go with that, as naturally VMWare runs both VMs and with Project Photon I could maybe mess with containers again at a later date.

Since rsync had worked so well for moving hundreds of gigabytes of data from the USA to Hong Kong, I figured it’d be trivial to just convert the existing RAW KVM/Qemu disk images back to the United States of America.  And that is where the fun begins.

Let me tell you!

While reading this great post on virten.net they drop mention of XSIBackup, which requires registration (yuck) and worse their stupid registration system is broken:

LOL WHUT?

Rest assured the email does show up!

Dear Neo Zeed, thank you for visiting 33hops.com
This is your download url http://a.33hops.com/downloads/?key=bq7l5ptPB70MJj9dkftxFegr3xWoBZwpdFPQOUC3Cm10KPSIl3v1532877224253

But of course it doesn’t work

The key is invalid or has expired, only two downloads are allowed per key, download a new one at 33hops.com

What

The

Fuck

I know this is an ongoing issue at large when you provide executable binaries on the internet, as you will no doubt get flagged with some false positive by some virus crusaiding idiot who just sets up a tool and never reads anything but sends out threatening emails.  I went through this with the need for the simple 404 redirect, all thanks to Gerhard W. Recher.

So since this wasn’t going to be an avenue to persue I dug a little deeper and ran across this post over at virtuallyGhetto.  So it turns out the userland for ESXi is a CentOS environment that uses busybox.  And if you just download and install CentOS 3.9 into a VM, and build whatever you want, and ideally add in the -static flag, and copy it over.  For those who want to look into more ‘internals’ of the userland, check out zemris.fer.hr.

Great!

Things like UID/GUID mappings are broken in the libc it seems among other things.  So for my simple rsync config I just put the numbers in myself.

uid = 0
gid = 0
use chroot = no
max connections = 4
syslog facility = local5
pid file = /var/run/rsyncd.pid
hosts allow = a.b.c.d

[datastore1]
path = /vmfs/volumes/datastore1
comment = WDC_WD5000AAJS2D00A8B0
read only = no

I have read that you really ought to keep your binaries/config on the datastore so they aren’t subject to upgrades overwriting them and other chaotic stuff.  So editing the file “/etc/rc.local.d/local.sh” I just added the following before the exit 0:

/vmfs/volumes/datastore1/tools/rsync –daemon –log-file=/tmp/rsync.log –config=/vmfs/volumes/datastore1/tools/rsyncd.conf

And then ran it manually to kick it off.

So now I don’t have to rely on someone’s elses broken downoad system, and now we can build other fun native stuff.

And the best part is that after all of this fighting Maraakate’s site is back online and I get this message from him:

holy crap that new server setup so much better
its like playing it locally honest to god
played a whole unit not a single fuck up
no rubber banding lag effect or any of that

So now things are actually performing better on VMWare than we were getting on KVM.  And yes I had flattened out the disk images, loaded up the paravirtual disk & network drives on KVM, but VMWare does such a surprisingly better job.

I honestly wasn’t expecting that.

And as a bonus, I messed with qemu-0.9.0 (I didn’t feel the need to go through glibc2 hell), and qemu-img works great with a simple raw to vmdk

[[email protected]:/vmfs/volumes/5b5806fd-339444da-f897-003048d70598] ./tools/qemu-i
mg.static info win30.raw
image: win30.raw
file format: raw
virtual size: 32M (33554432 bytes)
disk size: 32M
[[email protected]:/vmfs/volumes/5b5806fd-339444da-f897-003048d70598] ./tools/qemu-i
mg.static convert -f raw -O vmdk win30.raw win30.vmdk
[[email protected]:/vmfs/volumes/5b5806fd-339444da-f897-003048d70598] ./tools/qemu-i
mg.static info win30.vmdk
image: win30.vmdk
file format: vmdk
virtual size: 32M (33554432 bytes)
disk size: 27M

And it boots!

Transcopied Windows 3.0 VM

So yes, wrapping up you can in fact run stuff on ESXi, copy data, and even convert disk images.

Oh yeah, and so people can deal with my 404 based download system (the password is on the 404 page)

Let the games begin!

Converting a Physical disk to a Virtual disk with Qemu’s qemu-img on Windows

Just because even I forget from time to time.

You need to do this as administrator, even better if the disk doesn’t have a drive letter or mounted in any way under Windows.

Fujitsu MPB3021AT

In my case I picked up a 486SX with an aging Fujitsu disk.

qemu-img.exe convert -f raw -O qcow2 \\.\PhysicalDrive2 fujitsu_MPB3021AT.qcow2

And as fast as your machine can read the disk, you’ll have your Qcow2 disk image. As of Qemu 2.9.0 the formats include:

  • blkdebug
  • blkreplay
  • blkverify
  • bochs
  • cloop
  • dmg
  • luks
  • nbd
  • null-aio
  • null-co
  • parallels
  • qcow
  • qcow2
  • qed
  • quorum
  • raw
  • replication
  • sheepdog
  • vdi
  • vhdx
  • vmdk
  • vpc
  • vvfat

Which is quite a list.  Obviously since I’m reading a physical disk, the format is RAW.  I just output it to Qemu for my personal ease.

Also once the image was created I could quickly run it under Qemu, and discover that yes this was a machine running Windows 95.

qemu-system-i386.exe -hda fujitsu_MPB3021AT.qcow2 -soundhw es1370 -vga cirrus

So there you go from a “dead system” to at least fully recovered data in minutes.  KVM may get all the pres excited but it’s nothing without the awesome support of Qemu!

Reading .toast image files

Well I put out a cry for help all over the place, looking for Darwin 0.3

And much to my amazement, when I woke up, I not only got a reply but a link to a toast image.  Great, what is toast?  Well simply put toast is a format made popular by then Adaptec Toast.  Obviously the sane thing to do is to find Toast, install it, and mount the disk image inside of a Macintosh.

Adaptec toast 4.0

But, honestly, where is the fun in that?

Instead let’s have Cockatrice III do it!  Now I never did get around to writing proper CD-ROM emulation, nor integrating it, but that doesn’t matter!  Instead I’m going to rely on Daemon tools Lite, to do all the heavy lifting.  DTL will create a virtual SCSI adapter, add in a SCSI CD-ROM device, and mount the image.  Needless to say, I’m on Windows and that is where that part of the adventure ends, as Windows 10 cannot read HFS.

Now back to Cockatrice!

All I had to do was assign the SCSI 6 position to the mounted drive letter, and I’m set!  Just add this to the CockatriceIII_Prefs file:

scsi6 \\.\e:

And now I can mount the image from within Cockatrice III

Darwin 0.3 toast mounted

And there we go, now I can copy the files of just like having a real Mac.

 

Adding virtual disks to User Mode Linux

Running out of disk space

Well my good ‘friend’ with their inappropriately provisioned Linux VPS  that runs UML (User Mode Linux) inside of it, ran into an issue where he needed to add a second virtual disk device.

Creating the disk file is no big issue, adding a whopping 1GB is pretty simple!

Using the ‘dd’ command it is trivial to make a 1GB file like this:

dd if=/dev/zero of=node1_swap.ubda bs=1M count=1024

And then just append it to the script that they are using to run the UML:

/virtual/kernel ubda=/virtual/node1.ubda mem=384M eth0=slirp,,/virtual/sl1.sh

to this:

/virtual/kernel ubda=/virtual/node1.ubda ubdb=/virtual/node1_swap.ubda mem=384M eth0=slirp,,/virtual/sl1.sh

Of course the real fun comes from trying to find the devices.  Having to dig around I found that the device major is 98 for the UBD’s and that they incrament by 16, so that the first 3 devices are as follows:

mknod /dev/ubda b 98 0
mknod /dev/ubdb b 98 16
mknod /dev/ubdc b 98 32

Adding to that, you can partition them, and then they break out like this:

mknod /dev/ubda1 b 98 1
mknod /dev/ubda2 b 98 2
mknod /dev/ubda3 b 98 3
mknod /dev/ubdb1 b 98 17
mknod /dev/ubdb2 b 98 18

You get the idea.

With the disk added you can partition the ubd like a normal disk

node1:~# fdisk /dev/ubdb

Command (m for help): p

Disk /dev/ubdb: 1073 MB, 1073741824 bytes
128 heads, 32 sectors/track, 512 cylinders
Units = cylinders of 4096 * 512 = 2097152 bytes

Device Boot Start End Blocks Id System
/dev/ubdb1 1 245 501744 83 Linux
/dev/ubdb2 246 512 546816 82 Linux swap / Solaris

etc etc.  And yes, you can then format, mount and all that.

First let’s setup the swap:

mkswap /dev/ubdb2
swapon /dev/ubdb2

Now let’s format the additional /tmp partition

node1:~# mke2fs /dev/ubdb1
mke2fs 1.40-WIP (14-Nov-2006)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
125488 inodes, 501744 blocks
25087 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67633152
62 block groups
8192 blocks per group, 8192 fragments per group
2024 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

Writing inode tables: done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 24 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

Now adding the following to the /etc/fstab so it’ll automatically mount the /tmp directory and add the swap:

/dev/ubdb1 /tmp ext2 defaults 0 0
/dev/ubdb2 none swap defaults 0 0

Now he’s got a dedicated swap partition, and a separate /tmp filesystem.

DAEMON Tools Lite is on sale…

$5.99
$5.99
I went all crazy, and loaded up with all the options, but I’ve been wanting a virtual CD burner for a while, and I like being able to mount cue/bin files and not only being able to access the data but playing music.  It’s really cool to mount up a virtual disk, and then assign it out to a VM, so you can launch an older and more temperamental emulator…

UNZ in VMWare
UNZ in VMWare

Anyways, I don’t work for them, I have no affiliate, no ads etc, but I think this is interesting enough to pass on.

So I started to look at the SCSI passthru on Basilisk

And personally I’ve never seen the appeal for such things, but apparently for the world of emulation accessing physical media is a big deal. Of course what I didn’t think about was rescuing old machines by re-installing the OS under emulation, or copy protection.

The first thing I looked for was a GPL project that has SCSI disk support and isn’t too complicated.  The Previous project sure fits that bill, scsi.c is not even a thousand likes, and even better it works!

The only two major hurdles I ran into is that the Mac is sending a page request of 0x30 which as far as I can find is not listed anywhere else.  I ended up just making one up as a reply to see if it mattered.

The other is that it’s scatter/gather based, so when it’s going to read or write several contiguous sectors, it’ll blast down up to 256kb worth of data to be read or written to.  The ability to know that a large operation was in progress was already in Previous, it just wasn’t set to loop.  I guess the NeXT isn’t as aggressive, or it’s SG operations are better contained in the SCSI controller.

The final hurdle was in the Apple partitioning software.  I’ve been down this road a long while ago.  But the disktools from A/UX 3.0.1 doesn’t care about vendors and will thankfully format anything.

SCSI disk files
SCSI disk files

So not as exciting as talking to a real SCSI disk, but it’s safer.  I suspect that accessing a raw NT device name ought to work  I can test that on VMWare, but the trick is finding something that can read HFS and prove it’s a good exercise.

Another ‘feature’ I put back in is the ability to disable the math coprocessor on the 68040.  It feels more stable to rely on Apple’s old emulation code, but maybe that’s me.

As always files for this are on sourceforge.

DMSDos

While trying to see if STAC ever went into the disk compression busness again, I ran across this interesting bit of software, DMSDOS, which is a kernel module for versions 2.0.36 and 2.2.3 of the Linux kernel support read/write access to STACKER version 3 and 4 containers.  It also support’s MS-DOS’s drive space, and doublespace!

The latest version dmsdos-0.9.2.3-pre2-alt2.tar.gz even runs on modern x86_64 and i386 boxes!  Although in library mode.

# ./mcdmsdos list /tmp/STACVOL.000
mcdmsdos version 0.2.0 (for libdmsdos 0.9.x and newer)
libdmsdos: DMSDOS library version 0.9.2pl3-pre2(alpha test) compiled Jun 12 2014 17:14:06 with options: read-write, doublespace/drivespace(<3), drivespace 3, stacker 3, stacker 4
libdmsdos: mounting CVF on device 0x3 read-write…
libdmsdos: CVF end padding 1 sectors.
libdmsdos: CVF is in stacker 4 format.
-rwxr-xr-x 1 0 0 10396254 Dec 16 1993 13:35 doom.wad
-rwxr-xr-x 1 0 0 68923 Dec 15 1993 01:01 setup.exe
-rwxr-xr-x 1 0 0 20850 Dec 15 1993 01:01 readme.exe
-rwxr-xr-x 1 0 0 627155 Dec 16 1993 13:53 u1_94.exe
-rwxr-xr-x 1 0 0 579187 Dec 16 1993 13:47 doom.exe
-rwxr-xr-x 1 0 0 439 Dec 15 1993 01:01 file_id.diz
-rwxr-xr-x 1 0 0 190 Dec 15 1993 01:01 use1_95.bat
-rwxr-xr-x 1 0 0 218 Dec 15 1993 01:01 use1_94.bat
-rwxr-xr-x 1 0 0 7527 Dec 15 1993 01:01 license.doc

How cool?

Even better it can extract files from the stacker volume!

# ./mcdmsdos copyout /tmp/STACVOL.000 /doom.wad doom.wad
mcdmsdos version 0.2.0 (for libdmsdos 0.9.x and newer)
libdmsdos: DMSDOS library version 0.9.2pl3-pre2(alpha test) compiled Jun 12 2014 17:14:06 with options: read-write, doublespace/drivespace(<3), drivespace 3, stacker 3, stacker 4
libdmsdos: mounting CVF on device 0x3 read-write…
libdmsdos: CVF end padding 1 sectors.
libdmsdos: CVF is in stacker 4 format.
# md5sum doom.wad 981b03e6d1dc033301aa3095acc437ce doom.wad
#

And if you google ‘981b03e6d1dc033301aa3095acc437ce’ you’ll know it’s the registered DOOM version 1.1 data file!

The author tired to get it to work with Microsoft Visual C, and it does not.  It also doesn’t work with MinGW or Cygwin, and the reason is once more again the way GCC handles it’s structures on Linux vs Windows.  Sadly there is no silverbullet fix for this, the structures oddly enough are too small on Windows, and too big for what they should be on Linux.

But at any rate, I though it was cool.  For anyone interested all versions that I’ve found I put online on my cvs server, unix.superglobalmegacorp.com.

SAGE CP/M disk fun

Wow this was without a doubt one of the more confusing things I’ve ever done.

So here is the problem.  I want to delete some files from an IMD disk image, and then copy some new ones in.  Easy right? .. maybe.

Ok first up the easiest tool I’ve found to manipulate CP/M disk images is cpmtools.  Even better their pre-compiled binary is for Win32, so I’ll run it with Wine on OS X.  which works fine.  Although there is one slight problem, cpmtools doesn’t read the IMD disk format.  So you will have to download imd118.zip from a backup of the late author’s computer.

Now using IMD you need to convert the OS disk into a ‘raw’ or ‘binary’ file.  Naturally IMD is a MS-DOS program so firing up DOSBox, I ran:

Screen Shot 2013-12-05 at 8.35.58 PM
Uncompressing, so easy!

IMDU CPM68K12.IMD CPM.RAW /B

And a few seconds later I had my raw file.  Now the next thing was to manipulate the image in cpmtools.  cpmtools has a database of disk drive types, and naturally there is no definition for the SAGE2.  However thanks to a friend of mine (hi Lorenzo!) I took at look at 22disk, and found their demo version did in-fact have a definition for the SAGE:

BEGIN SAG2 Sage IV – DSDD 96 tpi 5.25″
DENSITY MFM,LOW
CYLINDERS 80
SIDES 2
SECTORS 8,512
SIDE1 0 1,2,3,4,5,6,7,8
SIDE2 1 1,2,3,4,5,6,7,8
ORDER SIDES
BSH 4 BLM 15 EXM 0 DSM 315 DRM 63 AL0 080H AL1 0 OFS 2
END

Which is great, however it took a bit of experimenting to work out how to format this information for cpmtools.  I compared a bunch of known formats, and then managed to work this out:

diskdef sage2
seclen 512
tracks 160
sectrk 8
blocksize 2048
maxdir 128
skew 1
boottrk 2
os 2.2
end

And now I can look at the image file!

$ wine cpmls -f sage2 CPM68K12.RAW
0:
ar68.68k
armath.sub
as.sub
as68.68k
as68symb.dat
asgo.sub
brwnies.txt
copy.68k
core.sub
cpm.sys
ddt.68k
ddt68000.68k
dump.68k
e.sub
find.68k
halt.68k
init.68k
linkcore.sub
lnk.sub
lo68.68k
m.sub
mcc.sub
mince.68k
mince.swp
mind.sub
orbit.sub
p.sub
pe.sub
pip.68k
print.68k
red.sub
redasm.sub
sage4utl.68k
sagebios.sys
screen.68k
setenv.68k
setprntr.68k
sig.txt
space.sub
spacem.sub
startup.sub
stat.68k
tlnk.sub

So I tidy up the image, and copy it back to the IMD program for compressing.  And this was, without a doubt the most difficult to figure out, until after a bunch of searching, and Lorenzo once more again pointed me in the direction of bin2imd

not intuitive!
not intuitive!

So yeah.

BIN2IMD X.RAW X.IMD DM=2 N=80 SS=512 SM=1-8 /2

And the best part is that it worked!  So now I was able to transfer over a binary version of com.68k, com2.68k, along with Zork, and fire it up!

8080 Zork on 68k CP/M
8080 Zork on 68k CP/M

Unfortunately the interpreter doesn’t work right.  It could be the disk transfers fault, maybe the SIMH SAGE emulator, or even the 8080 emulator.  But it worked this far.

Floppy emulation on the Commodore Amiga

Well for some unknown reason the floppy drive in my Amiga just stopped working.  I managed to find a drive out of an Amiga 500, but it too didn’t work.. I really don’t know if it is the controller or if it is the drives.  And I’m hesitant to sink £100 into a machine that for all I know could have a dead controller.

So this means while I do have the Compact Flash working as a hard disk, I am unable to load an old favorite Captain Blood.  Maybe I am more of a sucker for the Jean-Michael Jarre resampled sound track ethnicolor. Or maybe there is just something redeeming about blowing up random planets.

Then while digging around DICE, I looked at its examples, and one of them was the exec_dev one, FMS a floppy emulator!  I built the example installed the driver, and using it resulted in a nice crash in both AmigaDOS 2.04 and 2.1 .. I had to wonder if this was more so geared to AmigaDOS 1.3 ..?  A bit of googling around and I found an updated version, fms_20.lha.  Installation is a little cumbersome as you have to manually copy in the device driver, and then setup a mountlist describing the devices in question, which really reminded me the “fun” of 4.3 BSD’s disktab.

Another weird thing is that AmigaDOS 2.1 has the FastFileSystem driver built in, so I had to remove those lines…

Using an ADF I found (since there is no way I can read the physical disk now..) I was able to use DMS to make an image on an emulator, copy it onto the flash, then re-DMS it out on the emulated floppy diskette.  Thankfully this game doesn’t use any weird custom filesystem so it was easy enough to mount the disk, and run the game from the emulated floppy.. Or at least launch it.  Trying to use it however resulted in the game just crashing.

Some more digging with a hex editor showed that the string “DF0:” was strewn all through the executable, so a few minutes, and I changed them all to “FF7” and now the game plays properly.

Captain Blood on my Amiga 600
Captain Blood on my Amiga 600

I wonder if it is possible to load the bootblock from one of these custom filesystems then use a virtual void pointer in DICE C, and execute the bootblock directly?  I’ll have to experiment.