Some follow up on Stacker


From my OS/2 experiments before I roll it out… 
  • It only supports FAT.
  • Maximum of 2GB ‘compressed’ drives
  • Swapping drive letters assumes 1 disk 1 volume
  • You create the compressed disks in DOS

So, since I’m thinking of BBS space, I can leave part of the disk uncompressed for zip’s then the compresses partition for databases & doors.

I guess the thing to keep in mind is that 1991-1994 2GB disks were not in the hands of your average user.. And the idea of using that much storage seemed crazy.  It’s a shame they did that deal with Microsoft and basically got pushed out of the market.  It’s a shame that the OS/2 product doesn’t actually run on OS/2, nor support HPFS.  The idea of bigger disks, and long file names would have been nice.
Oh well I guess that is how the older stuff dies out.

Stac Electronics Stacker for OS/2

Once upon a time hard disks were expensive.  A device that could hold a terrabyte would cost hundreds of thousands in the late 1990’s!  I remember Windows NT 3.51 taking 3 days to format 890GB!!!

Even when I got my first 20MB hard disk, it along with the controller cost several hundred dollars.  I had upgraded to a 286 from a Commodore 64, and even a 720kb floppy felt massive!  I figured it’d take a long while to fill 20MB so I was set.  Well needless to say I was so wrong!  Not to mention there was no way I could afford a 30MB disk, I wasn’t looking for a questionable used 10MB disk, and a 40MB disk was just out of the question!

Until I saw this:

Stacker 2.0

Stacker changed everything for me, now via software compression not only could I fit 40+MB worth of crap on my 20MB disk, but I could even get more data on my floppies!  The best part about stacker, unlike pack/zoo/pkzip and friends is that the compression was transparent.  Meaning you load the driver, and pickup a new driver letter for the compressed volume, and from that point onward everything you do to that disk is compressed.  All MS-DOS programs work with it.  Yes really from Windows 3.0, to dbase, BBS packages, even pkzip (although it’ll get a 1:1 compression)

Stacker for OS/2

So for a long while life was good with Stacker although like everything else, things changed.  I wasn’t going to run MS-DOS forever, and when I switched to OS/2 I was saddened to find out that there was no support for OS/2. Also Linux support was not going to happen either.  Although they did eventually bring out a version for OS/2 it did not support HPFS volumes, only FAT.

And as you can see while Warp was the main target it could function with OS/2 2.0 and 2.1 as long as you had updated them to the latest fixpacks.  And preferably installed the OS onto a FAT partition as the setup process hinges on it being able to modify the config.sys & boot directories from a DR-DOS boot floppy included in the package.

One of those funny things about disk compression is that for very heavy disk access programs that tend to compress pretty good (say databases with lots of text records) is that compression can greatly speed them up.  If you are getting 16:1 compression (ie you have LOTS of spaces…) you only have to read the hard disk 1/16th as you would on an uncompressed volume.  It’s still true to this day, although people tend to think disk compression added a significant amount of overhead to your computer (I never noticed) it can make some things faster.

Another thing that STAC was involved in was selling compression coprocessors.

Stac co-processor ad

While I’m not aware of these cards being that big of a seller, It is interesting to note that these co-processors were also available for other platforms namely the cisco router platform.  Since people were using 56kb or less links, the idea of taking STAC’s LZS compression and applying it to the WAN was incredible!  Imagine if you were printing remotely and suddenly if you got even a 4:1 boost in speed, suddenly things are usable!  The best part is that while there were co-processors cisco also supplied the software engine in the IOS.  Enabling it was incredibly easy too:

interface serial0/0

compress-stac

And you were good to go.  The real shame today is that hardly anyone uses serial interfaces so the compression has basically ‘gone away’.  Cisco did not enable it for Ethernet interfaces as those are ‘high speed’… Clearly they didn’t foresee that people would one day get these infuriating slow “high speed” internet connections being handed off on Ethernet and how compressing them would make things all the better.

I think the general thrust has been to a ‘black box’ approach that can cache files in the data stream, provide compression and QoS all in one.

So until I re-install OS/2 on a FAT disk, let’s run this through the Windows 3.0 Synchronet BBS I was playing with.

Stacker starts up with some ascii art of a growing hard disk.  Maybe one day I’ll figure out how to dump Qemu’s video into something to make animated GIFs with.  Until then… well.

Setup is pretty straight forward, pick a disk to compress, then decide if you’ll do the whole disk and incorporate the drive swapping fun, or just do the free space to create a sub volume, and manually manage it.  I’m not in the mood to reconfigure anything so I’ll do what most people did and compress the whole drive.  What is kind of fun to note, is that in modern implementations of compress & encryption you have to select one or the other you cannot do both.  However using emulators that can support encrypted disks, you *could* then compress from within.  I don’t know why the new stuff doesn’t let you, maybe the layered containers gets too much.  But I do bet there is something out there for Windows that’ll mount a file as a disk image (like Windows 7 can mount VHD’s) on an encrypted volume, then turn on compression…….

Anyways with the target selected, it will then copy the files, then create the volume…

Which took a minute, then it’ll defrag the disk using Norton Speed disk (from that point onward it seems that speed disk disappears..?)

And once that is done, we are basically good to go.  So how did it do on my 80MB ‘test’ disk?

I’d have to say pretty good, 2.5:1 is pretty snazzy!  And my BBS is working, I didn’t have to change a single line, it’s 100% transparent.

Eventually IDE hard disks took over the world, and got larger capacities, faster and cheaper.  Not to mention the world was switching operating systems from MS-DOS to Windows 95, or Windows NT and STAC just got out of the market after the big lawsuit.

But it’s funny looking at old disk ads…  And what a catastrophic thing it was to fill the things up.

But before then, we had stacker…

Setting up a permanent swapfile under Windows 3.0

So while I was playing around with the idea of a Windows 3.0 powered (lol) BBS I ran into one weird thing with it swapping like crazy, and eventually depleting all my disk space!

Well when I actually used Windows 3.0 back in the day, I had a 286 so I never got to play with virtual memory.  By the time I got a 386, I was using Windows 3.11 & OS/2 which in Windows you configure it with the 386 icon in the control panel.  On OS/2 it’s a line in the config.sys.

But for Windows 3.0 it’s a standalone program, swapfile.exe.  And the best part is that you cannot run it in 386 Enhanced mode, nor in standard (286) mode.  You have to first start Windows in realmode.

So now I could force it to swap on a secondary drive, or more importantly contain it’s filegrowth so it doesn’t take up the entire disk.

Idling under MS-DOS 4.01 & Windows 3.0

Well I was playing some more with my MS-DOS setup, and the annoying thing is that it doesn’t idle worth anything.

And of course running old DOS doesn’t help that most everything needs MS-DOS 5.0 or higher.

But then I found WQGHLT, which is a VXD to support idling under Windows 3.1.  And happily it works on Windows 3.0 …

The only pain is that running MS-DOS applications kills the CPU (again).  Which means that the DOS programs themselves need to idle.  And yes I did check the ‘detect idle’ but it seems that it doesn’t do what I’d hope.

Maybe it’s just the software, I should try some more and see what I get.

BBS’ing with Windows/386 & Windows 3.0 under Qemu or how I learned to love rlfossil

A while back I had seen this fantastic site, “Hates the internet” with a great write up on setting up a BBS on Qemu. In retrospect it did inspire me a bit later to get my BBS going with Qemu, but I chose to use OS/2 once I found out about SIO’s vmodem feature.

HTI (Hates the internet) chose this program called rlfossil, which is for MS-DOS..

RLFOSSIL is an implementation of multi-line serial port driver corresponding to the Fido/Opus/Seadog level 5 specification and a simple HAYES-compatible modem emulator. It allows applications usually worked through BBS’s to run on the Internet, or in IP-based local net.er, and rlogin and telnet emulation using IP services numbers 513 & 23. RLFOSSIL allows combined work with other FOSSIL drivers (X00,BNU etc.).

So, I thought between that, and all the Windows/386 excitement I’d try for something even more insane. How about running a multiline BBS on Windows?

In the same effort, I was going to use Qemu 0.14.1, with MS-DOS 4.01 (the first version I could find that came with share.exe), and Windows/386 2.11. The installation of MS-DOS 4.01 worked fine on an 80MB disk image, thankfully it was one of the things that DOS 4 could do better than 3 is large disk images… Yes, I know 3.31 could as well, but it didn’t come with share so it was out.  One strange thing after install was this message…

It is kind of foreboding that DOS is warning me that because of my “large” disk I better run share. Since I plan on having a multi node BBS all in one computer, I need to run share anyways.

The next exciting part was installing Windows/386 2.11. The installation went pretty smooth, and with Qemu the mouse worked fine.  So far, so good.  I couldn’t use himem.sys that comes with Windows/386, nor could I use the himem.sys that comes with MS-DOS as the Windows/386 version complains that that A20 line is already active (?) and the MS-DOS one has Windows complaining that the HMA is already in use. Sadly, then my conventional memory footprint will be unsatisfactory, but I don’t see any way around it.

The next part is configuring rlfossil. rlfossil needs a driver to talk to the network card, and you can find them on crynwr, namely the ‘other‘ packet archive, which contains NE2000 drivers.  Keeping with HTI, I’m going to use the NE2000 and configure Qemu with the PCI NE2000 driver.

Packet drivers are loaded from the command line something like this:

ne2000 0x60 11 0xc100

This loads the driver on software interrupt 0x60, and by default the PCI NE2000 is configured for IRQ 11, port 0xc100.  Qemu 1.6.0 changed the PCI NE2000 to use port 0xc000 for what it is worth..

So keeping with the HTI tradition, I’m going to put my packet driver (ne2000.com) and unpack the rlfossil archive in c:\packet. The next thing to do is configure rlfossile which uses the wattcp configuration file.  Since I’m going to use the usermode NAT and a redirect, I configure my VM like this:

Wattcp.cfg

Address:10.0.2.15
Netmask:255.255.255.0
Gateway:10.0.2.2
DNS: 10.0.2.3

With that all in place now it’s time to configure the config.sys/autoexec.bat.  Some things are going to be different from a normal install because we plan to run a BBS, and multiple instances of it!

So my config.sys looks like:

FILES=96
STACKS=0,0
DEVICE=C:\DOS\ANSI.SYS
SHELL=C:\COMMAND.COM /P /E:768

And my autoexec.bat is like this:

PATH C:\WIN386;C:\DOS
PROMPT $P$G
SHARE
SET TEMP=C:\TEMP
CD \PACKET
NE2000 0x60 11 0xC100
RLFOSSIL 0 4 WIN386

And of course launching Qemu I do it like this:

qemu.exe -L pc-bios -m 16 -net nic,model=ne2k_pci -net user-redir tcp:23::23 -hda telegard.qcow2

This configures the VM for 16MB of ram (which would have cost a FORTUNE back then), the PCI NE2000, and it’ll redirect telnet from my host machine into the VM.

And just like HTI, I went with telegard, because it supports fossil based ports.

Well that sure was a *LOT* of work, and surprisingly testing it with a single node, actually works.  And you can bring up a few other MS-DOS prompts and it’ll work fine. But if you launch the second node…

Disaster struck.  So needless to say, while Windows/386 was pretty slick for the day it just couldn’t measure up.  So I figured for the hell of it, I’d try Windows 3.0 Â I mean I would have imagined that Windows 3.0 most certainly could NOT handle this kind of challenge.

So with some disks shuffled, I fired it up and..

Two node telegard under Windows 3.0

It actually worked!  So with a LOT of chaos going on I managed to get Trade Wars 2002 running, although I couldn’t figure out how to automatically figure out the node.. Hell the whole door configuration thing is.. bizarre. Synchronet really kicks ass in regards to easy of configuration.

Running TW2002, two copies

And using PIF’s to configure each node for some easy of launching, and some reduced memory, I could easily run all four nodes that rlfossil can support.

Four Nodes!

I have to admit, Windows 3.0 really is impressive considering all the UAE’s and how generally crappy we thought it was at the time.  I’m sure even emulated having a multiple Ghz cpu helps quite a bit.

460KB free!

And look at all that memory.. I guess it’s pretty impressive it even works.  Since Windows anything throttles the CPU at 100% I’m not going to put this online…. Although at the same time combined with an CPU idle program (is there a Windows 3.0 idle vxd?) it sits ok, but who wants a single user system in 2011?

World Of Warcraft has gone “free”…

Good lord like things could possibly look any worse!

The end of humanity

I don’t know what even overcame me as I read the news on thinkq, but I’m already thinking of backing out. LOL  But it’s now “free” to people for the first 20 levels.  But who would want to quit at level 20 and not go all the way to 87?

Oh the humanity.

For anyone crazy enough you can join HERE (Europe) or HERE (United States), HERE(Korea), HERE(Chinese)

Oddly enough no links for Japan.

Anyways don’t say I didn’t warn you but…. The full install is also some 10GB!  Good luck you people in capped areas.

OS/2 2.1 and TCP/IP 2.0

So I thought I could simply take my OS/2 2.0 install on Qemu with an ISA NE2000 and simply upgrade it to OS/2 2.1 and it’d work, right?

WRONG.

Dead WRONG.

So after an hour screwing around with various drivers, I found of all things, the PCI NE2000 will work.  The driver is available os2site.com

Thankfully the NIF’s and OS2 bits work right so you can actually use LAPS to configure it, once you copy the NIF & OS2 files into the c:\ibmcom\macs directory.

LAPS to the rescue (for once)

And then with that in place configured for user mode networking (10.0.2.15/255.255.255.0/10.0.2.2 DNS 10.0.2.3)

All is working as expected.  I’m hoping that the DPMI emulation is better in 2.1 vs 2.0.  Although what is interesting is that after applying XR06200, my OS/2 2.1 is now 2.11 …

terraria networking

Personally, I couldn’t care less about Terraria, although it reminds me a bit of Zelda III back in the day, but it does have this multiplayer capability.  But it never mentions the TCP port.  Well the default port is:

7777

And if you have an Apple airport network, a teenager that needs to have ‘cut off times’ and sits 2 levels of NAT in, you’ll have to forward TCP 7777 all the way to their computer.  It’s not the end of the world but UPNP doesn’t work in nat-nat scenarios (thankfully) but the blasted game doesn’t mention when setting up a network game, oh by the way you’ll need to NAT TCP 7777.

Oh well that is all.

Building Qemu for a VPS

So touching on my prior long winded post, I was mentioning that running Qemu ‘headless’ on a resource constrained VPS was different then running it interactively at home.  For starters you don’t want to install X11, and all the necessary libraries just for a single program if your VPS is limited to 2GB, as it is more then likely that you will exceed your disk space just by trying to get a workable environment.

Instead, I’ve found the way to go is to take advantage of Qemu’s built in VNC server.

I build my own version of Qemu at home, then just upload the binary.  While there may be packages, because of the expectation that you’ll be running X11 for the SDL output it could easily pull in several gigabytes, that I’d rather use for disk images.  I recommend people build with the same OS as their VPS uses, to keep things ‘sane’.  Not to mention you’ll have a far higher success rate by matching OS levels.  And yes I do this in a VM, all the more reason to have VMs.

Also I should mention that in constrained space, I find that 32bit OS stuff is smaller, if you have a 64MB memory limit, then yes size does matter.

The steps are pretty simple, it’s basically download & unpack the source, then configure it something like this:

./configure --target-list=i386-softmmu --disable-sdl --enable-vnc-thread --disable-bluez --disable-curl --disable-vnc-tls --disable-vnc-sasl --disable-spice

Then run make, and after a few minutes it should complete the build.  I then manually ‘strip’ the qemu executable, tar it up and then take it for a test run.  For the most part it really is that simple.

Again I’ve found building on the same distro as your VPS is using helped an incredible amount.  Not to mention trying to keep your ‘build’ machine as bare as possible so it’s not pulling in all kinds of wild dependencies.

The next thing with Qemu uploaded is you’ll want to test it.  It works like normal Qemu with the exception of how you then specify your display to go out via VNC. For me, I run it like this:

./qemu -m 16 -hda os2.disk -hdb data500.qcow2 -vnc <Public IP>:101 -net nic,model=ne2k_isa -net user -L pc-bios -fda empty.vfd -no-reboot  -redir tcp:23::23

Then you can connect with just about any VNC client on VNC port :101 (or TCP 6001)

My OS/2 BBS on Qemu via VNC

And from there I just wrap it in a while true/do true script so that way every time I reboot the VM or if it were to crash for what ever reason it’ll keep on trying to start it over and over.

Back to memory consumption, as you can see I’ve configured my Qemu intance for a whopping 16MB of ram, and this is what it looks like according to top:

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND 2785 root      20   0 72868  33m 2164 S    2 57.5   1:03.69 qemu

Which is the resident size is 33MB, out of my 64MB VPS, with lhttpd running and php for the socket support I have a whopping 1MB free!

That about covers it, and for the Debian people out there here is my build of Qemu 0.14.1. I should imagine it’ll run on other i386 Linux OS’s (or x86_64’s with the 32bit compatibility installed)

Another day in the cloud.

So my BBS thing is down… hard.  After asking the provider ….

It appears that one of the air conditioning facility at the data centre has ceased operation and so servers are struggling with the heat. Emergency work is already being carried out.

Such is life in the ‘cloud’.

It’s times like this it’d be nice if doubletake were cheap enough for us low end users.

I suppose there is rsync, but that’d take a little work with Qemu, or would it?