Extracting warc files

well since the catastrauphic outage, I’ve been looking through my backups trying to see how much of my ‘vpsland’ archive I have. And it’s not so hot. The good news is the physical machine that has the last known good copy is fine. It’s just in a place I can’t get to on the other side of the world. And I’m still in exile so shipping it really isn’t an option at the moment.

On the plus side I found a warc archive, some 22GB of the 400GB worth of files. So its a start.

So what are WARC files? why do people gzip them to get maybe 1% compression? How do magnets work anyways?

web archives are single snapshots in time of a site. Sounds like a MHT but something more ‘portable’ and open standard-ish. Which means there is a million tools, none of which seem to do exactly what you want.

All I want to do is extract all my files from the WARC, but that seems to not be what most things are geared to, mostly displaying the WARC like a web page, which means clicking hundreds of thousands of files. –yikes

Thankfully warcat seems to be able to fit the bill

python3 -m warcat extract ../user-giddythrill28@vpsland.superglobalmegacorp.com-old-install-2015-10-04-fc233ad0-00000.warc.gz

I didn’t see any package on Ubuntu so did the pip install:

pip3 install warcat

And that seems to have done the trick.

Now to figure out how to setup some cheap storage on azure and copy this stuff up or extract over there.

spot pricing

I’m using the new ‘spot‘ pricing model, to try to keep costs down. Obviously it’s not as good as dedicated slices, but it’ll not make me broke either. And I have a lot more messing around with containers to do, trying to string together nonsense.

Windows scp to remote machines with spaces in the directories

Well one nice thing about Windows 10 is that it has a built in ssh/scp client! Although telnet is optional, I get that it’s insecure but jeez what is a retro user to do?

Anyways the subject at hand is copying files from somewhere that has spaces in the path. In this case I need a copy of OS X Snow Leopard from my Mac Pro cylinder to this junk Fujitsu Celsius. I’m still having USB issues, but I’d like to get my data off of an external disk formatted in HFS+. And for ‘reasons’ I wanted to use something “native” but I don’t feel like building a Hackintosh. While not a strict tutorial on getting Snow Leopard running, I did upload my old download of Empire EFI on archive.org as this kind of stuff is damned near impossible to find.

So back to the matter at hand, I have this VM setup on my Mac Pro, and I want it on this Windows machine. You’d think it would be something like this:

scp -C [email protected]:"/Users/neozeed/Virtual Machines.localized/OSX 10.6/*" .
Password:
scp: /Users/neozeed/Virtual: No such file or directory
scp: Machines.localized/OSX: No such file or directory
scp: 10.6/*: No such file or directory

Okay so double quotes didn’t work. How about a Unix style escape for spaces? I mean it *is* scp after all, maybe it doesn’t know it’s on Windows.

C:\osx>scp -C [email protected]:"/Users/neozeed/Virtual\ Machines.localized/OSX\ 10.6/*" .
Password:
scp: /Users/neozeed/Virtual: No such file or directory
scp: Machines.localized/OSX: No such file or directory
scp: 10.6/*: No such file or directory

Well maybe it parses it like C, so you need double backslash? NO that doesn’t work either. Talk about frustrating. So, in an act of insanity, I tried single quoting the interior spaces around double quotes, something idiotic like a bash variable:

C:\osx>scp -C [email protected]:"/Users/neozeed/Virtual' 'Machines.localized/OSX' '10.6/*" .
Password:
Mac OS Snow Leopard.vmdk                                                               69%   11GB  16.0MB/s   05:16 ETA

And yes, now it’s transferring. I’m just using a cheap 50zt 5 port 100Mbit dumb switch. It’s good enough and it’ll probably take some 30 minutes to transfer all the bits, but it’s working.

So there you go. You may not need it now, or tomorrow but it’ll save you the 20 minutes of frustration!

So there has been a problem

I’ve been on the road for a few months, basically in exile. I’ve downsized my once large online presence to a desktop in an office I’m still obligated to for rent and internet. So my i7 workstation had been up to the task, although it seems I didn’t have the right power settings to turn back on from a power failure.

So now I’m faced with a stale backup as i can’t find my currents. so yeah content has been lost. So i also don’t want to race back to a single point of failure, so once more again I’m going to try a cloud. Microsoft keeps sending me these $200 trial things so i thought I would try it. There is a large learning curve about their networks , and deployments. although there is a bunch of ”shake and bake” deployments to speed things along. As always the key is reading the documents.

I still have a lot of stuff to upload, so for the moment the database is restored but there is a lot of messing around needed for the the old layout.

So appologies for the mess.

Special thanks for Tenox for helping me with all kinds of issues, and support from my muse.

Its always darker before its bright, and its already getting dark in northern Europe.

Happy 4th of July!

No kaboom!

Shockingly no explosions, I was recapping stuff to notice that the PSU I’m using is sliced. Of course a 35 year old PSU runs better. I need some transistors, and maybe some diodes, but I don’t have good access to any at the moment. So weird how 80’s DRAM could need +12/+5/-5v to operate. Oh well.

VOGONS.org turns 20 years old today!

Stiletto had dropped on by to share this amazing milestone!

Today is the 20th anniversary of http://VOGONS.org. I was there helping to brainstorm it into existence in June 2000 at @bravenet on vladr’s VDMSound forums before @zetafleet hosted it, I registered for it on July 1st, and I helped give it its name and “theme”!

Follow the twitter thread here!

For the longest time VOGONS was the place to get information about VDMsound the sound blaster emulator for NTVDM, allowing a far more rich gaming experience on NT, DOSBox, the ubiquitous PC/MS-DOS emulator that is simply everywhere, and of course where I was ‘discovered’ via ‘Quake1 with WATTCP built with DJGPP on DOSBox‘ some 10+ years ago!

So happy 20th to VOGONS!

NetHack now part of the Software Collection at MOMA

Asset 199863

It’s over online display here:

Jean-Christophe Collet made mention of it over on linked in of all places. Link to the article is here. Since I do not have a linked-in account (I was forced to open one at a prior employer, they were an early investor, back when absolutely *nobody* used the thing, before Microsoft had bought them), so I was unable to read the article. But before I had created a smurf account, I found that just switching to incognito mode got me the article in question.

It’s almost crazy to put it in perspective after all these years, but when NetHack became a thing in 1987, the GPL didn’t exist yet, instead opting for the BISON license. There was no NetBSD, nor was there a Net/1 release of BSD, meaning there was no wide-scale free Unix to the Masses.

I’ve tried numerous times to get into it, even playing it a fair bit when I got it running for the x68000. But I never got far, as I’m clearly out of my league. Which is a shame I’m clearly missing out on quite a bit.

Since it’s all news to me, there indeed is a software category over at MOMA. When I did live in New York, I was a member. I did all that artsy stuff, although I’m clearly not the artiste.

Launching Patreon

So yeah, it wasn’t anywhere near as tedious as I thought it would be.

https://www.patreon.com/virtuallyfun

I’ve set prices in Hong Kong dollars, but it claims it’ll re-adjust to your local currency, so it shouldn’t look too crazy as $40 HKD is about $5 USD. After they take all their fees and stuff I get like half. Which I’m not complaining, a week ago it was a big goose egg!

I’ve tried to make the tiers a little fun sounding, although with of course the most important thing we’ve learned on this fun trip is that you absolutely need either an American keyboard, or an Apple branded keyboard to enter recovery mode on a Macintosh while abroad.

As always everything will eventually wind up here. I’ll do some additional collaboration over on Patreon to give it ‘value’ even if it’s just me hitting my head against the wall.

I hate shilling for this kind of thing, ring the bell and click the thing. I’m really embarrassed that it’s come to that here, but like the ads I really don’t have much of a choice at the moment. And of course your direct support by either enabling ads on this page, or signing up just makes it all the easier to keep on through these times, as I’m aware that we are all going through some real fun challenges at the moment.

So a very big preemptive thank you very much, I’m always amazed that whenever I write anything that people read it, and I’m always shocked and amazed to get not only any kind of engagement, but an actual following. Just as I’ve been amazed to have already met so many of you in my travels around the world, through the years!

I would like very much to build on this into bigger things, create more diverse and engaging content, and have some fun doing so!

A big thank-you!

Jason

Much regrets to all my loyal fans

It’s come to that point where I’m basically forced to either step up and do a patreon and beg for money, or go the other way and passively shill for ads.

I’ve gone for plan b.

It’s annoying and yeah I know. It sucks.

This trip has… been a challenge, and I have a stack of unfinished crap I need to push out, but frankly i’ve been too lazy to do it.

Maybe I’ll call it depression as it’s more dressed up and sexy.

Anyways I now have far too much kit where I wanted 3 max, and now I have like far too many. Time to get busy with my Chinese soldering iron, patch up some stuff, and get some of this out of the door.

Anyways yeah, people not using blockers will see the ads.

Also today was frought with additional money stress so yeah.. let’s hope this meets in the middle passively.