WRP – Simple HTML / Reader Mode

(this is a guest post by Antoni Sawicki aka Tenox)

TL;DR
WRP now allows rendering web pages in to a simplified HTML, compatible with old browsers (in addition to Image Map).

Long version
WRP or “Web Rendering Proxy” is a proxy server that allows to use vintage web browsers on the modern web. Originally inspired by Opera Mini/Turbo rendering proxy for mobile devices. I wanted a similar service that would translate modern web pages, but in to it’s older HTML version. This not only proved very difficult, but I realized that the web is advancing in a way that it would not be very future proof. I’m talking about dynamic pages, JavaScript generated content and WASM. Instead, I took a different approach – generating a screenshot of a page with clickable Image Map. This allows to faithfully represent a fully rendered web page on a vintage machine + allow to click anywhere on it and perform actions. At a cost of performance. Rendering GIF or JPEG and transferring over network feels rather slow and clunky.

I have been using WRP for some 10 years now. I began to realize that, this approach, while pretty awesome for show and bragging, is not very practical for day to day use. In fact, my use of web browsers on vintage workstations typically revolves around reading documentation, blogs, wikis and other, “mostly text” websites. It would be much better if these were not clunky screenshots but rather some form of text output.

I again started poking around the original idea of simplified HTML. Looked at various reader modes, print to PDF, etc. In particular, I have noticed recent advancements in so called “web scraping”, extraction and html to markdown conversion services. Likely fueled by the recent AI/LLM craze, as robots scrape the web to learn about humans. What caught my attention are various “html to markdown” services. They can fully render dynamic JS pages and extract contents as it was in a browser. Also, Markdown, if you think about it, is in fact a simplified HTML.

After doing some research, in couple of evenings and less that 100 lines of code I got a basic version going. The principle is as follows: First capture the page HTML, convert to Markdown, do some manipulation like adding link prefixes and remove images (we’ll come back to that later). Then render Markdown back to HTML. Wrap it in a vintage HTML header an off we go. The results are amazing!!

Netscape 4.x on IRIX 5.3 using WRP 4.7

For the “mostly text” pages this is way better than screenshot mode. Not only is way faster and more responsive, you can select and copy text, but also you use the old web browser more like it was originally intended. At any time, if you want to view the screenshot mode, you can simply switch back to PNG/GIF/JPG mode with couple of clicks.

Netscape 3.x on HPUX 9.x using WRP 4.7 looking at Wikipedia

Another interesting aspect of this is extensibility and potential for improvement. For the screenshot mode there just isn’t that much stuff you could add. It’s just a screenshot. For Markdown and simple HTML there’s a million things one could add. Both down and up converters offer a wide variety of plugins and filters. We can improve formatting, layout, processing, add translation and other features. Perhaps also different features based on client browser version. Maybe even input forms and …images.

Lets talk about images. Right now they are completely deleted from markdown. This is for several reasons, compatibility, performance, load time, size, formatting, etc. I’m thinking that perhaps images could be added in some converted form. For example downsized to a small JPG or maybe converted in to ASCII art. Suggestions more than welcome!

Netscape 3.x on OpenVMS 8.x using WRP 4.7 looking at VSI VMS Documentation!

Download from here: https://github.com/tenox7/wrp/releases/tag/4.7.0

To switch to Reader / Simple HTML mode simply change image type to “TXT”. This can also be done using -t txt flag.

Happy browsing!

relax: Segmentation fault

Wasting time doing more “research” on old GCC, and thanks to suggestions I thought that in addition to the old 1.x stuff, but I should include my old favorite 2.5.8, and the stalled 2.7.2.3, and the EGCS Pentium improved GCC fork. I figured re-treading on old ground with the xMach/OSKit build on x86_64 should be safe/quick & easy.

My cross chain fails when trying to build libgcc.a How annoying but I already have one, so I bypass it, and GCC then tries to build the crt (c runtime library startup code) and that fails too!

../binutils-990818-bulid/gas/as-new crtstuff.S -o crtstuff.o
Segmentation fault

I’m using GCC 12.2.0 on Debian 12. Ok maybe I’ve finally hit drift, so let me try some other binutils. binutils-2.10.1, binutils-2.14. I had originally been lying saying I’m a Dec Alpha running either OSF or Linux as it matches the size & endian alignment, but no dice. I found out about the ‘linux32’ command that’ll fake it’s environment as an i686 processor to fake out a lot of builds. But the same result over and over. So, I break down and fire up GDB.

(gdb) r
Starting program: /root/src/xmach/binutils-2.14-bulid/gas/as-new crtstuff.S -o crtstuff.o
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x0000555555592ef0 in md_estimate_size_before_relax (fragP=fragP@entry=0x555555668fa8, segment=segment@entry=0x555555668730) at ../../binutils-2.14/gas/config/tc-i386.c:4441
4441      return md_relax_table[fragP->fr_subtype]->rlx_length;
(gdb) bt
#0  0x0000555555592ef0 in md_estimate_size_before_relax (fragP=fragP@entry=0x555555668fa8, segment=segment@entry=0x555555668730) at ../../binutils-2.14/gas/config/tc-i386.c:4441
#1  0x000055555558bce2 in relax_segment (segment_frag_root=0x555555668f30, segment=segment@entry=0x555555668730) at ../../binutils-2.14/gas/write.c:2266
#2  0x000055555558c39c in relax_seg (abfd=<optimized out>, sec=0x555555668730, xxx=0x7fffffffe960) at ../../binutils-2.14/gas/write.c:659
#3  0x000055555559b01f in bfd_map_over_sections (abfd=0x55555565e030, operation=operation@entry=0x55555558c370 <relax_seg>, user_storage=user_storage@entry=0x7fffffffe960)
    at ../../binutils-2.14/bfd/section.c:1101
#4  0x000055555558b501 in write_object_file () at ../../binutils-2.14/gas/write.c:1565
#5  0x000055555556e288 in main (argc=2, argv=0x5555556302d0) at ../../binutils-2.14/gas/as.c:924
(gdb) quit

The whole issue revolves around md_relax_table! I’d seen a ‘fix’ where you add in a pointer, and it’ll satisfy GCC and sure it’ll compile. Years ago, I had #ifdef’d it out until when I needed it, but the real answer is to embrace 1989 and set the compiler flags to “-std=gnu89”

I can’t help but think at some point soon 1989 will be removed as it’s only wierdos like me building this stuff.

Just as the old Unix error status of sys_nerr has been removed for ‘reasons’ so may as well amputate all the old code:

-  if (e > 0 && e < sys_nerr)
-    return sys_errlist[e];

Nothing much you can do about it, Linux isn’t trying to be Unix anymore.

64/32

In the end it doesn’t seem to matter. OSkit fails to build:

i586-linux-gcc -c -o base_multiboot_init_cmdline.o -MD -DHAVE_CONFIG_H  -DOSKIT_X86 -DOSKIT_X86_PC -DINDIRECT_OSENV=1 -I. -I../../oskit-20020317/kern/x86 -I../../oskit-20020317/kern/x86/pc -I../../oskit-20020317/kern/x86/dos -I../../oskit-20020317/kern  -I- -I../../oskit-20020317/oskit/c -I.. -I../../oskit-20020317 -nostdinc -Wall  -O2 -g  ../../oskit-20020317/kern/x86/pc/base_multiboot_init_cmdline.c
i586-linux-gcc: Internal compiler error: program cc1 got fatal signal 11
make[1]: *** [../../oskit-20020317/GNUmakerules:124: base_multiboot_init_cmdline.o] Error 1

And surprisingly mig does build, but Mach does not.

i586-linux-gcc -c   -MD -DLINUX_DEV=1 -DHAVE_VPRINTF=1 -DHAVE_STRERROR=1  -Di386 -DMACH -DCMU -I- -I. -I../../../kernel/libmach/standalone -I../../../kernel/libmach/c -I../../../kernel/libmach -I/root/src/xmach/xMach/object-kern/libmach -I/root/src/xmach/xMach/object-kern/../kernel/generic/libmach/standalone -I/root/src/xmach/xMach/object-kern/../kernel/generic/libmach/c -I/root/src/xmach/xMach/object-kern/../kernel/generic/libmach -I../../../kernel/include/mach/sa -I../../../kernel/include -I/root/src/xmach/xMach/object-kern/../kernel/generic/include -I/root/src/xmach/xMach/object-kern/include -I/root/src/xmach/xMach/object-kern/../kernel/generic/include/mach/sa -nostdinc  -O1 /root/src/xmach/xMach/object-kern/libmach/bootstrap_server.c
/root/src/xmach/xMach/object-kern/libmach/bootstrap_server.c: In function `_Xbootstrap_privileged_ports':
/root/src/xmach/xMach/object-kern/libmach/bootstrap_server.c:90: `null' undeclared (first use this function)
/root/src/xmach/xMach/object-kern/libmach/bootstrap_server.c:90: (Each undeclared identifier is reported only once
/root/src/xmach/xMach/object-kern/libmach/bootstrap_server.c:90: for each function it appears in.)

Needless to say, this is why I don’t use OS X anymore. Not having a 32bit userland basically killed it for me.

I guess the next step is to go ahead with qemu-user mode wrappers to fake it.

Sorry if you were hoping for some great conclusion.

Two things that really annoy me!

Moving homes. again.

First off, I got a new VPS to house this on, size wise, I’d just plain outgrown the old one, even with SquashFS. Over on lowend box, I had spotted this one: LuxVPS

It’s not an AD, just thought the pricing seemed pretty good for 5€. One of the nice things about converting so much of my data to SquashFS is that moving single files is WAY easier to deal with!

Mice in my 1970’s teletype text editor?!

Using Mice with a 1970’s text editor

But editing text files had me facing off some feature of VIM I’d somehow not dealt with that Debian 11 set by default, and that is mouse integration!

CAN YOU BELIVE IT?

Somewhere out there, is people who use a mouse with a VI clone. 

It bares repeating

SOMEONE THINKS YOU NEED A MOUSE TO USE VI.

So much so, it’s the system default.

Good lord.

The fix is to edit /etc/vim/vimrc:


set mouse=
set ttymouse=

Problem solved. Obviously, I’m not going to remember this, but now I can right click/paste the way G’d intended it!

Stale encryption

Old rusty locks

The next source of annoyance is the ancient stunnel 4.17 that I use for altavista.superglobalmegacorp.com. I’m kind of trapped with this setup as it needs to be a 32bit ‘workstation’ OS, and I don’t want to run something as heavy as XP or Vista when NT 4.0 is more than enough. Anyways OpenSSL won’t talk to this ancient encryption, throwing this error trying to do a connection with “openssl s_client -connect 192.168.23.6:443”:

error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol
Unable to establish SSL connection.

The fix, thanks to dave_thompson_085 is simple enough.

Basically, modify /etc/ssl/openssl.conf and place this at the top:


openssl_conf = default_conf
#
# OpenSSL example configuration file.
# This is mostly being used for generation of certificate requests.
#

then place this at the bottom:


[ default_conf ]

ssl_conf = ssl_sect

[ssl_sect]

system_default = ssl_default_sect

[ssl_default_sect]
MinProtocol = TLSv1
CipherString = DEFAULT:@SECLEVEL=1

Now when I connect to stunnel, I can verify that I am indeed using ancient crap level security:


New, SSLv3, Cipher is AES256-SHA
Server public key is 1024 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1
    Cipher    : AES256-SHA
    Session-ID: 19D20D30E0026E8417E00402DE939E90770D4658C3A9CFE4DB4E5F2A5454DE9D
    Session-ID-ctx:
    Master-Key: 498C648E77E9B9C944A8B1D16242240A161A05A087881C6AD300718DD9B8C443EA12FB76440B666B7C6634A7E7DBE9D5
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1718352960
    Timeout   : 7200 (sec)
    Verify return code: 10 (certificate has expired)
    Extended master secret: no
---
DONE

I don’t care about the encryption, I could as a matter of fact just run without it, as I only need the reverse proxy aspect of it, to make the AltaVista web server accessible over the LAN/WAN/INTERNET. It’s all fronted with CloudFlare so from the end use POV it’s all encrypted anyways

A rainbow of happiness

Sunshine & rainbows!

Another nice side benefit of this SquashFS setup is that I can forever rebase the disks as the content never changes.


#!/bin/bash
# rebase the disk
rm /usr/local/vm/AltaVista/altavista-c.vmdk
rm /usr/local/vm/AltaVista/altavista-d.vmdk
rm /usr/local/vm/AltaVista/altavista-u.vmdk

qemu-img create -f vmdk -b /usr/local/vmdk/AltaVista_C/altavista-c.vmdk -F vmdk /usr/local/vm/AltaVista/altavista-c.vmdk
qemu-img create -f vmdk -b /usr/local/vmdk/AltaVista_D/altavista-d.vmdk -F vmdk /usr/local/vm/AltaVista/altavista-d.vmdk
qemu-img create -f vmdk -b /usr/local/vmdk/AltaVista_U/altavista-u.vmdk -F vmdk /usr/local/vm/AltaVista/altavista-u.vmdk

qemu-system-i386 -vga std -cpu pentium -m 64 \
        -vnc 192.168.23.1:6 \
        -net none  \
        -hda /usr/local/vm/AltaVista/altavista-c.vmdk \
        -hdb /usr/local/vm/AltaVista/altavista-d.vmdk \
        -hdd /usr/local/vm/AltaVista/altavista-u.vmdk \
        -device pcnet,netdev=alta,mac=5a:00:11:55:22:22  \
        -netdev tap,ifname=tap6,id=alta,script=/usr/local/vm/AltaVista/alta-up,downscript=/usr/local/vm/AltaVista/alta-down

One thing is for sure, it makes hosting AltaVista a bit easier to deal with. And for the sake of archiving, I uploaded a pre-loaded & indexed dataset Altavista Pre-Loaded (squashfs). I found that you can just copy the databases into a new VM, as long as you keep the drive letters the same as your source. So luckily, I had kept the OS on C:, installed AltaVista on D: with all the usenet posts on U:. Even better, for those strapped for space, you don’t technically need the U: drive, if you just want to search. Of course, you probably do want to look at them, but we’ve gone down this road before. And we know where it leads.

Index All the things!

Let’s build a Linux kernel from Windows!

Very exciting

Some of you may remember some 9 years ago, I had put together a package to compile Linux 0.10 under Windows.

Time goes on, and things are lost, and it’d come up somewhere about actually building Linux from Windows, so I thought I’d show it off.

The one thing is that modern machines are just so fast, that it’s almost hard to believe that a 386DX 16 with 4MB of ram would struggle for seemingly hours, what an i7 can churn out in mere seconds.

Time sure flies!

It’s my usual ‘DO IT LIVE’ style, I tried to clean up the audio, but I lost the steps… One day I’ll try to script & build a PowerPoint so it’s more cohesive.

But today is not that day.

Squeezing the bytes out of webhosting & Linux with SquashFS & Overlay

Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        29G   27G  2.1G  93% /

It’s a problem that we will all face sooner or later in shared environments, running out of disk space. Back in the old days we would just run stacker and be done with it, but what on earth can we do in this modern age?

Well, there is squashfs which is great at creating ultra-compressed read-only filesystems! Well, that is great, but it is READ-ONLY after all, so that is going to suck right? Well thanks to the magic of file system overlays, we can compress our website, and get the much-needed COW (Copy on Write) to another directory giving us the best of both worlds. It’s a common thing in many live CD’s or any seemingly appliance-based OS where you have a hardened read-only OS core that a user cannot delete/infect but gives the appearance of allowing you to update files. Well, that’s all nice but how do you do it manually?

The first thing I did was shut down Apache so I could get a clean compress of my web document root: mksquashfs is pretty easy to use, and in a few minutes of downtime I was able to create a read-only version of my blog’s filesystem. (NOTE that this doesn’t include the database! so anyone wanting to quick & easily archive WordPress, remember there is always more than just the files!).

root@ukweb:/srv/www/blog# mksquashfs . /usr/local/blog.sqshfs
Parallel mksquashfs: Using 1 processor
Creating 4.0 filesystem on /usr/local/blog.sqshfs, block size 131072.
[===================================================================================================-] 67497/67497 100%

Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
        compressed data, compressed metadata, compressed fragments,
        compressed xattrs, compressed ids
        duplicates are removed
Filesystem size 4604333.36 Kbytes (4496.42 Mbytes)
        82.78% of uncompressed filesystem size (5562424.58 Kbytes)
Inode table size 480413 bytes (469.15 Kbytes)
        33.86% of uncompressed inode table size (1418977 bytes)
Directory table size 430607 bytes (420.51 Kbytes)
        32.31% of uncompressed directory table size (1332573 bytes)
Number of duplicate files found 519
Number of inodes 38856
Number of files 32640
Number of fragments 7872
Number of symbolic links 0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 6216
Number of ids (unique uids + gids) 2
Number of uids 2
        www-data (33)
        root (0)
Number of gids 2
        www-data (33)
        root (0)

Before compression the blog sat at 5.6GB worth of space. After compressing, it now sits at 4.4GB. Not that awesome, but not that bad either! the blog.sqshfs file can be easily mounted on the command-line like this:

mount -o loop /usr/local/blog.sqshfs /srv/www/blog

And it mounted up just fine, and astonishingly the blog worked. Although it being a read-only filesystem means that I cannot upload new content so all the media would be frozen in time, just as I would no-longer be able to make any updates to the pluggins or the software.

Enter the overlayfs, which lets you specify an ‘upper’ and ‘lower’ level for your filesystem where you can have a read-only lower level, and a read-write upper level. Perfect!

I moved the blog read-only mount to /srv/www/blog-ro created a blog-tmp & blog-rw directories as well and mounted up in overlay mode like this:

mount -t overlay -o lowerdir=/srv/www/blog-ro,upperdir=/srv/www/blog-rw,workdir=/srv/www/blog-tmp overlay /srv/www/blog

You’ll notice that despite all the documentation mentioning overlayfs, along with all the posts, as of Linux 5.15 it’s now called overlay.

root@ukweb:/lib/modules/5.15.0-101-generic/kernel/fs/overlayfs# ls
overlay.ko

At least that was easy enough to find.

But you might say, THATS ALL MANUAL! How on earth are you going to deal with a reboot? rc.local?!

Well, you could but /etc/fstab is much easier!

/usr/local/blog.sqshfs /srv/www/blog-ro squashfs ro,defaults 0 0
overlay /srv/www/blog overlay defaults,lowerdir=/srv/www/blog-ro,upperdir=/srv/www/blog-rw,workdir=/srv/www/blog-tmp 0 2

And just like that, I now have a read-only version of the blog data, in a single easy to backup file, along with writes going to a much more manageable directory for updates.

I guess I should add that for sites that use caching, you’ll want to purge the wp-content/cache directory as it’ll become stale, and there really is no point having a read only version of the chache.

If you can see this, then clearly the site is working!

**UPDATE

So I do have a qemu image piggy-backing on my VPS that runs the Apache on NT 3.1 (superglobalmegacorp.com) site. It’s not very complicated, just NT 3.1 with my terrible apache site. Content doesn’t change, it’s a “just because I can” thing.

So you can happily shut down the VM, and in this case I’m using VMDK’s but it really doesn’t matter, I just like having a more neutral container if I want to move stuff around. Just squash the VMDK by itself into a new squash fs file:

# mksquashfs nt31as.vmdk /usr/local/vmdk/NT31_AdvancedServer.vmdk.squashfs
Parallel mksquashfs: Using 1 processor
Creating 4.0 filesystem on /usr/local/vmdk/NT31_AdvancedServer.vmdk.squashfs, block size 131072.
[=====================================================================================================-] 1390/1390 100%

Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
        compressed data, compressed metadata, compressed fragments,
        compressed xattrs, compressed ids
        duplicates are removed
Filesystem size 72383.38 Kbytes (70.69 Mbytes)
        40.68% of uncompressed filesystem size (177925.66 Kbytes)
Inode table size 3918 bytes (3.83 Kbytes)
        69.64% of uncompressed inode table size (5626 bytes)
Directory table size 31 bytes (0.03 Kbytes)
        93.94% of uncompressed directory table size (33 bytes)
Number of duplicate files found 0
Number of inodes 2
Number of files 1
Number of fragments 0
Number of symbolic links 0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
        root (0)
Number of gids 1
        root (0)

The disk image went from 174MB to 71MB. Not bad!

Next, setup a mount point in /etc/fstab

/usr/local/sqshfs/NT31_AdvancedServer.vmdk.squashfs /usr/local/vmdk/NT31_AdvancedServer squashfs ro,defaults 0 0

Now we create the backing file to point to the original VMDK where all write operations will take place. And of course this means that the site can be reverted very quickly if something goes wrong.

qemu-img create -f vmdk -b /usr/local/vmdk/NT31_AdvancedServer/nt31as.vmdk -F vmdk nt31as.vmdk
Windows NT 3.1 with compressed backing store VMDK

And just like that, we’ve now up and running!

Building MS-DOS 4.00 under OS/2 2.x

Building & Booting DOS on OS/2!

Now that we’ve moved beyond the initial shockwave of the MS-DOS 4.00 source code dump, I thought it was time to try to pull off the ultimate trick of the time, building under OS/2 and using the exciting feature of the time “DOS from Drive A:” Long before VMware / Virtual PC for the PC OS/2 took Intel’s 80386’s hardware virtualization mode, “v86 mode” to the logical conclusion allowing you to boot native MS-DOS under OS/2. Sadly, the old 1989-1991 OS/2 betas do not include this feature. Although I have to wonder if it did exist and just wasn’t publicly available.

Many of the programs used to build MS-DOS are off the shelf, the MASM assembler, Microsoft C 5.1, and its associated tools are just retail versions. To change things up, I did use the 386MASM assembler just to see if it maintained MASM 5.1 compatibility. And it does. The only gotcha is that all the tools are *NOT* marked Presentation Manager compatible, so launching them from a window opens a full screen session. Very annoying!

exehdr

I’m guessing the fix is in a toolkit? Either way, in Microsoft C 6.0, the utility exehdr lets us modify an OS/2 executable so it’ll now be WINDOWCOMPAT. So at least it ‘feels’ better now.

One thing is for sure, building DOS under OS/2 is a lot more enjoyable than doing a native build as you can minimize the build task, although the MS-DOS only programs do pop up when it generates text indexes & tables. But you do retain some control of your machine during the build, which is great! Although E is a terrible editor for source code, and the one in 6.78 has a nasty bug where it’ll truncate large files. Were people really using text mode editors for everything back then? I guess i like the fonts of the GUI, despite having used machines of the era.

Otherwise, the end result is the same, you get a build of DOS 4.

I went ahead and tried to build using 6.78 and no doubt compiling DOS is an absolute torture test. So far, the DOS Box has locked OS/2 once, and PM Shell has crashed once as well.

I altered the Makefiles to use ‘rm’ instead of the built in ‘del’ command, because if you try to delete a file that doesn’t exist, del returns an error, which then triggers an end to the NMAKE process. Very annoying! However, the ‘rm’ included in Microsoft C 5.1 doesn’t suffer the same defect. Using 86Box with an 83Mhz Pentium OverDrive it took about 18 minutes to build DOS-4.

Building DOS 4 on OS/2 6.78

I did capture the video and converted it to a GIF so you can quickly see the reboot & the UI crashing. FUN!

Compiled under OS/2

And it even boots!

For anyone interested I’ve put zips on archive.org that can be extracted under OS/2. I also made a pkzip disk set incase loading a 6MB zip file is an issue.

Building MS-DOS 4.00 under OS/2 2.x : neozeede : Free Download, Borrow, and Streaming : Internet Archive