WRP 4.0 Preview

(This is a guest post from Antoni Sawicki aka Tenox)

Welcome a completely new and absolutely insane mode of Web Rendering Proxy. ISMAP on steroids!

While v3.0 was largely just a port from Python/Webkit to GoLang/Chromedp, the new version is a whole new game. Previously WRP worked by walking the DOM and making a clickable imagemap out of <A HREF> nodes. Version 4.0 works by using x,y coordinates obtained from ISMAP to perform a simulated mouse click in Chrome browser. This way you can click on any element of the page. From annoying cookie warnings, to various drop down menus and even play some online games. Also pagination has been replaced with a clickable scroll bar.

Enough talking, you can watch this video:

Or download the new version and try it yourself!

Please report bugs on github.com. Thank you!

WRP 3.0 Beta ready for testing

(This is a guest post from Antoni Sawicki aka Tenox)

I have released WRP 3.0 for testing. It’s currently a browser-in-browser server rather than a true proxy, but that’s in the works. Please try it out and let me know. Usage instructions are on the main github project page.

Today using trickery I was able to login to my reddit account from Mosaic:

Update: just added the missing image quantizer so that the color number input box actually does something useful. Now you can browse porn even with 16 colors:

WRP Runs on Windows

(This is a guest post by Antoni Sawicki aka Tenox)

Thats right, the new beta version of Web Rendering Proxy runs natively on Windows. Single EXE, no libraries or dependencies required. Only Chrome Browser.

I took a Internet Explorer 1.5 for a spin today while WRP was running on my Windows 10 PC. Worked just fine.

I have added Prev/Next buttons so that you can easily “scroll” through long pages.

ISMAP support has been added, proof:

You can download a preview build on github.

Web Rendering Proxy – Overdue Status Update

(This is a guest post from Antoni Sawicki aka Tenox)

There hasn’t been a major update to WRP (Web Rendering Proxy) in 5 years or so. Some new features have been added thanks to efforts of Claunia but the whole project was mostly impeded with mass migration of the whole Internet to SSL/TLS/https. It does semi work somehow thanks to sslstrip but the whole stack is an unmaintainable pile of crap which I’m not going to update any more.

A new rewrite from scratch is well under way. This time written in GoLang and using Chrome DevTools Protocol. Things should be much more stable and future proof.

Far from complete but I have a fully functional prototype now working in just under 100 lines of code:

UPDATE 1: You can play with it if you want. Please do not submit any bug reports just yet, as this is just a development version. Note that WRP is currently not a true HTTP proxy but rather browser-in-browser. Proxy may be supported later.

UPDATE 2: As of today online setting of size, scaling and scrolling is supported. I’m specifically happy about the scrolling feature albeit it probably needs a better user input, like prev/next page.

Windows version still doesn’t work due to an upstream bug, which is probably easy to fix.

ISMAP is currently in development.

Comcast / AT&T Router and SNMP

(this is a guest post by Antoni Sawicki aka Tenox)

This is a lame duck, low effort post. If you already know it’s 🤦 obvious. However this question seem to be asked a lot on the intertubes. I was stupid and searched for it too. I hope this will help someone else, as there is no good readily available answer out there.

Problem: I want to have SNMP on my Comcast/Xfinity/AT&T router so I can monitor current bandwidth usage.

Research:

  • Possible to enable on vanilla router? – Nope
  • Do 3rd party, compatible routers do? – Nope
  • Can you SSH or hack in to the router to do it? – Nope
  • Can you load custom / hacked firmware to do it? – Nope*

Nope or very hard / unsupported.

So is it possible at all? Yes, but with a separate device.

Comcast/Xfinity routers have so called “Bridge Mode” which essentially turns them in to a DOCSIS modem without the router / firewall / wifi access point. AT&T has a passthrough mode which works similarly.

Solution: Turn on Bridge/Passthrough Mode in your vanilla router and buy a WRT firmware router / access point. I got Linksys AC3200 for $99 on Amazon. Ssh to the router and run: opkg install snmpd. 🤦

Done.

Why BSD/OS is the best candidate for being the only tested legally open UNIX.

This is an anonymous guest post. Disclaimer: Nothing in this post constitutes legal advice. The author is not a lawyer. Consult a legal professional for legal advice.

Introduction

The UNIX® system is an old operating system, possibly older than many of the readers of this post. However, despite its age, it still has not been open sourced completely. In this post, I will try to detail which parts of which UNIX systems have not yet been open sourced. I will focus on the legal situation in Germany in particular, taking it representative of European law in general – albeit that is a stretch, knowing the diversity of European jurisdictions. Please note that familiarity with basic terms of copyright law is assumed.

Ancient UNIX

The term “Ancient UNIX” refers to the versions of UNIX up to and including Seventh Edition UNIX (1979) including the 32V port to the VAX. Ancient UNIX was created at Bell Laboratories, a subsidiary of AT&T at the time. It was later transferred of the AT&T UNIX Support Group, then AT&T Information Systems and finally the AT&T subsidiary UNIX System Laboratories, Inc. (USL). The legal situation differs between the United States of America and Germany.

In a ruling as part of the UNIX System Laboratories, Inc. v. Berkeley Software Design, Inc. (USL v. BSDi) case, a U.S. court found that USL had no copyright to the Seventh Edition UNIX system and 32V – arguably, by extension, all earlier versions of Ancient UNIX as well – because USL/AT&T had failed to affix copyright notices and could not demonstrate a trade secret. Due to the obsessive tendency of U.S. courts to consider themselves bound to precedents (cf. the infamous Pierson v. Post case), it can be reasonably expected that this ruling would be honored and applied in subsequent cases. Thus under U.S. law, Ancient UNIX can be safely assumed to belong in the public domain.

The situation differs in Germany. Unlike the U.S., copyright never needed registration in order to exist. Computer programs are works in the sense of the German 1965 Act on Copyright and Related Rights (Copyright Act, henceforth CopyA) as per CopyA Â§ 2(1) no. 1. Even prior to the amendment of CopyA Â§ 2(1) to include computer programs, computer programs have been recognized as copyrightable works by the German Supreme Court (BGHZ 112, 264 Betriebssystem, no. 19); CopyA Â§ 137d(1) rightly clarifies that. The copyright holder at 1979 would still have been USL via Bell Labs and AT&T. Copyright of computer programs is transferred to the employer upon creation under CopyA Â§ 69(1).

Note that this does not affect expiry (Daniel Kaboth/Benjamin Spies, commentary on CopyA Â§Â§ 69a‒69g, in: Hartwig Ahlberg/Horst-Peter Götting (eds.), Urheberrecht: UrhG, KUG, VerlG, VGG, Kommentar, 4th ed., C. H. Beck, 2018, no. 16 ad CopyA Â§ 69b; cf. Bundestag-Drucksache [BT-Drs.] 12/4022, p. 10). Expiry occurs 70 years after the death of the (co-)author that died most recently as per CopyA Â§ 65(1) and 64; this has been the case since at least the 1960s, meaning there is no way for copyright to have expired already (old version, as per Bundesgesetzblatt Part I No. 51 of September 16, 1965, pp. 1273‒1294).

In Germany, private international law applies the so-called “Territorialitätsprinzip” for intellectual property rights. This means that the effect of an intellectual property right is limited to the territory of a state (Anne Lauber-Rönsberg, KollisionsR, in: Hartwig Ahlberg/Horst-Peter Götting (eds.), ibid., pp. 2241 et seqq., no. 4). Additionally, the “Schutzlandprinzip” applies; this means that protection of intellectual property follows the lex loci protectionis, i.e. the law of the country for which protection is sought (BGH GRUR 2015, 264 HiHotel II, no. 25; BGH GRUR 2003, 328 Sender Felsberg, no. 24), albeit this is criticized in parts of doctrine (Lauber-Rönsberg, ibid., no. 10). The “Schutzlandprinzip” requires that the existence of an intellectual property right be verified as well (BGH ZUM 2016, 522 Wagenfeld-Leuchte II, no. 19).

Thus, in Germany, copyright on Ancient UNIX is still alive and well. Who has it, though? A ruling by the U.S. Court of Appeals, Tenth Circuit, in the case of The SCO Group, Inc. v. Novell, Inc. (SCO v. Novell) in the U.S. made clear that Novell owns the rights to System V – thus presumably UNIX System III as well – and Ancient UNIX, though SCO acquired enough rights to develop UnixWare/OpenServer (Ruling 10-4122 [D.C. No. 2:04-CV-00139-TS], pp. 19 et seq.). Novell itself was purchased by the Attachmate Group, which was in turn acquired by the COBOL vendor Micro Focus. Therefore, the rights to SVRX and – outside the U.S. – are with Micro Focus right now. If all you care about is the U.S., you can stop reading about Ancient UNIX here.

So how does the Caldera license factor into all of this? For some context, the license was issued January 23, 2002 and covers Ancient UNIX (V1 through V7 including 32V), specifically excluding System III and System V. Caldera, Inc. was founded in 1994. The Santa Cruz Operation, Inc. sold its rights to UNIX to Caldera in 2001, renamed itself to Tarantella Inc. and Caldera renamed itself The SCO Group. Nemo plus iuris ad alium transferre potest quam ipse habet; no one can transfer more rights than he has. The question now becomes whether Caldera had the rights to issue the Caldera license.

I’ve noted it above but it needs restating: Foreign decisions are not necessarily accepted in Germany due to the “Territorialitätsprinzip” and “Schutzlandprinzip” – however, I will be citing a U.S. ruling for its assessment of the facts for the sake of simplicity. As per ruling 10-4122, “The district court found the parties intended for SCO to serve as Novell’s agent with respect to the old SVRX licenses and the only portion of the UNIX business transferred outright under the APA [asset purchase agreement] was the ability to exploit and further develop the newer UnixWare system. SCO was able to protect that business because it was able to copyright its own improvements to the system. The only reason to protect the earlier UNIX code would be to protect the existing SVRX licenses, and the court concluded Novell retained ultimate control over that portion of the business under the APA.” The relevant agreements consist of multiple pieces:

The APA dates September 19, 1995, from before the Caldera license. Caldera cannot possibly have acquired rights that The Santa Cruz Operation, Inc. itself never had. Furthermore, I’ve failed to find any mention of Ancient UNIX; all that is transferred is rights to SVRX. Overall, I believe that the U.S. courts’ assesment of the facts represents the situation accurately. Thus for all intents and purposes, UNIX up to and including System V remained with Novell/Attachmate/Micro Focus. Caldera therefore never had any rights to Ancient UNIX, which means it never had the rights to issue the Caldera license. The Caldera license is null and void – in the U.S. because the copyright has been lost due to formalities, everywhere else because Caldera never had the rights to issue it.

The first step to truly freeing UNIX would this be to get Micro Focus to re-issue the Caldera license for Ancient UNIX, ideally it would now also include System III and System V.

BSD/OS

Another operating system near UNIX is of interest. The USL v. BSDi lawsuit includes two parties: USL, which we have seen above, and Berkeley Software Design, Inc. BSDi sold BSD/386 (later BSD/OS), which was a derivative of 4.4BSD. The software parts of the BSDi company were acquired by Wind River Systems, whereas the hardware parts went to iXsystems. Copyright is not disputed there, though Wind River Systems ceased selling BSD/OS products 15 years ago, in 2003. In addition, Wind River System let their trademark on BSD expire, though this is without consequence for copyright.

BSD/OS is notable in the sense that it powered much of early internet infrastructure. Traces of its legacy can still be found on Richard Stevens’ FAQ.

To truly make UNIX history free, BSD/OS would arguably also need to see a source code release. BSD/OS at least in its earliest releases under BSDi would ship with source code, though under a non-free license, far from BSD or even GPL licensing.

System V

The fate of System V as a whole is difficult to determine. Various licenses have been granted to a number of vendors (Dell UNIX comes to mind; HP for HP-UX, IBM for AIX, SGI UNIX, etc.). Sun released OpenSolaris – notoriously, Oracle closed the source to Solaris again after its release –, which is a System V Release 4 descendant. However, this means nothing for the copyright or licensing status of System V itself. Presumably, the rights with System V still remain with Novell (now Micro Focus): SCO managed to sublicense rights to develop and sell UnixWare/OpenServer, themselves System V/III descendants, to unXis, Inc. (now known as Xinuos, Inc.), which implies that Xinuos is not the copyright holder of System V.

Obviously, to free UNIX, System V and its entire family of descendants would also need to be open sourced. However, I expect tremendous resistance on part of all the companies mentioned. As noted in the “Ancient UNIX” section, Micro Focus alone would probably be sufficient to release System V, though this would mean nothing for the other commercial System V derivatives.

Newer Research UNIX

The fate of Bell Labs would be a different one; it would go on to be purchased by Lucent, now part of Nokia. After commercial UNIX got separated out to USL, Research UNIX would continue to exist inside of Bell Labs. Research UNIX V8, V9 and V10 were not quite released by Alcatel-Lucent USA Inc. and Nokia in 2017.

However, this is merely a notice that the companies involved will not assert their copyrights only with respect to any non-commercial usage of the code. It is still not possible, over 30 years later, to freely use the V8 code.

Conclusion

In the U.S., Ancient UNIX is freely available. People located everywhere else, however, are unable to legally obtain UNIX code for any of the systems mentioned above. The exception being BSD/OS, assuming a purchase of a legitimate copy of the source code CD. This is deeply unsatisfying and I implore all involved companies to consider open sourcing (preferably under a BSD-style license) their code older than a decade, if nothing else, then at least for the sake of historical purposes. I would like to encourage everybody reading this to consider reaching out to Micro Focus and Wind River Systems about System V and BSD/OS, respectively. Perhaps the masses can change their minds.

A small note about patents: Some technologies used in newer iterations of the UNIX system (in particular the System V derivatives) may be encumbered with software patents. An open source license will not help against patent infringement claims. However, the patents on anything used in the historical operating systems will certainly have expired by now. In addition, European readers can ignore this entirely – software patents just aren’t a thing.


UNIX® is a registered trademark of The Open Group.

Microsoft XENIX 286 BASIC Compiler

(This is a guest post by Antoni Sawicki aka Tenox)

I have recently acquired this artifact:

It’s the Microsoft BASIC compiler for XENIX 286 Operating System. Compiler as opposed to just BASIC interpreter, it can produce executable a.out files, similar to C compiler for example.  

Carefully removed the shrink wrap. Inside were couple of 5.25″ floppies, registration card and a manual:

Interestingly the 32 year old disks read just fine on a first attempt. I need to start backing up important files to 5.25″ floppy disks as they seem to outlast everything else.

Thanks to efforts of Michal Necasek from OS/2 Museum now you can run Microsoft XENIX 286 in Virtual Box

The disks can be installed in to XENIX running on Vbox following a few simple steps:

tar xvf /dev/fd0
./msinstall /dev/fd0

Upon installation you invoke the compiler like this:

bascom demo.bas
./a.out

And it produced an a.out executable which worked perfectly fine.

It’s fun to write BASIC code in vi editor, which I just realized I never done before.

Curiously the compiler also worked on the brand spanking new Xenix 2018, or rather I should call it Open Server 6, which you can download here.

The BASIC compiler is available for download from my archive along with the manual in pdf.

ttyplot – a real time plotting utility for the terminal

(This is a guest post from Antoni Sawicki aka Tenox)

I spend most of time in a day staring at a terminal window often running various performance monitoring tools and reading metrics.

Inspired by tools like gtop, vtop and gotop I wished for a more generic terminal based tool that would visualize data coming from unix pipeline directly on the terminal. For example graph some column or field from sar, iostat, vmstat, snmpget, etc. continuously in real time.

Yes gnuplot and several other utilities can plot on terminal already but none of them easily read data from stdin and plot continuously in real time.

In just couple of evenings ttyplot was born. The utility reads data from stdin and plots it on a terminal with curses. Simple as that. Here is a most trivial example:

To make it happen you take ping command and pipe the output via sed to extract the right column and remove unwanted characters:

ping 8.8.8.8 | sed -u 's/^.*time=//g; s/ ms//g' | ttyplot 

Ttyplot can also read two inputs and plot with two lines, the second being in reverse-video. This is useful when you want to plot in/out or read/write at the same time.

A lot of performance metrics are presented in as a “counter” type which needs to be converted in to a “rate”. Prometheus and Graphana have rate() or irate() function for that. I have added a simple -r option. The time difference is calculated automatically. This is an example using snmpget which is show in screenshot above:

{ while true; do snmpget -v 2c -c public 10.23.73.254 1.3.6.1.2.1.2.2.1.{10,16}.9 | gawk '{ print $NF/1000/1000 }'; sleep 10; done } | ttyplot -2 -r -u "MB/s"

I now find myself plotting all sorts of useful stuff which otherwise would be cumbersome. This includes a lot of metrics from Prometheus for which you normally need a web browser. And how do you plot metrics from Prometheus? With curl:

{ while true; do curl -s http://10.4.7.180:9100/metrics | grep "^node_load1 " | cut -d" " -f2; sleep 1; done } | ttyplot

If you need to plot a lot of different metrics ttyplot fits nicely in to panels in tmux, which also allows the graphs to run for longer time periods.

Of course in text mode the graphs are not very precise, but this is not the intent. I just want to be able to easily spot spikes here and there plus see some trends like up/down – which works exactly as intended.I do dig fancy braille line graphs and colors but this is not my priority at the moment. They may get added later, but most importantly I want the utility to work reliably on most operating systems and terminals. 

You can find compiled binaries here and source code and examples to get you started – here.

If you get to plot something cool that deserves to be listed as an example please send it on!