Proot, defeating security by ignoring it in user-space

From the proot project site :

PRoot is a user-space implementation of chroot, mount –bind, and binfmt_misc.

Android among other Linux systems creates a very restricted user mode where the end user is denied the root user privileges. This is annoying as to ‘root’ a phone can be incredibly complicated l, and beyond a normal user that wants to use their phone for more than some kind of cat video social machine (don’t tell me the incredible popularity of cat videos isn’t toxoplasmosis!).

Many new phone SOC’s are supporting external HDMI, and USB host capabilities allowing you to dock your phone and use it with a keyboard and mouse.

Well thanks to the app, aptly named UserLand, running a light weight Linux distro is just a few screen presses away!

What is cool is that by emulating a scant few system calls it makes the deployment quick and seemingly trivial. And a lot more lightweight compared to docker, User Mode Linux, or Qemu (in full system emulation). But it can invoke qemu to run foreign architecture binaries to give Intel users an Arm UserLand.

Yes, via VNC you can run X11! And yes on my phone it shows all 8 Cores.

Although I’ve been using qemu, UML and other strategies to sidestep restrictive environments, proot proves itself as an exciting new tool!

End of an era, Linux to Deprecate a.out support

From the post on kernel.org:

Linux supports ELF binaries for ~25 years now. a.out coredumping has bitrotten quite significantly and would need some fixing to get it into shape again but considering how even the toolchains cannot create a.out executables in its default configuration, let’s deprecate a.out support and remove it a couple of releases later, instead.

I can’t say I’m all that surprised, maintaining backwards compatibility has not really been a Linux thing, as most people are incapable of doing any troubleshooting in the myriad of hundreds to thousands of independent packages, and instead find it far easier to switch to a different distro entirely.

At the same time, the vast majority of Linux packages are available in source code, so re-building things as ELF most likely has happened in the last 25 years.

During the great ELF migration, it was a gigantic PITA having basically 2 copies of all the libraries as things were converted over, and a.out stuff quickly evaporated. For me, the beauty of a.out was for later kernels to be able to mount and run older stuff. But as we are in the era of both ‘cheap’ user mode kernels along with virtualization will the old executable format truly be missed?

Linux has survived the removal of native support for the 80386, and even the detection logic for the NexGen processors (yes they were real, and yes they did ship), so no doubt this further amputation won’t matter to the vast majority.

I have to wonder how long until the i386 32bit target is removed? Distros like Debian have long since removed support for 80486/80586 classed processors to bring the minimum up to requiring SSE-2 based instructions, and I can’t imagine anyone who is running a 32bit OS for their main OS in this day and era.

diff --git a/arch/x86/um/Kconfig b/arch/x86/um/Kconfig
index f518b47..494eeb5 100644
--- a/arch/x86/um/Kconfig
+++ b/arch/x86/um/Kconfig
@@ -16,7 +16,6 @@ config 64BIT
 
 config X86_32
   def_bool !64BIT
-  select HAVE_AOUT
   select ARCH_WANT_IPC_PARSE_VERSION
   select MODULES_USE_ELF_REL
   select CLONE_BACKWARDS

Farewell a.out

Update: from further along in the changelog

And Alan Cox points out that the a.out binary loader _could_ be done in user space if somebody wants to, but we might keep just the loader in the kernel if somebody really wants it, since the loader isn’t that big and has no really odd special cases like the core dumping does.

So I was building a Windows 2019 server

As I got a ‘totally legit’ serial code in my box of cereal.

After the install I thought it’d be fun to install the Linux Subsystem.

While following the powershell instructions here, I thought the list of quick links of distros to download was interesting:

That’s right ARM Linux userland! I still have high hopes for Windows on ARM (I have 2 Windows RT devices now!!) although I’m not holding my breath.

Maybe there will be some ARM boards that are suitable for the desktop that aren’t over 1k USD.. That’d be nice.

Interesting trivia is that the Linux Subsystem started it’s life on ARM as a way to run Android binaries on Windows Phone. And true to everything Microsoft does, it got to the point where it could start to run things (albeit poorly) and was summarily killed. Although it’s found life despite the original false start as a general ‘text mode’ subsystem for Windows.

However running Linux binaries on Windows currently just shows that NTOS isn’t as efficient as the Linux kernel when it comes to emulating the Linux ABI. Although this was the original ‘dream’ of the microkernel, and a POSIX subsystem for NT was always part of the original design, although it really was more of a checkbox for GSA contracts, and outside of being able to use pax & vi it really was handicapped by not having BSD extensions, and especially by not having any access to the TCP/IP stack.

Oldlinux.org got updated!

I know that on the surface it may not seem like much, but considering the last update was nearly 3 years ago…

2019-02-08   

Time flies, it’s flying fast, and it’s been two years till now. The latest Chinese revision, the Fifth revision of the book:”A Heavily Commemted Linux Kernel Source Code” is now HERE, and, more importantly, the English version of the book is also given HERE. After nearly one year of translation, the English version of this book has finally been completed. Of course, there must be translation errors and tipos in the book. I hope you may point them out for me. Finally, thanks Trent Jarvi who helped me a lot, thank you friends in the Linux communities, and thank you all, and Happy Chinese Spring Festival!

Yes, that’s right the book is now available in English. And in PDF form! It’s an extensive dive into the 0.12 source, complete with diagrams, notes and annotated source code.

It’s a whopper though, 1109 pages, and weighing in at 11 MB! This is not a light read!

As always the site is http://www.oldlinux.org/

ttyplot – a real time plotting utility for the terminal

(This is a guest post from Antoni Sawicki aka Tenox)

I spend most of time in a day staring at a terminal window often running various performance monitoring tools and reading metrics.

Inspired by tools like gtop, vtop and gotop I wished for a more generic terminal based tool that would visualize data coming from unix pipeline directly on the terminal. For example graph some column or field from sar, iostat, vmstat, snmpget, etc. continuously in real time.

Yes gnuplot and several other utilities can plot on terminal already but none of them easily read data from stdin and plot continuously in real time.

In just couple of evenings ttyplot was born. The utility reads data from stdin and plots it on a terminal with curses. Simple as that. Here is a most trivial example:

To make it happen you take ping command and pipe the output via sed to extract the right column and remove unwanted characters:

ping 8.8.8.8 | sed -u 's/^.*time=//g; s/ ms//g' | ttyplot 

Ttyplot can also read two inputs and plot with two lines, the second being in reverse-video. This is useful when you want to plot in/out or read/write at the same time.

A lot of performance metrics are presented in as a “counter” type which needs to be converted in to a “rate”. Prometheus and Graphana have rate() or irate() function for that. I have added a simple -r option. The time difference is calculated automatically. This is an example using snmpget which is show in screenshot above:

{ while true; do snmpget -v 2c -c public 10.23.73.254 1.3.6.1.2.1.2.2.1.{10,16}.9 | gawk '{ print $NF/1000/1000 }'; sleep 10; done } | ttyplot -2 -r -u "MB/s"

I now find myself plotting all sorts of useful stuff which otherwise would be cumbersome. This includes a lot of metrics from Prometheus for which you normally need a web browser. And how do you plot metrics from Prometheus? With curl:

{ while true; do curl -s http://10.4.7.180:9100/metrics | grep "^node_load1 " | cut -d" " -f2; sleep 1; done } | ttyplot

If you need to plot a lot of different metrics ttyplot fits nicely in to panels in tmux, which also allows the graphs to run for longer time periods.

Of course in text mode the graphs are not very precise, but this is not the intent. I just want to be able to easily spot spikes here and there plus see some trends like up/down – which works exactly as intended.I do dig fancy braille line graphs and colors but this is not my priority at the moment. They may get added later, but most importantly I want the utility to work reliably on most operating systems and terminals. 

You can find compiled binaries here and source code and examples to get you started – here.

If you get to plot something cool that deserves to be listed as an example please send it on!

SQL 2017 from 4.21a..

I would never ever recommend this, but…


isqlw connecting to a Linux SQL Server

I didn’t do anything to set this up.  I just searched for ISQLW and for some reason this ancient one showed up in the search path, and it connected.  I didn’t notice it at first until it didn’t like the newer shift insert/delete operations, as back then you needed to use Control C/V .. 

Not being able to stop there, I fired up the admin tool.  It complains that the stored procedure sp_MSAdmin_version  is missing.  However you can go ahead and create it…

create procedure sp_MSAdmin_version as
select “Microsoft SQL Administrator script version 4.20.22.1”
go

And it’ll connect.

Yes you can track stats in sort of real time

Oddly enough things that talk to the server work okay.  Things related to the databases don’t work at all.

SQLServer 2017 on Linux users

I even can admin users from 4.21’s admin program.

I guess the sp_MSAdmin_* scripts could be fixed up for 2017, allowing for a more robust experience, but I really can’t think of any reason why to do it.  I’m more surprised that all the new ODBC drivers since Vista won’t talk to SQL Server 4.21, 6.0, & 7.0, but it seems the client tools can talk to the new server.

I’ve even created the infamous ‘PUBS’ database from the 4.21a script as well.  Again not very useful, but all the more fun!

PUBS

Installation wasn’t too hard, but a little weird to re-produce.  Anyways you’ll need to trust the MS key

wget -qO- https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -

And then I added this into the /etc/apt/sources.list:

deb [arch=amd64] https://packages.microsoft.com/debian/9/prod stretch main
deb [arch=amd64] https://packages.microsoft.com/ubuntu/16.04/mssql-server-2017 xenial main

And then run the following to download MSSQL & the needed bits.  It’ll prompt a few times to agree to the License:

apt-get update;apt-get upgrade
apt-get install apt-transport-https
ACCEPT_EULA=Y apt-get install mssql-tools mssql-server && /opt/mssql/bin/mssql-conf setup

And if everything goes correctly you will then be prompted for the edition to use, the SA password, and then you can start the server with:

systemctl restart mssql-server.service

And away you go.

My output was like this:

# cat /etc/issue
Debian GNU/Linux 9 \n \l

[email protected]:/# apt-get update;apt-get upgrade
Hit:1 http://security.debian.org stretch/updates InRelease
Ign:2 http://debian.uchicago.edu/debian stretch InRelease
Hit:3 http://debian.uchicago.edu/debian stretch Release
Hit:4 https://dl.yarnpkg.com/debian stable InRelease
Hit:5 http://ftp.debian.org/debian stretch-backports InRelease
Hit:7 https://deb.nodesource.com/node_8.x stretch InRelease
Hit:8 https://packages.microsoft.com/debian/9/prod stretch InRelease
Hit:9 https://packages.microsoft.com/ubuntu/16.04/mssql-server-2017 xenial InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
[email protected]:/# apt-get install mssql-tools mssql-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libc++1 libodbc1 libsss-nss-idmap0 libunwind8 msodbcsql17 odbcinst odbcinst1debian2 unixodbc
Suggested packages:
  clang libmyodbc odbc-postgresql tdsodbc unixodbc-bin
The following NEW packages will be installed:
  libc++1 libodbc1 libsss-nss-idmap0 libunwind8 msodbcsql17 mssql-server mssql-tools odbcinst odbcinst1debian2 unixodbc
0 upgraded, 10 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/181 MB of archives.
After this operation, 932 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Preconfiguring packages ...
Selecting previously unselected package libc++1:amd64.
(Reading database ... 53362 files and directories currently installed.)
Preparing to unpack .../0-libc++1_3.5-2_amd64.deb ...
Unpacking libc++1:amd64 (3.5-2) ...
Selecting previously unselected package libodbc1:amd64.
Preparing to unpack .../1-libodbc1_2.3.4-1_amd64.deb ...
Unpacking libodbc1:amd64 (2.3.4-1) ...
Selecting previously unselected package libunwind8.
Preparing to unpack .../2-libunwind8_1.1-4.1_amd64.deb ...
Unpacking libunwind8 (1.1-4.1) ...
Selecting previously unselected package odbcinst1debian2:amd64.
Preparing to unpack .../3-odbcinst1debian2_2.3.4-1_amd64.deb ...
Unpacking odbcinst1debian2:amd64 (2.3.4-1) ...
Selecting previously unselected package odbcinst.
Preparing to unpack .../4-odbcinst_2.3.4-1_amd64.deb ...
Unpacking odbcinst (2.3.4-1) ...
Selecting previously unselected package unixodbc.
Preparing to unpack .../5-unixodbc_2.3.4-1_amd64.deb ...
Unpacking unixodbc (2.3.4-1) ...
Selecting previously unselected package libsss-nss-idmap0.
Preparing to unpack .../6-libsss-nss-idmap0_1.15.0-3_amd64.deb ...
Unpacking libsss-nss-idmap0 (1.15.0-3) ...
Selecting previously unselected package msodbcsql17.
Preparing to unpack .../7-msodbcsql17_17.2.0.1-1_amd64.deb ...
Unpacking msodbcsql17 (17.2.0.1-1) ...
Selecting previously unselected package mssql-server.
Preparing to unpack .../8-mssql-server_14.0.3037.1-2_amd64.deb ...
Unpacking mssql-server (14.0.3037.1-2) ...
Selecting previously unselected package mssql-tools.
Preparing to unpack .../9-mssql-tools_17.2.0.1-1_amd64.deb ...
Unpacking mssql-tools (17.2.0.1-1) ...
Setting up libsss-nss-idmap0 (1.15.0-3) ...
Setting up libodbc1:amd64 (2.3.4-1) ...
Setting up libunwind8 (1.1-4.1) ...
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Processing triggers for man-db (2.7.6.1-2) ...
Setting up libc++1:amd64 (3.5-2) ...
Setting up mssql-server (14.0.3037.1-2) ...
Setting up odbcinst1debian2:amd64 (2.3.4-1) ...
Setting up odbcinst (2.3.4-1) ...
Setting up unixodbc (2.3.4-1) ...
Setting up msodbcsql17 (17.2.0.1-1) ...
Setting up mssql-tools (17.2.0.1-1) ...
Processing triggers for libc-bin (2.24-11+deb9u3) ...
[email protected]:/# /opt/mssql/bin/mssql-conf setup
Choose an edition of SQL Server:
  1) Evaluation (free, no production use rights, 180-day limit)
  2) Developer (free, no production use rights)
  3) Express (free)
  4) Web (PAID)
  5) Standard (PAID)
  6) Enterprise (PAID)
  7) Enterprise Core (PAID)
  8) I bought a license through a retail sales channel and have a product key to enter.

Details about editions can be found at
https://go.microsoft.com/fwlink/?LinkId=852748&clcid=0x409

Use of PAID editions of this software requires separate licensing through a
Microsoft Volume Licensing program.
By choosing a PAID edition, you are verifying that you have the appropriate
number of licenses in place to install and run this software.

Enter your edition(1-8): 2
The license terms for this product can be found in
/usr/share/doc/mssql-server or downloaded from:
https://go.microsoft.com/fwlink/?LinkId=855862&clcid=0x409

The privacy statement can be viewed at:
https://go.microsoft.com/fwlink/?LinkId=853010&clcid=0x409

Do you accept the license terms? [Yes/No]:yes

Enter the SQL Server system administrator password:
Confirm the SQL Server system administrator password:
Configuring SQL Server...

ForceFlush is enabled for this instance.
ForceFlush feature is enabled for log durability.
Created symlink /etc/systemd/system/multi-user.target.wants/mssql-server.service → /lib/systemd/system/mssql-server.service.

Additionally you may not want to listen on every single IP address, but rather only on the loopback.  So you would run this to configure the listening address:

/opt/mssql/bin/mssql-conf  set network.ipaddress  127.0.0.1

I also use the SQL Agent, to enable that just simply run this:

/opt/mssql/bin/mssql-conf set sqlagent.enabled true 
systemctl restart mssql-server

Many more settings for the /var/opt/mssql/mssql.conf file can be found here: https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-configure-mssql-conf?view=sql-server-2017.  I would take a look at them, and possible enable stuff like TLS so that someone with management tools circa 1993 can’t just login to your server.  Then again maybe that is the kind of thing you want.

And if you don’t want Microsoft SQL Server, just do the following to uninstall MSSQL, destroying all data as well.

apt-get purge  mssql-tools mssql-server msodbcsql17
apt-get auto-remove
rm -rf /var/opt/mssql

I kept on getting this error which I didn’t see any way to cleanly resolve to fix for running MSSQL on Debian.  The best hint is the OpenSSL is either too new (unlikely) or too old (far too likely).  Instead I just changed distros as that is what people do, they don’t troubleshoot problems in Linux, just change distros so why bother fighting it?

# /opt/mssql-tools/bin/sqlcmd -Usa -PMYPa55w0rd!# -S127.0.0.1
Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : TCP Provider: Error code 0x2746.
Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Client unable to establish connection.
OpenSSL?

Going further though, as much as I liked Debian it really does run better on Ubuntu.  So as an addendum, use these sources (at the moment!).  Since the SQL Agent wouldn’t run, and I couldn’t connect locally it was worse than useless.

deb [arch=amd64] https://packages.microsoft.com/ubuntu/16.04//prod xenial main
deb [arch=amd64] https://packages.microsoft.com/ubuntu/16.04/mssql-server-2017 xenial main

Now the first time I tried to do anything on Ubuntu I got this lovely error:

# /opt/mssql-tools/bin/sqlcmd
terminate called after throwing an instance of 'std::runtime_error'
  what():  locale::facet::_S_create_c_locale name not valid

And it just hung the process.  I had to control-Z & kill -9 %1 it to get it out of the way.  Well it turns out that this VM didn’t have it’s locale set.  Fixing that was pretty simple, once you know how:

apt-get install locales && dpkg-reconfigure locales

Another thing that really bugs me is the lack of cryptography by default. So I found this nice recipie to for setting it up quickly.  Just watch your hostname!

systemctl stop mssql-server 
cat /var/opt/mssql/mssql.conf 
mkdir /var/opt/mssql/ssl
mkdir /var/opt/mssql/ssl/certs/
mkdir /var/opt/mssql/ssl/private/
cd /var/opt/mssql/
chown -R mssql:mssql *
cd
openssl req -x509 -nodes -newkey rsa:2048 -subj '/CN=HOSTNAME' -keyout mssql.key -out mssql.pem -days 3650
chown mssql:mssql mssql.pem mssql.key 
chmod 600 mssql.pem mssql.key
mv mssql.pem /var/opt/mssql/ssl/certs/
mv mssql.key /var/opt/mssql/ssl/private/
/opt/mssql/bin/mssql-conf set network.tlscert /var/opt/mssql/ssl/certs/mssql.pem 
/opt/mssql/bin/mssql-conf set network.tlskey /var/opt/mssql/ssl/private/mssql.key

This will build out a self signed certificate for 10 years and put them into the local MSSQL directory where it can read them.

Compiling rsync for VMWare ESXi 6.5

So what had started this little ‘adventure’ is that years ago there was this great sale over at Joe’s Datacenter, where I had picked up a physial server for the mere price of $20 USD a month!  What a deal!  No more quotas, CPU sharing issues and generally having to share.  Awesome!

So I have them install Debian, and load up the KVM modules, and away I go and life is good.  So foolishly years later, I jumped onto the whole container thing, installed docker, and that is where everything went south.

Every few seconds while doing a tcpdump on the 100% virtual bridge I’d see a massive influx of arp traffic.  I tried static arp’s on the host & the guest and it was ‘better’ but now the network traffic would hang.  Things like TCP would break after a minute and stuff like UDP game state would break so bad it’d end up unplayable.  This basically broke maraakate.org and hosting stuff like Quake I/ Quake II/ Daikatana and other iD based games.

My existing virtual machines now had a major issue where they would stop responding to traffic.  I never could find a fix, and it ended up with me moving my blog out to sloppy.io to keep running as a container based service until it magically stopped working and I gave up and did a full dump/reload on a hosted WordPress over at ChicagoVPS.  What a nightmare!

But what about all those virtual machines?

Well even after apt-get purge of everything docker, upgrading and downgrading the kernel nothing helped.  The VMs still dropped off the network periodically.  So with some spare time I decided to just go ahead and backup the box, and wipe the machine.

Since the physical network was working fine I was able to rsync the 300GB+ worth of data over the span of a few days.  That was fine, and considering how crap the server had been, I figured some straight downtime wouldn’t hurt anything too much.  While looking for an OS to install onto the machine, I saw that Joe offers VMWare ESXi 6.5, so I thought I’d just go with that, as naturally VMWare runs both VMs and with Project Photon I could maybe mess with containers again at a later date.

Since rsync had worked so well for moving hundreds of gigabytes of data from the USA to Hong Kong, I figured it’d be trivial to just convert the existing RAW KVM/Qemu disk images back to the United States of America.  And that is where the fun begins.

Let me tell you!

While reading this great post on virten.net they drop mention of XSIBackup, which requires registration (yuck) and worse their stupid registration system is broken:

LOL WHUT?

Rest assured the email does show up!

Dear Neo Zeed, thank you for visiting 33hops.com
This is your download url http://a.33hops.com/downloads/?key=bq7l5ptPB70MJj9dkftxFegr3xWoBZwpdFPQOUC3Cm10KPSIl3v1532877224253

But of course it doesn’t work

The key is invalid or has expired, only two downloads are allowed per key, download a new one at 33hops.com

What

The

Fuck

I know this is an ongoing issue at large when you provide executable binaries on the internet, as you will no doubt get flagged with some false positive by some virus crusaiding idiot who just sets up a tool and never reads anything but sends out threatening emails.  I went through this with the need for the simple 404 redirect, all thanks to Gerhard W. Recher.

So since this wasn’t going to be an avenue to persue I dug a little deeper and ran across this post over at virtuallyGhetto.  So it turns out the userland for ESXi is a CentOS environment that uses busybox.  And if you just download and install CentOS 3.9 into a VM, and build whatever you want, and ideally add in the -static flag, and copy it over.  For those who want to look into more ‘internals’ of the userland, check out zemris.fer.hr.

Great!

Things like UID/GUID mappings are broken in the libc it seems among other things.  So for my simple rsync config I just put the numbers in myself.

uid = 0
gid = 0
use chroot = no
max connections = 4
syslog facility = local5
pid file = /var/run/rsyncd.pid
hosts allow = a.b.c.d

[datastore1]
path = /vmfs/volumes/datastore1
comment = WDC_WD5000AAJS2D00A8B0
read only = no

I have read that you really ought to keep your binaries/config on the datastore so they aren’t subject to upgrades overwriting them and other chaotic stuff.  So editing the file “/etc/rc.local.d/local.sh” I just added the following before the exit 0:

/vmfs/volumes/datastore1/tools/rsync –daemon –log-file=/tmp/rsync.log –config=/vmfs/volumes/datastore1/tools/rsyncd.conf

And then ran it manually to kick it off.

So now I don’t have to rely on someone’s elses broken downoad system, and now we can build other fun native stuff.

And the best part is that after all of this fighting Maraakate’s site is back online and I get this message from him:

holy crap that new server setup so much better
its like playing it locally honest to god
played a whole unit not a single fuck up
no rubber banding lag effect or any of that

So now things are actually performing better on VMWare than we were getting on KVM.  And yes I had flattened out the disk images, loaded up the paravirtual disk & network drives on KVM, but VMWare does such a surprisingly better job.

I honestly wasn’t expecting that.

And as a bonus, I messed with qemu-0.9.0 (I didn’t feel the need to go through glibc2 hell), and qemu-img works great with a simple raw to vmdk

[[email protected]:/vmfs/volumes/5b5806fd-339444da-f897-003048d70598] ./tools/qemu-i
mg.static info win30.raw
image: win30.raw
file format: raw
virtual size: 32M (33554432 bytes)
disk size: 32M
[[email protected]:/vmfs/volumes/5b5806fd-339444da-f897-003048d70598] ./tools/qemu-i
mg.static convert -f raw -O vmdk win30.raw win30.vmdk
[[email protected]:/vmfs/volumes/5b5806fd-339444da-f897-003048d70598] ./tools/qemu-i
mg.static info win30.vmdk
image: win30.vmdk
file format: vmdk
virtual size: 32M (33554432 bytes)
disk size: 27M

And it boots!

Transcopied Windows 3.0 VM

So yes, wrapping up you can in fact run stuff on ESXi, copy data, and even convert disk images.

Oh yeah, and so people can deal with my 404 based download system (the password is on the 404 page)

Let the games begin!

Qemu now supports the HPPA in softmmu mode!

I’ve worked on machines with HP-UX, but never owned one.  Well Qemu now has system emulation thanks to Richard Henderson! You can find information over at:

https://parisc.wiki.kernel.org/index.php/Qemu

Being the unfair person I am, I thought I’d try NeXTSTEP to see how far it gets.

Processor   Speed            State           Coprocessor State  Cache Size
---------  --------   ---------------------  -----------------  ----------
0      250 MHz    Active                 Functional            0 KB

Available memory: 512 MB
Good memory required: 16 MB

Primary boot path: FWSCSI.6.0
Alternate boot path: LAN.0.0.0.0.0.0
Console path: SERIAL_1.9600.8.none
Keyboard path: PS2

Available boot devices:
1. DVD/CD [lsi 00:00.0 2:0 Drive QEMU QEMU CD-ROM 2.5+]
2. lsi 00:00.0 0:0 Drive QEMU QEMU HARDDISK 2.5+

Booting from lsi 00:00.0 0:0 Drive QEMU QEMU HARDDISK 2.5+

Booting...
Boot IO Dependent Code (IODC) revision 153

HARD Booted.
Can't determine I/O subsystem type

NEXTSTEP boot v3.3.4.17
524288 memory

NEXTSTEP will start up in 10 seconds, or you can:
Type -v and press Return to start up NEXTSTEP with diagnostic messages
Type ? and press Return to learn about advanced startup options
Type any other character to stop NEXTSTEP from starting up automatically

boot:

And amazingly the bootloader works, although that is about it. Trying to boot up OpenBSD gets about this far:
PDC_CHASSIS: Initialize (3), CHASSIS  cec0
>> OpenBSD/hppa CDBOOT 0.2
booting dk0a:/bsd.rd: 2703360+851960+2675712+547840=0x8631f0

SeaBIOS: Unimplemented PDC_CACHE function 1 8ddad0 1 1 1

I found on Windows though that the Debian 8 CD’s work the best, as the earlier ones lock up after loading a kernel, and the later one doesn’t fully initialize.  I’ve been using this one: debian-8.0-hppa-NETINST-1.iso  Serial console interaction is the way to go, so I ran Qemu like this:

qemu-system-hppa.exe -L . -serial telnet:127.0.0.1:4444,server,wait -boot d -cdrom debian-8.0-hppa-NETINST-1.iso -hda Deiban8HPPA.vmdk

So this way I can get get the install kicked off.  Although I should probably have just downloaded debian-8.0-hppa-CD-1.iso

Linux on HPPA

Otherwise, yeah, it’s Linux, on HPPA

And for anyone who is interested, the only version of HP-UX I have hanging around, HP-UX 10.20 [HP9000 S700] gives me the following:

HP-UX 10.20 on Qemu

IRC necromancy

I’m xorhash, a guest poster, here to talk about my tale going down a trip on the memory lane with QuakeNet’s service bot Q. If you’re not interested in IRC, you can probably skip this one.

On the Trails of Q

As far as I know, QuakeNet’s service bot Q went through these three major codebases:

a. the old Perl Q,
b. the first version written in C, and
c. Q as part of newserv.

There’s a reason I didn’t have anything to link for (a). That’s because to the best of my knowledge and research, no version has survived these past decades.

As for (b), it seems only the linked version 3.99 from the year 2003 was saved.
The CVS repository and thus commit history has been lost.

If anyone has either actual code for the old Perl Q or the CVS repo for the old
Q written in C, please reach out to me via `xorhash ++at++ protonmail.com’.
I’m most interested in looking through it.

However, not all hope was lost with the old Perl Q. As it turns out, most likely, the old Perl Q was actually based on an off-the-shelf product called “CServe”. What makes me think so?

Let’s take a look at [the QuakeNet Q command listing from 1998.

I picked the command “WHOIS” and googled its use “Will calculate a [email protected] mask for you from the whois information of this nick.” This lead me to a help file for StarLink IRC. At the top, it reads:

CStar3.x User Command Help File **** 09/10/99
Information extracted from CServe Channel Service
Version 2.0 and up Copyright (c)1997 by Michael Dabrowski
Help Text (c)1997 (c)1997 StarLink-IRC (with permission)

Wait a second, “CServe Channel Service”? I know that from somewhere.

[email protected]

So the commands between that help file and the QuakeNet Q command listing match up and so does Q’s host today. Most likely, I’m on the right track with this. What’s left is to track down a copy of CServe.

Note: I’ve been on the old Perl Q for a while and this strategy didn’t use to work. It seems Google newly indexed these pages. For once I can sincerely say: Thank you, Google.

I found that CServe was hosted on these websites:

a. Version 3.0 on http://www.cs.cuc.edu/~mdabrows/cserve/,
b. Version 3.1 on http://www.wam.umd.edu/~devy/cserve/,
c. Version 4.0 on http://www.othernet.org/devon/cserve/, and
d. Version 5.0 and above on http://www.ircore.com/.

The only surviving versions are 3.0 and 5.1. CServe got renamed to “CS” starting with 5.0 and was rewritten in C by someone other than the original CServe author, going by the comments in the file header of CS5.1 `src/show_access.c’. CS was actually sold as a commercial product. I wonder how many people bought it.

QuakeNet most likely took a version between 2.0 and 4.0, inclusive, as the basis for the old Perl Q. Which one in particular it was, we may never know. If you have any details, please reach out to me at the e-mail address above.

I can’t make any clever guesses anyway since the only versions that the web archive has are 3.0 and 5.1. The latter is written in C, so it quite obviously can’t be the old Perl Q.

Making It Run

So now that I have CServe 3.0, I wanted to actually see it running.

There are three ways to reasonably accomplish this:

a. port CServe to a modern IRCd’s server-to-server protocol,
b. port an old IRCd to a modern platform,
c. emulate an old platform and run both IRCd and CServe there.

I chose option (b).
Once upon a time, I did option (a) for the old UnderNet X bot. It was a very painful exercise to port a bot that predates the concept of UIDs (or numeric nicks/numnicks as ircu’s P10 server-to-server protocol calls them). There’s nothing too exciting about doing (c) by just emulating a 486 or so and FreeBSD, just sounds like a boring roundtrip of emulation and network bridging.

Fortunately, the author was a nice person and wrote on the CServe website that version 3.0 requires “ircu2.9.32 and above”.

It seems the ircu2.10 series followed right after ircu2.9.32. While I’m sure there’s some linking backwards compatibility, determining which ircu in the ircu2.10 series still spoke enough P09 to link with CServe sounded like an exercise in boring excruciating pain. Modern-day ircu most certainly no longer speaks P09. Besides, what’s the fun in just doing the manual equivalent of `git bisect’?

So after grabbing ircu2.9.32, I tried to just straightforward compile and run it.

There’s a `Config’ script that’s supposed to be kind of like autoconf `configure’, but I’ve found it extremely non-deterministic. It generates `include/setup.h’. I’ve made a diff for your convenience. It targets Debian stable, and should work with any reasonably modern Linux. There are special `#ifdef’ branches for  FreeBSD/NetBSD in the code. This patchset may break for BSDs in general.

Do not touch `Config’, meddle with `include/setup.h’ manually. Remember this is an ancient IRCd, there are actual tunables in `include/config.h’.

The included example configuration file is correct for the most part, but the documentation on U:lines is wrong. U:lines do what modern-day U:lines do, i.e., designate services servers with uber privileges.

U:cserve.mynetwork.example:*:*

Excuse Me, But What The Fuck?

Of course, I’m dealing with old code. It wouldn’t be old code if I didn’t have some things that just make me go “Excuse me, but what the fuck?”

Looping at the speed of light

aClient *find_match_server(mask)
char *mask;
{
  aClient *acptr;
  if (BadPtr(mask))
    return NULL;
  for (acptr = client, (void)collapse(mask); acptr; acptr = acptr->next) 
  {
  if (!IsServer(acptr) && !IsMe(acptr))
    continue;
    if (!match(mask, acptr->name))
      break;                                                                                                    continue;
  }
  return acptr;
}

See that `continue’ way on the left? What is it doing there? Telling the compiler to loop faster?

Carol of the Old Varargs

So apparently some of this code predates C89. Which means it uses old-style declarations, but that’s okay. It also uses old-style varargs, which is adorable.

The hacks around not even that being there are adorable, too:

#ifndefUSE_VARARGS
/*VARARGS*/
voidsendto_realops(pattern, p1, p2, p3, p4, p5, p6, p7)
char*pattern, *p1, *p2, *p3, *p4, *p5, *p6, *p7;
{
#else
voidsendto_realops(pattern, va_alist)
char*pattern;
va_dcl
{
  va_list vl;
#endif
  Reg1 aClient *cptr;
  Reg2 int i;
  char fmt[1024];
  Reg3 char *fmt_target;

#ifdef USE_VARARGS
  va_start(vl);
#endif

  (void)sprintf(fmt, ":%s NOTICE ", me.name);
  fmt_target = &fmt[strlen(fmt)];

  for (i = 0; i <= highest_fd; i++)
if ((cptr = local[i]) && IsOper(cptr))
  {
  strcpy(fmt_target, cptr->name);
  strcat(fmt_target, " :*** Notice -- ");
  strcat(fmt_target, pattern);
  #ifdef USE_VARARGS
  vsendto_one(cptr, fmt, vl);
  #else
  sendto_one(cptr, fmt, p1, p2, p3, p4, p5, p6, p7);
  #endif
  }
#ifdef USE_VARARGS
va_end(vl);
#endif
return;
}

These functions were declared like this (the example chosen above actually has
no declaration because why not):
/*VARARGS1*/
extern    void    sendto_ops();

Whatcmp

There are `mycmp’ and `myncmp’ for doing RFC1459 casemapping string comparisons. `strcasecmp’ got `#define’d to `mycmp’, but in one case `mycmp’ got `#define’d back to `strcasecmp’. It seemed easier to just remove `mycmp’, replacing it with `strcasecmp’ and forgo RFC1459 casemapping. This is doubly useful because CServe doesn’t actually honor RFC1459 casemapping.

Waiting for the Cookie

ircu uses PING cookies. I was rather confused when I didn’t get one immediately after sending `NICK’ and `USER’. In fact, it took so long that I thought the IRCd got stuck in a deadloop somewhere. That would’ve been a disaster since the last thing I wanted to do is get up close and personal with the networking stack.

As it turns out, it can’t send the cookie:

/*
 * Nasty.  Cant allow any other reads from client fd while we're
 * waiting on the authfd to return a full valid string.  Use the
 * client's input buffer to buffer the authd reply.
 * Oh. this is needed because an authd reply may come back in more
 * than 1 read! -avalon
 */


Nasty indeed.

I lowered `CONNECTTIMEOUT’ to 10 in the diff linked above. This makes the wait noticeably shorter when you aren’t running an identd.

CServe Isn’t Much Better

Not that CServe is much better. I have to hand it to Perl, I only needed to undo the triple-`undef’ on line 450 of `cserve.pl’ and it worked with no modifications. God bless the backwards compatibility of Perl 5.

That said, it has its own interesting ideas of code. This is the main command execution:

foreach $i (keys %commands)
{
    if($com eq $i)
    { $found = 1; break; }
}
if($found == 1)
{
    open(COMMAND, "<./include/$com");
    @evalstring = <COMMAND>; close(COMMAND);
    foreach $i (@evalstring) { $evals .= $i; }
    eval($evals);
}
else
{
    &notice("2No such command 1[4$com1]. /msg $unick SHOWCOMMANDS\n");
}

Yep, it opens, reads into an array, closes and then evals. For every command it recognizes. Of course, this means code hot swapping, but it also means terrible performance with any non-trivial amount of users.

Oh, and all passwords are hashed. But they’re hashed with `crypt()’. And a never-changing salt of ZZ.

End Result

up & running

Was it worth it?
No, not really.
Would I do it again?
Absolutely.

You probably do not want to expose this to the outside world.
The IRCd code is scary in all the wrong ways.

Further Links

Some other things if you’re into ancient IRC stuff: