Slackware finally goes to Patreon

It’s been a long time overdue, where helping to keep afloat the oldest continues Linux distro is now super easy to do!

Hi folks, Slackware founder Patrick Volkerding here. Welcome to the official page supporting the Slackware Linux Project. Slackware was founded in 1993 and made its first public releases that same year, making it the longest active Linux distribution project.

Today there are many Linux distributions available, but I’ve remained dedicated to this project as I believe it still holds an important place in the Linux ecosystem. Slackware is free, open source software that gives you everything you need to recompile the entire operating system and make any changes you like. The package system is constructed from shell scripts using standard utilities, and easily modified scripts are used throughout so that users may customize their operating system without having to recompile anything. Included software is changed very little (usually not at all) from the way the upstream developers intended. Einstein once said, “Everything should be made as simple as possible, but not simpler” — and when it comes to an operating system, trying to make things “easy” does not always achieve that goal. Anyone who has ever struggled with a package dependency system that doesn’t like the replacement of a system component knows what I’m talking about here. Slackware users know that once you’ve learned how to use the system, it’s easier to use than something that tries to put a GUI setup tool in front of the underlying text files that actually configure the system. And they know that Slackware can be trusted not to constantly change the way things work, so that your investment in learning Slackware lasts longer than it would with a system that’s a moving target. Slackware still builds upon a foundation of the latest development tools, libraries, and applications, and is supported by a vibrant user/developer community that welcomes new users and provides well-tested build scripts for nearly any software you may require.

Your support is greatly appreciated, and will make it possible for me to continue to maintain this project. Cheers!

For me, Slackware holds a special place as just as soon as SLS became a thing it quickly fell into disrepair and rot. The final straw was trying to bootstrap that new fangled ELF thing (it’s almost impossible to find any talk of that COFF thing) and it was just an incredible amount of work. At that point just getting Slackware on a shovelware CD-ROM set was just nothing short of incredible. And it was obvious at the time of ELF that the whole ‘I can’t xyz with little to no effort, therefore dump that distro and try another one’ was born. And Linux never recovered. Look no further at the insane cruft and required infrastructure for Docker to just load an application.

With that said when building a lot of crap from source it’s sometimes nice to have something that’ll just get out of the way, and Slackware does a reasonable job of that. Although last time I really fought with Slackware 13.37 it really didn’t go so well. But at the same time I am willing to make monetary support to ensure the future of Slackware.

The project currently stands at 230 supporters. Which is pretty good, as when it comes to supporting things, few will actually put their money into it, so hopefully this will allow Slackware to continue to exist, and maybe even flourish. I for one find this anti systemd distro important for the sake of software diversity in the ‘me too Debian’ sea of clones out there. Just as supporting OpenBSD is important in the standpoint of security.

You too can support Patrick over on patreon.com.

Not having fun with Python/Debian 9.1

Well after my last Star Wars Galaxies adventure, where I tried to run MySQL on Linux Subsystem for Windows v1, I got some weird shared memory error, and it wouldn’t run. Even the old BSDDB engine was bombing out trying to create files. So fine, whatever I thought I could move on, and that is when I found out that somehow OpenSSL & Python had utterly collided.

Python 1.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import ssl Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/ssl.py", line 98, in import _ssl # if we can't import it, let the error propagate ImportError: /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1: version `OPENSSL_1_1_0' not found (required by /usr/lib/python2.7/lib-dynload/_ssl.x86_64-linux-gnu.so) >>>

Well isn’t that great. I tried un-installing & re-installing Python over and over, along with trying to force re-install OpenSSL. No dice.

So what finally got it working for me was to purge OpenSSL.

apt-get purge libssl1.1

And after that it pulled out everything that was using it, well over 500MB of stuff I’d installed. And for good measure I followed up with the autoremove for an additional 384MB of stuff to remove. And then for the final step, of just installing Python:

[email protected]:~# apt-get install python2.7 Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: libpython2.7-stdlib libssl1.1 Suggested packages: python2.7-doc The following NEW packages will be installed: libpython2.7-stdlib libssl1.1 python2.7 0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded. Need to get 3,537 kB of archives. After this operation, 12.8 MB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://deb.debian.org/debian stretch/main amd64 libssl1.1 amd64 1.1.0j-1~deb9u1 [1,354 kB] Get:2 http://deb.debian.org/debian stretch/main amd64 libpython2.7-stdlib amd64 2.7.13-2+deb9u3 [1,897 kB] Get:3 http://deb.debian.org/debian stretch/main amd64 python2.7 amd64 2.7.13-2+deb9u3 [285 kB] Fetched 3,537 kB in 0s (13.2 MB/s) Preconfiguring packages ... Selecting previously unselected package libssl1.1:amd64. (Reading database ... 27441 files and directories currently installed.) Preparing to unpack .../libssl1.1_1.1.0j-1~deb9u1_amd64.deb ... Unpacking libssl1.1:amd64 (1.1.0j-1~deb9u1) ... Selecting previously unselected package libpython2.7-stdlib:amd64. Preparing to unpack .../libpython2.7-stdlib_2.7.13-2+deb9u3_amd64.deb ... Unpacking libpython2.7-stdlib:amd64 (2.7.13-2+deb9u3) ... Selecting previously unselected package python2.7. Preparing to unpack .../python2.7_2.7.13-2+deb9u3_amd64.deb ... Unpacking python2.7 (2.7.13-2+deb9u3) ... Processing triggers for mime-support (3.60) ... Processing triggers for libc-bin (2.24-11+deb9u4) ... Setting up libssl1.1:amd64 (1.1.0j-1~deb9u1) ... Processing triggers for man-db (2.7.6.1-2) ... Setting up libpython2.7-stdlib:amd64 (2.7.13-2+deb9u3) ... Setting up python2.7 (2.7.13-2+deb9u3) ... Processing triggers for libc-bin (2.24-11+deb9u4) ...

So now you think its going to be broken right? It’s the same libssl package! I didn’t even run an ‘apt-get update’. And guess what?! You would be wrong.

[email protected]:~# python Python 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import ssl >>>

That’s right, it’s now working.

Speaking of Picard, I hear there will be a new series, inspiringly named ‘Picard’. Apparently it’s going down the same path as STD, complete with a lack of backers, and merch. I’m hoping it’ll be something watchable, although they certainly cannot afford any of the TNG cast as they apparently have zero budget. Maybe it’ll end up as another ‘Jake Skywalker’ or ‘Not my Picard’. But the bad reboot 25% contractual difference could be the difference between something new, or something so divergent that there was no point in even attempting to be a Trek.

Proot, defeating security by ignoring it in user-space

From the proot project site :

PRoot is a user-space implementation of chroot, mount –bind, and binfmt_misc.

Android among other Linux systems creates a very restricted user mode where the end user is denied the root user privileges. This is annoying as to ‘root’ a phone can be incredibly complicated l, and beyond a normal user that wants to use their phone for more than some kind of cat video social machine (don’t tell me the incredible popularity of cat videos isn’t toxoplasmosis!).

Many new phone SOC’s are supporting external HDMI, and USB host capabilities allowing you to dock your phone and use it with a keyboard and mouse.

Well thanks to the app, aptly named UserLand, running a light weight Linux distro is just a few screen presses away!

What is cool is that by emulating a scant few system calls it makes the deployment quick and seemingly trivial. And a lot more lightweight compared to docker, User Mode Linux, or Qemu (in full system emulation). But it can invoke qemu to run foreign architecture binaries to give Intel users an Arm UserLand.

Yes, via VNC you can run X11! And yes on my phone it shows all 8 Cores.

Although I’ve been using qemu, UML and other strategies to sidestep restrictive environments, proot proves itself as an exciting new tool!

End of an era, Linux to Deprecate a.out support

From the post on kernel.org:

Linux supports ELF binaries for ~25 years now. a.out coredumping has bitrotten quite significantly and would need some fixing to get it into shape again but considering how even the toolchains cannot create a.out executables in its default configuration, let’s deprecate a.out support and remove it a couple of releases later, instead.

I can’t say I’m all that surprised, maintaining backwards compatibility has not really been a Linux thing, as most people are incapable of doing any troubleshooting in the myriad of hundreds to thousands of independent packages, and instead find it far easier to switch to a different distro entirely.

At the same time, the vast majority of Linux packages are available in source code, so re-building things as ELF most likely has happened in the last 25 years.

During the great ELF migration, it was a gigantic PITA having basically 2 copies of all the libraries as things were converted over, and a.out stuff quickly evaporated. For me, the beauty of a.out was for later kernels to be able to mount and run older stuff. But as we are in the era of both ‘cheap’ user mode kernels along with virtualization will the old executable format truly be missed?

Linux has survived the removal of native support for the 80386, and even the detection logic for the NexGen processors (yes they were real, and yes they did ship), so no doubt this further amputation won’t matter to the vast majority.

I have to wonder how long until the i386 32bit target is removed? Distros like Debian have long since removed support for 80486/80586 classed processors to bring the minimum up to requiring SSE-2 based instructions, and I can’t imagine anyone who is running a 32bit OS for their main OS in this day and era.

diff --git a/arch/x86/um/Kconfig b/arch/x86/um/Kconfig index f518b47..494eeb5 100644 --- a/arch/x86/um/Kconfig +++ b/arch/x86/um/Kconfig @@ -16,7 +16,6 @@ config 64BIT config X86_32 def_bool !64BIT - select HAVE_AOUT select ARCH_WANT_IPC_PARSE_VERSION select MODULES_USE_ELF_REL select CLONE_BACKWARDS

Farewell a.out

Update: from further along in the changelog

And Alan Cox points out that the a.out binary loader _could_ be done in user space if somebody wants to, but we might keep just the loader in the kernel if somebody really wants it, since the loader isn’t that big and has no really odd special cases like the core dumping does.

Linux hits version 5.0!

YES OMG I can’t take it anymore.

So many emails/blogs/videos/messages/tweets/whatevers!

I get it, ok ok, Linus thought 4.21 seemed silly so went straight to 5.0. Contrary to the numbering it’s not a significant divergence.

There.

I said it.

Am I the only one being hit with 50,000 things about Linux 5.0?!

So I was building a Windows 2019 server

As I got a ‘totally legit’ serial code in my box of cereal.

After the install I thought it’d be fun to install the Linux Subsystem.

While following the powershell instructions here, I thought the list of quick links of distros to download was interesting:

That’s right ARM Linux userland! I still have high hopes for Windows on ARM (I have 2 Windows RT devices now!!) although I’m not holding my breath.

Maybe there will be some ARM boards that are suitable for the desktop that aren’t over 1k USD.. That’d be nice.

Interesting trivia is that the Linux Subsystem started it’s life on ARM as a way to run Android binaries on Windows Phone. And true to everything Microsoft does, it got to the point where it could start to run things (albeit poorly) and was summarily killed. Although it’s found life despite the original false start as a general ‘text mode’ subsystem for Windows.

However running Linux binaries on Windows currently just shows that NTOS isn’t as efficient as the Linux kernel when it comes to emulating the Linux ABI. Although this was the original ‘dream’ of the microkernel, and a POSIX subsystem for NT was always part of the original design, although it really was more of a checkbox for GSA contracts, and outside of being able to use pax & vi it really was handicapped by not having BSD extensions, and especially by not having any access to the TCP/IP stack.

Oldlinux.org got updated!

I know that on the surface it may not seem like much, but considering the last update was nearly 3 years ago…

2019-02-08   

Time flies, it’s flying fast, and it’s been two years till now. The latest Chinese revision, the Fifth revision of the book:”A Heavily Commemted Linux Kernel Source Code” is now HERE, and, more importantly, the English version of the book is also given HERE. After nearly one year of translation, the English version of this book has finally been completed. Of course, there must be translation errors and tipos in the book. I hope you may point them out for me. Finally, thanks Trent Jarvi who helped me a lot, thank you friends in the Linux communities, and thank you all, and Happy Chinese Spring Festival!

Yes, that’s right the book is now available in English. And in PDF form! It’s an extensive dive into the 0.12 source, complete with diagrams, notes and annotated source code.

It’s a whopper though, 1109 pages, and weighing in at 11 MB! This is not a light read!

As always the site is http://www.oldlinux.org/

ttyplot – a real time plotting utility for the terminal

(This is a guest post from Antoni Sawicki aka Tenox)

I spend most of time in a day staring at a terminal window often running various performance monitoring tools and reading metrics.

Inspired by tools like gtop, vtop and gotop I wished for a more generic terminal based tool that would visualize data coming from unix pipeline directly on the terminal. For example graph some column or field from sar, iostat, vmstat, snmpget, etc. continuously in real time.

Yes gnuplot and several other utilities can plot on terminal already but none of them easily read data from stdin and plot continuously in real time.

In just couple of evenings ttyplot was born. The utility reads data from stdin and plots it on a terminal with curses. Simple as that. Here is a most trivial example:

To make it happen you take ping command and pipe the output via sed to extract the right column and remove unwanted characters:

ping 8.8.8.8 | sed -u 's/^.*time=//g; s/ ms//g' | ttyplot 

Ttyplot can also read two inputs and plot with two lines, the second being in reverse-video. This is useful when you want to plot in/out or read/write at the same time.

A lot of performance metrics are presented in as a “counter” type which needs to be converted in to a “rate”. Prometheus and Graphana have rate() or irate() function for that. I have added a simple -r option. The time difference is calculated automatically. This is an example using snmpget which is show in screenshot above:

{ while true; do snmpget -v 2c -c public 10.23.73.254 1.3.6.1.2.1.2.2.1.{10,16}.9 | gawk '{ print $NF/1000/1000 }'; sleep 10; done } | ttyplot -2 -r -u "MB/s"

I now find myself plotting all sorts of useful stuff which otherwise would be cumbersome. This includes a lot of metrics from Prometheus for which you normally need a web browser. And how do you plot metrics from Prometheus? With curl:

{ while true; do curl -s http://10.4.7.180:9100/metrics | grep "^node_load1 " | cut -d" " -f2; sleep 1; done } | ttyplot

If you need to plot a lot of different metrics ttyplot fits nicely in to panels in tmux, which also allows the graphs to run for longer time periods.

Of course in text mode the graphs are not very precise, but this is not the intent. I just want to be able to easily spot spikes here and there plus see some trends like up/down – which works exactly as intended.I do dig fancy braille line graphs and colors but this is not my priority at the moment. They may get added later, but most importantly I want the utility to work reliably on most operating systems and terminals. 

You can find compiled binaries here and source code and examples to get you started – here.

If you get to plot something cool that deserves to be listed as an example please send it on!

SQL 2017 from 4.21a..

I would never ever recommend this, but…


isqlw connecting to a Linux SQL Server

I didn’t do anything to set this up.  I just searched for ISQLW and for some reason this ancient one showed up in the search path, and it connected.  I didn’t notice it at first until it didn’t like the newer shift insert/delete operations, as back then you needed to use Control C/V .. 

Not being able to stop there, I fired up the admin tool.  It complains that the stored procedure sp_MSAdmin_version  is missing.  However you can go ahead and create it…

create procedure sp_MSAdmin_version as
select “Microsoft SQL Administrator script version 4.20.22.1”
go

And it’ll connect.

Yes you can track stats in sort of real time

Oddly enough things that talk to the server work okay.  Things related to the databases don’t work at all.

SQLServer 2017 on Linux users

I even can admin users from 4.21’s admin program.

I guess the sp_MSAdmin_* scripts could be fixed up for 2017, allowing for a more robust experience, but I really can’t think of any reason why to do it.  I’m more surprised that all the new ODBC drivers since Vista won’t talk to SQL Server 4.21, 6.0, & 7.0, but it seems the client tools can talk to the new server.

I’ve even created the infamous ‘PUBS’ database from the 4.21a script as well.  Again not very useful, but all the more fun!

PUBS

Installation wasn’t too hard, but a little weird to re-produce.  Anyways you’ll need to trust the MS key

wget -qO- https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -

And then I added this into the /etc/apt/sources.list:

deb [arch=amd64] https://packages.microsoft.com/debian/9/prod stretch main
deb [arch=amd64] https://packages.microsoft.com/ubuntu/16.04/mssql-server-2017 xenial main

And then run the following to download MSSQL & the needed bits.  It’ll prompt a few times to agree to the License:

apt-get update;apt-get upgrade
apt-get install apt-transport-https
ACCEPT_EULA=Y apt-get install mssql-tools mssql-server && /opt/mssql/bin/mssql-conf setup

And if everything goes correctly you will then be prompted for the edition to use, the SA password, and then you can start the server with:

systemctl restart mssql-server.service

And away you go.

My output was like this:

# cat /etc/issue
Debian GNU/Linux 9 \n \l

[email protected]:/# apt-get update;apt-get upgrade
Hit:1 http://security.debian.org stretch/updates InRelease
Ign:2 http://debian.uchicago.edu/debian stretch InRelease
Hit:3 http://debian.uchicago.edu/debian stretch Release
Hit:4 https://dl.yarnpkg.com/debian stable InRelease
Hit:5 http://ftp.debian.org/debian stretch-backports InRelease
Hit:7 https://deb.nodesource.com/node_8.x stretch InRelease
Hit:8 https://packages.microsoft.com/debian/9/prod stretch InRelease
Hit:9 https://packages.microsoft.com/ubuntu/16.04/mssql-server-2017 xenial InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
[email protected]:/# apt-get install mssql-tools mssql-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libc++1 libodbc1 libsss-nss-idmap0 libunwind8 msodbcsql17 odbcinst odbcinst1debian2 unixodbc
Suggested packages:
  clang libmyodbc odbc-postgresql tdsodbc unixodbc-bin
The following NEW packages will be installed:
  libc++1 libodbc1 libsss-nss-idmap0 libunwind8 msodbcsql17 mssql-server mssql-tools odbcinst odbcinst1debian2 unixodbc
0 upgraded, 10 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/181 MB of archives.
After this operation, 932 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Preconfiguring packages ...
Selecting previously unselected package libc++1:amd64.
(Reading database ... 53362 files and directories currently installed.)
Preparing to unpack .../0-libc++1_3.5-2_amd64.deb ...
Unpacking libc++1:amd64 (3.5-2) ...
Selecting previously unselected package libodbc1:amd64.
Preparing to unpack .../1-libodbc1_2.3.4-1_amd64.deb ...
Unpacking libodbc1:amd64 (2.3.4-1) ...
Selecting previously unselected package libunwind8.
Preparing to unpack .../2-libunwind8_1.1-4.1_amd64.deb ...
Unpacking libunwind8 (1.1-4.1) ...
Selecting previously unselected package odbcinst1debian2:amd64.
Preparing to unpack .../3-odbcinst1debian2_2.3.4-1_amd64.deb ...
Unpacking odbcinst1debian2:amd64 (2.3.4-1) ...
Selecting previously unselected package odbcinst.
Preparing to unpack .../4-odbcinst_2.3.4-1_amd64.deb ...
Unpacking odbcinst (2.3.4-1) ...
Selecting previously unselected package unixodbc.
Preparing to unpack .../5-unixodbc_2.3.4-1_amd64.deb ...
Unpacking unixodbc (2.3.4-1) ...
Selecting previously unselected package libsss-nss-idmap0.
Preparing to unpack .../6-libsss-nss-idmap0_1.15.0-3_amd64.deb ...
Unpacking libsss-nss-idmap0 (1.15.0-3) ...
Selecting previously unselected package msodbcsql17.
Preparing to unpack .../7-msodbcsql17_17.2.0.1-1_amd64.deb ...
Unpacking msodbcsql17 (17.2.0.1-1) ...
Selecting previously unselected package mssql-server.
Preparing to unpack .../8-mssql-server_14.0.3037.1-2_amd64.deb ...
Unpacking mssql-server (14.0.3037.1-2) ...
Selecting previously unselected package mssql-tools.
Preparing to unpack .../9-mssql-tools_17.2.0.1-1_amd64.deb ...
Unpacking mssql-tools (17.2.0.1-1) ...
Setting up libsss-nss-idmap0 (1.15.0-3) ...
Setting up libodbc1:amd64 (2.3.4-1) ...
Setting up libunwind8 (1.1-4.1) ...
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Processing triggers for man-db (2.7.6.1-2) ...
Setting up libc++1:amd64 (3.5-2) ...
Setting up mssql-server (14.0.3037.1-2) ...
Setting up odbcinst1debian2:amd64 (2.3.4-1) ...
Setting up odbcinst (2.3.4-1) ...
Setting up unixodbc (2.3.4-1) ...
Setting up msodbcsql17 (17.2.0.1-1) ...
Setting up mssql-tools (17.2.0.1-1) ...
Processing triggers for libc-bin (2.24-11+deb9u3) ...
[email protected]:/# /opt/mssql/bin/mssql-conf setup
Choose an edition of SQL Server:
  1) Evaluation (free, no production use rights, 180-day limit)
  2) Developer (free, no production use rights)
  3) Express (free)
  4) Web (PAID)
  5) Standard (PAID)
  6) Enterprise (PAID)
  7) Enterprise Core (PAID)
  8) I bought a license through a retail sales channel and have a product key to enter.

Details about editions can be found at
https://go.microsoft.com/fwlink/?LinkId=852748&clcid=0x409

Use of PAID editions of this software requires separate licensing through a
Microsoft Volume Licensing program.
By choosing a PAID edition, you are verifying that you have the appropriate
number of licenses in place to install and run this software.

Enter your edition(1-8): 2
The license terms for this product can be found in
/usr/share/doc/mssql-server or downloaded from:
https://go.microsoft.com/fwlink/?LinkId=855862&clcid=0x409

The privacy statement can be viewed at:
https://go.microsoft.com/fwlink/?LinkId=853010&clcid=0x409

Do you accept the license terms? [Yes/No]:yes

Enter the SQL Server system administrator password:
Confirm the SQL Server system administrator password:
Configuring SQL Server...

ForceFlush is enabled for this instance.
ForceFlush feature is enabled for log durability.
Created symlink /etc/systemd/system/multi-user.target.wants/mssql-server.service → /lib/systemd/system/mssql-server.service.

Additionally you may not want to listen on every single IP address, but rather only on the loopback.  So you would run this to configure the listening address:

/opt/mssql/bin/mssql-conf  set network.ipaddress  127.0.0.1

I also use the SQL Agent, to enable that just simply run this:

/opt/mssql/bin/mssql-conf set sqlagent.enabled true 
systemctl restart mssql-server

Many more settings for the /var/opt/mssql/mssql.conf file can be found here: https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-configure-mssql-conf?view=sql-server-2017.  I would take a look at them, and possible enable stuff like TLS so that someone with management tools circa 1993 can’t just login to your server.  Then again maybe that is the kind of thing you want.

And if you don’t want Microsoft SQL Server, just do the following to uninstall MSSQL, destroying all data as well.

apt-get purge  mssql-tools mssql-server msodbcsql17
apt-get auto-remove
rm -rf /var/opt/mssql

I kept on getting this error which I didn’t see any way to cleanly resolve to fix for running MSSQL on Debian.  The best hint is the OpenSSL is either too new (unlikely) or too old (far too likely).  Instead I just changed distros as that is what people do, they don’t troubleshoot problems in Linux, just change distros so why bother fighting it?

# /opt/mssql-tools/bin/sqlcmd -Usa -PMYPa55w0rd!# -S127.0.0.1
Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : TCP Provider: Error code 0x2746.
Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Client unable to establish connection.
OpenSSL?

Going further though, as much as I liked Debian it really does run better on Ubuntu.  So as an addendum, use these sources (at the moment!).  Since the SQL Agent wouldn’t run, and I couldn’t connect locally it was worse than useless.

deb [arch=amd64] https://packages.microsoft.com/ubuntu/16.04//prod xenial main
deb [arch=amd64] https://packages.microsoft.com/ubuntu/16.04/mssql-server-2017 xenial main

Now the first time I tried to do anything on Ubuntu I got this lovely error:

# /opt/mssql-tools/bin/sqlcmd
terminate called after throwing an instance of 'std::runtime_error'
  what():  locale::facet::_S_create_c_locale name not valid

And it just hung the process.  I had to control-Z & kill -9 %1 it to get it out of the way.  Well it turns out that this VM didn’t have it’s locale set.  Fixing that was pretty simple, once you know how:

apt-get install locales && dpkg-reconfigure locales

Another thing that really bugs me is the lack of cryptography by default. So I found this nice recipie to for setting it up quickly.  Just watch your hostname!

systemctl stop mssql-server 
cat /var/opt/mssql/mssql.conf 
mkdir /var/opt/mssql/ssl
mkdir /var/opt/mssql/ssl/certs/
mkdir /var/opt/mssql/ssl/private/
cd /var/opt/mssql/
chown -R mssql:mssql *
cd
openssl req -x509 -nodes -newkey rsa:2048 -subj '/CN=HOSTNAME' -keyout mssql.key -out mssql.pem -days 3650
chown mssql:mssql mssql.pem mssql.key 
chmod 600 mssql.pem mssql.key
mv mssql.pem /var/opt/mssql/ssl/certs/
mv mssql.key /var/opt/mssql/ssl/private/
/opt/mssql/bin/mssql-conf set network.tlscert /var/opt/mssql/ssl/certs/mssql.pem 
/opt/mssql/bin/mssql-conf set network.tlskey /var/opt/mssql/ssl/private/mssql.key

This will build out a self signed certificate for 10 years and put them into the local MSSQL directory where it can read them.

Compiling rsync for VMWare ESXi 6.5

So what had started this little ‘adventure’ is that years ago there was this great sale over at Joe’s Datacenter, where I had picked up a physial server for the mere price of $20 USD a month!  What a deal!  No more quotas, CPU sharing issues and generally having to share.  Awesome!

So I have them install Debian, and load up the KVM modules, and away I go and life is good.  So foolishly years later, I jumped onto the whole container thing, installed docker, and that is where everything went south.

Every few seconds while doing a tcpdump on the 100% virtual bridge I’d see a massive influx of arp traffic.  I tried static arp’s on the host & the guest and it was ‘better’ but now the network traffic would hang.  Things like TCP would break after a minute and stuff like UDP game state would break so bad it’d end up unplayable.  This basically broke maraakate.org and hosting stuff like Quake I/ Quake II/ Daikatana and other iD based games.

My existing virtual machines now had a major issue where they would stop responding to traffic.  I never could find a fix, and it ended up with me moving my blog out to sloppy.io to keep running as a container based service until it magically stopped working and I gave up and did a full dump/reload on a hosted WordPress over at ChicagoVPS.  What a nightmare!

But what about all those virtual machines?

Well even after apt-get purge of everything docker, upgrading and downgrading the kernel nothing helped.  The VMs still dropped off the network periodically.  So with some spare time I decided to just go ahead and backup the box, and wipe the machine.

Since the physical network was working fine I was able to rsync the 300GB+ worth of data over the span of a few days.  That was fine, and considering how crap the server had been, I figured some straight downtime wouldn’t hurt anything too much.  While looking for an OS to install onto the machine, I saw that Joe offers VMWare ESXi 6.5, so I thought I’d just go with that, as naturally VMWare runs both VMs and with Project Photon I could maybe mess with containers again at a later date.

Since rsync had worked so well for moving hundreds of gigabytes of data from the USA to Hong Kong, I figured it’d be trivial to just convert the existing RAW KVM/Qemu disk images back to the United States of America.  And that is where the fun begins.

Let me tell you!

While reading this great post on virten.net they drop mention of XSIBackup, which requires registration (yuck) and worse their stupid registration system is broken:

LOL WHUT?

Rest assured the email does show up!

Dear Neo Zeed, thank you for visiting 33hops.com
This is your download url http://a.33hops.com/downloads/?key=bq7l5ptPB70MJj9dkftxFegr3xWoBZwpdFPQOUC3Cm10KPSIl3v1532877224253

But of course it doesn’t work

The key is invalid or has expired, only two downloads are allowed per key, download a new one at 33hops.com

What

The

Fuck

I know this is an ongoing issue at large when you provide executable binaries on the internet, as you will no doubt get flagged with some false positive by some virus crusaiding idiot who just sets up a tool and never reads anything but sends out threatening emails.  I went through this with the need for the simple 404 redirect, all thanks to Gerhard W. Recher.

So since this wasn’t going to be an avenue to persue I dug a little deeper and ran across this post over at virtuallyGhetto.  So it turns out the userland for ESXi is a CentOS environment that uses busybox.  And if you just download and install CentOS 3.9 into a VM, and build whatever you want, and ideally add in the -static flag, and copy it over.  For those who want to look into more ‘internals’ of the userland, check out zemris.fer.hr.

Great!

Things like UID/GUID mappings are broken in the libc it seems among other things.  So for my simple rsync config I just put the numbers in myself.

uid = 0
gid = 0
use chroot = no
max connections = 4
syslog facility = local5
pid file = /var/run/rsyncd.pid
hosts allow = a.b.c.d

[datastore1]
path = /vmfs/volumes/datastore1
comment = WDC_WD5000AAJS2D00A8B0
read only = no

I have read that you really ought to keep your binaries/config on the datastore so they aren’t subject to upgrades overwriting them and other chaotic stuff.  So editing the file “/etc/rc.local.d/local.sh” I just added the following before the exit 0:

/vmfs/volumes/datastore1/tools/rsync –daemon –log-file=/tmp/rsync.log –config=/vmfs/volumes/datastore1/tools/rsyncd.conf

And then ran it manually to kick it off.

So now I don’t have to rely on someone’s elses broken downoad system, and now we can build other fun native stuff.

And the best part is that after all of this fighting Maraakate’s site is back online and I get this message from him:

holy crap that new server setup so much better
its like playing it locally honest to god
played a whole unit not a single fuck up
no rubber banding lag effect or any of that

So now things are actually performing better on VMWare than we were getting on KVM.  And yes I had flattened out the disk images, loaded up the paravirtual disk & network drives on KVM, but VMWare does such a surprisingly better job.

I honestly wasn’t expecting that.

And as a bonus, I messed with qemu-0.9.0 (I didn’t feel the need to go through glibc2 hell), and qemu-img works great with a simple raw to vmdk

[[email protected]:/vmfs/volumes/5b5806fd-339444da-f897-003048d70598] ./tools/qemu-i
mg.static info win30.raw
image: win30.raw
file format: raw
virtual size: 32M (33554432 bytes)
disk size: 32M
[[email protected]:/vmfs/volumes/5b5806fd-339444da-f897-003048d70598] ./tools/qemu-i
mg.static convert -f raw -O vmdk win30.raw win30.vmdk
[[email protected]:/vmfs/volumes/5b5806fd-339444da-f897-003048d70598] ./tools/qemu-i
mg.static info win30.vmdk
image: win30.vmdk
file format: vmdk
virtual size: 32M (33554432 bytes)
disk size: 27M

And it boots!

Transcopied Windows 3.0 VM

So yes, wrapping up you can in fact run stuff on ESXi, copy data, and even convert disk images.

Oh yeah, and so people can deal with my 404 based download system (the password is on the 404 page)

Let the games begin!