Upgrading to WSL2

I finally got the update to version 2004 of Windows 10 (OS Build 19041.329) which now includes support for running a Linux kernel inside a VM, using paravirtual drivers letting it hook into Windows 10. It reminds me of WinOS/2 where, OS/2 would run Windows 3.1 in a virtual machine, but using special drivers it could redirect it on the native filesystem, and paint the windows on the OS/2 desktop.

So the first thing is make sure you have 2004, winver should look like this:

You will need to go to the old control panel, and enable the Virtual Machine Platform. For those of you using VMware, you’ll need at least version 15.5.5 (15.5.6 is current as of the time of me writing this). I’d suggest you do that before turning on this, but some people like to live dangerously. Also fully shut down all your VM’s otherwise things will end up giving you errors (as you’ll see later).

After the Virtual Machine Platform is installed, your machine will need to reboot. After a reboot, you’ll need to download the Linux Update to the system, available at:

https://aka.ms/wsl2-install

It’s a simple install, and only takes a few seconds. I didn’t have to reboot.

now for the fun parts, you’ll need to list and shut down (terminate) your running WSL’s if you have any running.

C:\Users\neozeed>wsl --list --verbose
NAME STATE VERSION
Debian Running 1

As you can see I only have one, and it’s called Debian, and it is currently running something. Let’s kill it!

C:\Users\neozeed>wsl -t Debian
C:\Users\neozeed>wsl --list --verbose
NAME STATE VERSION
Debian Stopped 1

And now it’s stopped. The command to upgrade is pretty simple:

C:\Users\neozeed>wsl --set-version Debian 2
Conversion in progress, this may take a few minutes…
For information on key differences with WSL 2 please visit https://aka.ms/wsl2

And yes, you can also stop the WSL instance, and move it back to version 1, if you find 2 not doing what you want. However on first impressions, WSL2 is significantly faster.

If you are running VMware, and you didn’t shut down your VM’s you’ll get this lovely message when you try to resume:

The save state will be deleted, but rest assured it’ll launch okay (at least mine did!)

10.6 the last great OS X

One sad consequence of using the Microsoft Virtual Machine framework is that nesting is no longer available.

which is kind of a bummer, as that was the killer feature.

Qemu 5 was recently released

It’s been jumping numbers like crazy, and I’m still holding onto 0.9 because I’m weird. Anyways there is something amazing hiding in all those release notes and stuff:

68040!

The m68k emulation is good enough to run Linux! Granted the target machine is the Macintosh 800, although the Mac ROM doesn’t boot enough to do anything Apple enough, using a serial console however does get us into the system. On my ancient Mac Pro I get emulation in the speed range of a 1Ghz G4!

[email protected]:~# cat /proc/cpuinfo 
CPU: 68040
MMU: 68040
FPU: 68040
Clocking: 1304.9MHz
BogoMips: 869.99
Calibration: 4349952 loops

Absolutely amazing!

Installation is a bit tricky as there is no true bootrom / boot process, so I had to load the installer with a dummy ‘raw’ disk, and tar the kernel & initrd to that raw disk so I could later extract it on the native OS to boot into the disk. I followed mostly the instructions here.

And what is that? NeXT CUBE emulation?

The peripherals are nowhere near complete enough to boot, HOWEVER it does boot the PROM, complete with keyboard support.

qemu-system-m68k -M next-cube -bios Rev_2.5_v66.BIN

It’s fun enough to play with. And thanks to Qemu’s fast emulation, perhaps this is a speedy way to run stuff in the future?

Now isn’t that cool?

Slackware finally goes to Patreon

It’s been a long time overdue, where helping to keep afloat the oldest continues Linux distro is now super easy to do!

Hi folks, Slackware founder Patrick Volkerding here. Welcome to the official page supporting the Slackware Linux Project. Slackware was founded in 1993 and made its first public releases that same year, making it the longest active Linux distribution project.

Today there are many Linux distributions available, but I’ve remained dedicated to this project as I believe it still holds an important place in the Linux ecosystem. Slackware is free, open source software that gives you everything you need to recompile the entire operating system and make any changes you like. The package system is constructed from shell scripts using standard utilities, and easily modified scripts are used throughout so that users may customize their operating system without having to recompile anything. Included software is changed very little (usually not at all) from the way the upstream developers intended. Einstein once said, “Everything should be made as simple as possible, but not simpler” — and when it comes to an operating system, trying to make things “easy” does not always achieve that goal. Anyone who has ever struggled with a package dependency system that doesn’t like the replacement of a system component knows what I’m talking about here. Slackware users know that once you’ve learned how to use the system, it’s easier to use than something that tries to put a GUI setup tool in front of the underlying text files that actually configure the system. And they know that Slackware can be trusted not to constantly change the way things work, so that your investment in learning Slackware lasts longer than it would with a system that’s a moving target. Slackware still builds upon a foundation of the latest development tools, libraries, and applications, and is supported by a vibrant user/developer community that welcomes new users and provides well-tested build scripts for nearly any software you may require.

Your support is greatly appreciated, and will make it possible for me to continue to maintain this project. Cheers!

For me, Slackware holds a special place as just as soon as SLS became a thing it quickly fell into disrepair and rot. The final straw was trying to bootstrap that new fangled ELF thing (it’s almost impossible to find any talk of that COFF thing) and it was just an incredible amount of work. At that point just getting Slackware on a shovelware CD-ROM set was just nothing short of incredible. And it was obvious at the time of ELF that the whole ‘I can’t xyz with little to no effort, therefore dump that distro and try another one’ was born. And Linux never recovered. Look no further at the insane cruft and required infrastructure for Docker to just load an application.

With that said when building a lot of crap from source it’s sometimes nice to have something that’ll just get out of the way, and Slackware does a reasonable job of that. Although last time I really fought with Slackware 13.37 it really didn’t go so well. But at the same time I am willing to make monetary support to ensure the future of Slackware.

The project currently stands at 230 supporters. Which is pretty good, as when it comes to supporting things, few will actually put their money into it, so hopefully this will allow Slackware to continue to exist, and maybe even flourish. I for one find this anti systemd distro important for the sake of software diversity in the ‘me too Debian’ sea of clones out there. Just as supporting OpenBSD is important in the standpoint of security.

You too can support Patrick over on patreon.com.

Not having fun with Python/Debian 9.1

Well after my last Star Wars Galaxies adventure, where I tried to run MySQL on Linux Subsystem for Windows v1, I got some weird shared memory error, and it wouldn’t run. Even the old BSDDB engine was bombing out trying to create files. So fine, whatever I thought I could move on, and that is when I found out that somehow OpenSSL & Python had utterly collided.

Python 1.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import ssl Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/ssl.py", line 98, in import _ssl # if we can't import it, let the error propagate ImportError: /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1: version `OPENSSL_1_1_0' not found (required by /usr/lib/python2.7/lib-dynload/_ssl.x86_64-linux-gnu.so) >>>

Well isn’t that great. I tried un-installing & re-installing Python over and over, along with trying to force re-install OpenSSL. No dice.

So what finally got it working for me was to purge OpenSSL.

apt-get purge libssl1.1

And after that it pulled out everything that was using it, well over 500MB of stuff I’d installed. And for good measure I followed up with the autoremove for an additional 384MB of stuff to remove. And then for the final step, of just installing Python:

[email protected]:~# apt-get install python2.7 Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: libpython2.7-stdlib libssl1.1 Suggested packages: python2.7-doc The following NEW packages will be installed: libpython2.7-stdlib libssl1.1 python2.7 0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded. Need to get 3,537 kB of archives. After this operation, 12.8 MB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://deb.debian.org/debian stretch/main amd64 libssl1.1 amd64 1.1.0j-1~deb9u1 [1,354 kB] Get:2 http://deb.debian.org/debian stretch/main amd64 libpython2.7-stdlib amd64 2.7.13-2+deb9u3 [1,897 kB] Get:3 http://deb.debian.org/debian stretch/main amd64 python2.7 amd64 2.7.13-2+deb9u3 [285 kB] Fetched 3,537 kB in 0s (13.2 MB/s) Preconfiguring packages ... Selecting previously unselected package libssl1.1:amd64. (Reading database ... 27441 files and directories currently installed.) Preparing to unpack .../libssl1.1_1.1.0j-1~deb9u1_amd64.deb ... Unpacking libssl1.1:amd64 (1.1.0j-1~deb9u1) ... Selecting previously unselected package libpython2.7-stdlib:amd64. Preparing to unpack .../libpython2.7-stdlib_2.7.13-2+deb9u3_amd64.deb ... Unpacking libpython2.7-stdlib:amd64 (2.7.13-2+deb9u3) ... Selecting previously unselected package python2.7. Preparing to unpack .../python2.7_2.7.13-2+deb9u3_amd64.deb ... Unpacking python2.7 (2.7.13-2+deb9u3) ... Processing triggers for mime-support (3.60) ... Processing triggers for libc-bin (2.24-11+deb9u4) ... Setting up libssl1.1:amd64 (1.1.0j-1~deb9u1) ... Processing triggers for man-db (2.7.6.1-2) ... Setting up libpython2.7-stdlib:amd64 (2.7.13-2+deb9u3) ... Setting up python2.7 (2.7.13-2+deb9u3) ... Processing triggers for libc-bin (2.24-11+deb9u4) ...

So now you think its going to be broken right? It’s the same libssl package! I didn’t even run an ‘apt-get update’. And guess what?! You would be wrong.

[email protected]:~# python Python 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import ssl >>>

That’s right, it’s now working.

Speaking of Picard, I hear there will be a new series, inspiringly named ‘Picard’. Apparently it’s going down the same path as STD, complete with a lack of backers, and merch. I’m hoping it’ll be something watchable, although they certainly cannot afford any of the TNG cast as they apparently have zero budget. Maybe it’ll end up as another ‘Jake Skywalker’ or ‘Not my Picard’. But the bad reboot 25% contractual difference could be the difference between something new, or something so divergent that there was no point in even attempting to be a Trek.

Proot, defeating security by ignoring it in user-space

From the proot project site :

PRoot is a user-space implementation of chroot, mount –bind, and binfmt_misc.

Android among other Linux systems creates a very restricted user mode where the end user is denied the root user privileges. This is annoying as to ‘root’ a phone can be incredibly complicated l, and beyond a normal user that wants to use their phone for more than some kind of cat video social machine (don’t tell me the incredible popularity of cat videos isn’t toxoplasmosis!).

Many new phone SOC’s are supporting external HDMI, and USB host capabilities allowing you to dock your phone and use it with a keyboard and mouse.

Well thanks to the app, aptly named UserLand, running a light weight Linux distro is just a few screen presses away!

What is cool is that by emulating a scant few system calls it makes the deployment quick and seemingly trivial. And a lot more lightweight compared to docker, User Mode Linux, or Qemu (in full system emulation). But it can invoke qemu to run foreign architecture binaries to give Intel users an Arm UserLand.

Yes, via VNC you can run X11! And yes on my phone it shows all 8 Cores.

Although I’ve been using qemu, UML and other strategies to sidestep restrictive environments, proot proves itself as an exciting new tool!

End of an era, Linux to Deprecate a.out support

From the post on kernel.org:

Linux supports ELF binaries for ~25 years now. a.out coredumping has bitrotten quite significantly and would need some fixing to get it into shape again but considering how even the toolchains cannot create a.out executables in its default configuration, let’s deprecate a.out support and remove it a couple of releases later, instead.

I can’t say I’m all that surprised, maintaining backwards compatibility has not really been a Linux thing, as most people are incapable of doing any troubleshooting in the myriad of hundreds to thousands of independent packages, and instead find it far easier to switch to a different distro entirely.

At the same time, the vast majority of Linux packages are available in source code, so re-building things as ELF most likely has happened in the last 25 years.

During the great ELF migration, it was a gigantic PITA having basically 2 copies of all the libraries as things were converted over, and a.out stuff quickly evaporated. For me, the beauty of a.out was for later kernels to be able to mount and run older stuff. But as we are in the era of both ‘cheap’ user mode kernels along with virtualization will the old executable format truly be missed?

Linux has survived the removal of native support for the 80386, and even the detection logic for the NexGen processors (yes they were real, and yes they did ship), so no doubt this further amputation won’t matter to the vast majority.

I have to wonder how long until the i386 32bit target is removed? Distros like Debian have long since removed support for 80486/80586 classed processors to bring the minimum up to requiring SSE-2 based instructions, and I can’t imagine anyone who is running a 32bit OS for their main OS in this day and era.

diff --git a/arch/x86/um/Kconfig b/arch/x86/um/Kconfig index f518b47..494eeb5 100644 --- a/arch/x86/um/Kconfig +++ b/arch/x86/um/Kconfig @@ -16,7 +16,6 @@ config 64BIT config X86_32 def_bool !64BIT - select HAVE_AOUT select ARCH_WANT_IPC_PARSE_VERSION select MODULES_USE_ELF_REL select CLONE_BACKWARDS

Farewell a.out

Update: from further along in the changelog

And Alan Cox points out that the a.out binary loader _could_ be done in user space if somebody wants to, but we might keep just the loader in the kernel if somebody really wants it, since the loader isn’t that big and has no really odd special cases like the core dumping does.

Linux hits version 5.0!

YES OMG I can’t take it anymore.

So many emails/blogs/videos/messages/tweets/whatevers!

I get it, ok ok, Linus thought 4.21 seemed silly so went straight to 5.0. Contrary to the numbering it’s not a significant divergence.

There.

I said it.

Am I the only one being hit with 50,000 things about Linux 5.0?!

So I was building a Windows 2019 server

As I got a ‘totally legit’ serial code in my box of cereal.

After the install I thought it’d be fun to install the Linux Subsystem.

While following the powershell instructions here, I thought the list of quick links of distros to download was interesting:

That’s right ARM Linux userland! I still have high hopes for Windows on ARM (I have 2 Windows RT devices now!!) although I’m not holding my breath.

Maybe there will be some ARM boards that are suitable for the desktop that aren’t over 1k USD.. That’d be nice.

Interesting trivia is that the Linux Subsystem started it’s life on ARM as a way to run Android binaries on Windows Phone. And true to everything Microsoft does, it got to the point where it could start to run things (albeit poorly) and was summarily killed. Although it’s found life despite the original false start as a general ‘text mode’ subsystem for Windows.

However running Linux binaries on Windows currently just shows that NTOS isn’t as efficient as the Linux kernel when it comes to emulating the Linux ABI. Although this was the original ‘dream’ of the microkernel, and a POSIX subsystem for NT was always part of the original design, although it really was more of a checkbox for GSA contracts, and outside of being able to use pax & vi it really was handicapped by not having BSD extensions, and especially by not having any access to the TCP/IP stack.

EDIT*

I should add these notes from the future past for the future me, when messing around with Windows Server 2019 build 1809 when they finally brought the Linux Subsystem into the fold. Unpacking the distribution and running the ‘setup’ sets it up DIRECTLY into that directory. So put it where you want it.

When you mess that up, you have to use the wslconfig program!

Of note is:

wslconfig /list /all
Lists all distributions, including ones that aren’t currently usable. They may be in the process of installing, uninstalling, or are in a broken state.

wslconfig /unregister <DistributionName>
Unregisters the distribution from WSL so it can be reinstalled or cleaned up.

This way you can now clean up your mess, and get Linux installed right.

Oldlinux.org got updated!

I know that on the surface it may not seem like much, but considering the last update was nearly 3 years ago…

2019-02-08   

Time flies, it’s flying fast, and it’s been two years till now. The latest Chinese revision, the Fifth revision of the book:”A Heavily Commemted Linux Kernel Source Code” is now HERE, and, more importantly, the English version of the book is also given HERE. After nearly one year of translation, the English version of this book has finally been completed. Of course, there must be translation errors and tipos in the book. I hope you may point them out for me. Finally, thanks Trent Jarvi who helped me a lot, thank you friends in the Linux communities, and thank you all, and Happy Chinese Spring Festival!

Yes, that’s right the book is now available in English. And in PDF form! It’s an extensive dive into the 0.12 source, complete with diagrams, notes and annotated source code.

It’s a whopper though, 1109 pages, and weighing in at 11 MB! This is not a light read!

As always the site is http://www.oldlinux.org/

ttyplot – a real time plotting utility for the terminal

(This is a guest post from Antoni Sawicki aka Tenox)

I spend most of time in a day staring at a terminal window often running various performance monitoring tools and reading metrics.

Inspired by tools like gtop, vtop and gotop I wished for a more generic terminal based tool that would visualize data coming from unix pipeline directly on the terminal. For example graph some column or field from sar, iostat, vmstat, snmpget, etc. continuously in real time.

Yes gnuplot and several other utilities can plot on terminal already but none of them easily read data from stdin and plot continuously in real time.

In just couple of evenings ttyplot was born. The utility reads data from stdin and plots it on a terminal with curses. Simple as that. Here is a most trivial example:

To make it happen you take ping command and pipe the output via sed to extract the right column and remove unwanted characters:

ping 8.8.8.8 | sed -u 's/^.*time=//g; s/ ms//g' | ttyplot 

Ttyplot can also read two inputs and plot with two lines, the second being in reverse-video. This is useful when you want to plot in/out or read/write at the same time.

A lot of performance metrics are presented in as a “counter” type which needs to be converted in to a “rate”. Prometheus and Graphana have rate() or irate() function for that. I have added a simple -r option. The time difference is calculated automatically. This is an example using snmpget which is show in screenshot above:

{ while true; do snmpget -v 2c -c public 10.23.73.254 1.3.6.1.2.1.2.2.1.{10,16}.9 | gawk '{ print $NF/1000/1000 }'; sleep 10; done } | ttyplot -2 -r -u "MB/s"

I now find myself plotting all sorts of useful stuff which otherwise would be cumbersome. This includes a lot of metrics from Prometheus for which you normally need a web browser. And how do you plot metrics from Prometheus? With curl:

{ while true; do curl -s http://10.4.7.180:9100/metrics | grep "^node_load1 " | cut -d" " -f2; sleep 1; done } | ttyplot

If you need to plot a lot of different metrics ttyplot fits nicely in to panels in tmux, which also allows the graphs to run for longer time periods.

Of course in text mode the graphs are not very precise, but this is not the intent. I just want to be able to easily spot spikes here and there plus see some trends like up/down – which works exactly as intended.I do dig fancy braille line graphs and colors but this is not my priority at the moment. They may get added later, but most importantly I want the utility to work reliably on most operating systems and terminals. 

You can find compiled binaries here and source code and examples to get you started – here.

If you get to plot something cool that deserves to be listed as an example please send it on!