Category Archives: virtual networking

Building a private mesh network

Posted on by
2

Well it was bound to happen sooner or later, but working on projects at the office I have segmented networks for various things, and the stuff I host only has to talk to one specific thing out on the internet.  But things change, and flexibility is demanded as now I suddenly have half a dozen VPS’s out on the internet, that not only need to talk back to my office, but also need to talk to eachother.  And of course adding that n+1 becomes tedious, and worse when one host goes down, and you have people pointing the wrong way.  If only there was a solution…

And there is, as this ‘problem’ was solved forever ago with this magical thing called routing protocol.  Before I was doing OpenVPN with a network to network type setup, but now I want to dynamically route, and considering it’s a small network, I’m free to use any protocol, even RIP is fine, as I have about 15 networks to advertise.  In the end I went with BGP for really no real reason.  RIP/OSPF,ISIS would have worked just as well.

The first thing is that to configure OpenVPN is that I didn’t want the normal ‘shared network’ setup, nor the tun style where OpenVPN intercepts the invalid destination address.  While it does bring up links, and work for dynamic setups, I’m just bringing up static links that are point to point, and single host only.  Also BGP wants the source and destination originating address to match, so a tunnel interface that is 10.0.1.6 10.0.1.5, but expects the destination to be 10.0.1.1 isn’t going to work.  So I went with the old p2p style config.

As a template I used this site right here:

First generate a static key:

openvpn --genkey --secret static.key

Next setup the config for the ‘server’ side.  In this case, each of my VPS’es is a server since they all have registered addresses, and don’t move as often.  My office is going to move soon, and in the off even that I need to travel with my office setup it’s nice to bring it online wherever, and have it connect.  At the same time if my office is down, this is why I want the VPSes to be able to talk to eachother.

Server config:

mode p2p
port 1194 #  We listen to port 1194
dev tun # possibly tun0 Linux
proto udp # protocol UDP, TCP: proto tcp-server

ifconfig 10.5.0.1 10.5.0.2 #  Local IP <-> Remote IP

# Here adjust the path to point to the generated Key
secret /etc/openvpn/static.key

ping 10 # Sends every 10 seconds a ping to the remote site
ping-restart # 180 after 3 minutes without pings reconnecting the remote
ping-timer-rem # only after we've let another peer ping-restart

verb 3 # increase to debugging
mute 50 # uncomment to debugging

Client config:

mode p2p
remote my-openvpn-server.dyndns.com 1194 # hostname / external IP of another peer, port accordingly
proto udp # protocol UDP, TCP: proto tcp-client
dev tun #  possibly tun0 Linux

ifconfig 10.5.0.2 10.5.0.1 #  Local IP <-> Remote IP

# Here adjust the path to point to the generated Key
secret /etc/openvpn/static.key

ping 10 # Sends every 10 seconds a ping to the remote site
ping-restart # 180 after 3 minutes without pings reconnecting the remote
ping-timer-rem # only after we've let another peer ping-restart

verb 3 # increase to debugging
mute 50 # uncomment to debugging

Now with the config’s in place, remembering to save with the filenames ‘server.conf’ and ‘client.conf’ in the /etc/openvpn directory, I was ready to restart the OpenVPN service (service openvpn restart) and now I could ping!

tun1 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.5.0.2 P-t-P:10.5.0.1 Mask:255.255.255.255

and a quick ping…

root@Office-OpenVPN:/etc/openvpn# ping 10.5.0.1
PING 10.5.2.1 (10.5.2.1) 56(84) bytes of data.
64 bytes from 10.5.0.1: icmp_req=1 ttl=64 time=1.66 ms
64 bytes from 10.5.0.1: icmp_req=2 ttl=64 time=3.39 ms

Now for the routing protocol.  I went with quagga, as it is descended from zebra, and one that I’ve used before.

Configuration is pretty straight forward.  first the daemons.conf file needs to be edited for what services you are going to run.  In this case I want zebra & bgpd.

zebra=yes
bgpd=yes
ospfd=no
ospf6d=no
ripd=no
ripngd=no
isisd=no
babeld=no

Next, the zebra.conf file.  The passwords are plaintext, I wouldn’t of course paste my real passwords:

hostname office
password zebra
enable password zebra

And next is my bgpd.conf file, for my office:

hostname office
password zebra
router bgp 8888
bgp router-id 192.168.1.49
network 192.168.1.0/24
network 10.1.0.0/24
network 10.1.1.0/24
neighbor 10.5.0.1 remote-as 555
log file /var/log/quagga/bgpd.log
!log stdout

Everything is the same on the VPS, except for it’s BGP config, which is the following:

hostname vps1
password zebra
router bgp 555
bgp router-id 10.13.0.1
network 10.13.0.0/24
neighbor 10.5.0.2 remote-as 8888
log file /var/log/quagga/bgpd.log
!log stdout

So now I’ve setup a p2p connection, and now defined the networks that I’m going to share from my office, in this case it’s 192.168.1.0/24, 10.1.0.0/24, and 10.1.1.0/24.  The VPS is going to share it’s 10.13.0.0/24 network.

Now to start up the router with a ‘service quagga restart’ and all being well I can now talk to the BGPD.

telnet localhost 2605
Trying ::1…
Trying 127.0.0.1…
Connected to localhost.
Escape character is ‘^]’.

Hello, this is Quagga (version 0.99.22.4).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

User Access Verification

Password:
office>

Now to check for my bgp neigbours

office> ena
office# show bgp neighbors
BGP neighbor is 10.5.0.1, remote AS 555, local AS 8888, external link
BGP version 4, remote router ID 127.0.0.2
BGP state = Established, up for 3d21h32m
Last read 00:00:33, hold time is 180, keepalive interval is 60 seconds
Neighbor capabilities:
4 Byte AS: advertised and received
Route refresh: advertised and received(old & new)
Address family IPv4 Unicast: advertised and received
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 1 0
Notifications: 0 0
Updates: 18 1
Keepalives: 5613 5612
Route Refresh: 0 0
Capability: 0 0
Total: 5632 5613
Minimum time between advertisement runs is 30 seconds

For address family: IPv4 Unicast
Community attribute sent to this neighbor(both)
1 accepted prefixes

Connections established 1; dropped 0
Last reset never
Local host: 10.5.0.2, Local port: 179
Foreign host: 10.5.0.1, Foreign port: 35722
Nexthop: 10.5.0.2
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
Read thread: on Write thread: off

office#

And how about the routes?

office# show ip bgp
BGP table version is 0, local router ID is 192.168.1.49
Status codes: s suppressed, d damped, h history, * valid, > best, i – internal,
r RIB-failure, S Stale, R Removed
Origin codes: i – IGP, e – EGP, ? – incomplete

Network Next Hop Metric LocPrf Weight Path
*> 10.1.0.0/24 0.0.0.0 0 32768 i
*> 10.1.1.0/24 0.0.0.0 0 32768 i
*> 10.13.0.0/24 10.5.13.1 0 0 555 i
*> 192.168.1.0 0.0.0.0 0 32768 i

Total number of prefixes 4

Well isn’t that great!  For each additional connection on my client side, I change the client port.  It doesn’t matter what number I choose as my client VPN machine isn’t exposed on the internet, only the OpenVPN servers on the VPSes.  I can also combine this with a traditional OpenVPN setup where my clients get addresses on my Office Lan, and can now access all of my VPSes at once.

All I have left to do is define the additional servers into the mesh, and add in the BGP peers.  And doing it this way two VPSes in the same data centre now have a 1ms ping, instead of traversing the internet to my office, and back up.  Plus they can stay in communication if my office, or even their internet is down, as long as the internal communications of the data centre is fine.

User Mode Linux

Posted on by
9
UML

UML

Well back in the day, before the hardware visualization craze of the 2000’s back in the dark days of 1999, there was UML, or better known as User Mode Linux.  Sadly the freshmeat announcement is all but lost, but sourceforge is still with us so we have the archives.

UML, simply put is a Linux kernel that has been modified to run in user space.  So to the OS, it is just another usermode program.  Because it runs in user space, there is no kernel dependencies, or special hardware required, as UML is just a user program.  Much in the way Qemu emulates a full machine, UML instead is the kernel process running with paravirtualized drivers so it should give overall a better/faster experience than hosting via Qemu.  Keeping in mind that VMware was a 1.0 launch product in may of 1999, and Bochs was back from 1994.

So what happened to UML?  The market was just too young to understand the power of hosted virtual machines, although there were some at the time.  Overall the market was in the idea of large servers that could ‘share’ thousands of websites, and most people didn’t either want to run root in their own little world, or didn’t even know there were other options besides getting an entire physical box.  I used to rent some UML VM out from a Canadian outfit back when I’d stream audio out on shoutcast, as I found my crappy connection at home couldn’t handle more than 5 listeners, but I could stream to the UML instance, and have it advertise out on shoutcast, and handle the 20-30 users I’d get back then.  Now that we live in the gated walled world of corporate hosting, I don’t think most people would even imagine going through the hell of building their own hosting infrastructure.

So it’s 2016, who cares today?

Well this… ‘friend’ of a ‘friend’ bought what they thought was a dedicated machine in a one year contract.  Except it turned out to be a KVM VM, with ‘dedicated’ processors to the instance.  Ouch.  They also got a bunch of ip addresses, but the idea of being able to run their web servers in VMs, along with an OpenVPN concentrator seemed to become an impossibility.  Obviously user mode Qemu is an option but it would be far too slow.  So I figured this would be a good time to use something from the past, UML.

So the ‘base’ host is running Debian 8.1.  So the first thing to do is to install the 2 neeed packages for networking, the bridge & uml utilities.

apt-get install bridge-utils uml-utilities
apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386

And with that in place I want to run 32bit UML kernels, which means I need the 32bit ‘runtime’ so the Debian way to enable 32bit exe’s is:

dpkg –add-architecture i386
apt-get update
apt-get upgrade
apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386

One thing to keep in mind about UML, or x86 Linux in general is that you cannot mix a 64bit kernel with a 32bit userland, the 64 bit kernel can run 32bit executables and libraries, but it really needs a full 64bit userland.  So from the UML page, to get going I went with the kernel linux-2.6.24-rc7.bz2 and an ancient copy of Debian 4.0 I found here.

For anyone interested in more ‘modern’ kernels and filesystems, be sure to check out uml.devloop.org.uk & fs.devloop.org.uk.

Ok, now we want to add a bridge interface for this type of setup.  The idea is that the UML’s will have ONLY an internal connection to the bridge, as the base machine owns all the physical addresses.

So it being Debian we modify the /etc/network/interfaces file to add:

auto br0
iface br0 inet static
address 10.13.0.1
netmask 255.255.255.0
network 10.13.0.0
broadcast 10.13.0.255
bridge_fd 9
bridge_hello 2
bridge_maxage 12
bridge_stp off
pre-up brctl addbr br0
post-down brctl delbr br0

The choice of 10.13.0.0/24 is 100% up to you.  You can use any RFC 1918 address without any issues.  Now I put this stuff in a script to fire up a UML host that will create my tap0 interface, bring it up, add it to the bridge, and fire up the UML.  On termination it’ll remove the tap from the bridge, and delete the interface.

tunctl -t tap0
ifconfig tap0 0.0.0.0 promisc up
brctl addif br0 tap0
./linux-2.6.24-rc7 ubda=test_fs eth0=tuntap,tap0 mem=512M
brctl delif br0 tap0
tunctl -d tap0

And once UML has booted up, a simple ifconfig can bring up the network, and I should be able to ping the bridge address

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
(none):~# ifconfig eth0 10.13.0.10 netmask 255.255.255.0
(none):~# ping 10.13.0.1
PING 10.13.0.1 (10.13.0.1) 56(84) bytes of data.
64 bytes from 10.13.0.1: icmp_seq=1 ttl=64 time=0.043 ms
64 bytes from 10.13.0.1: icmp_seq=2 ttl=64 time=0.067 ms

You get the idea.

Because I’m running some ancient debian root filesystem, I had to set the /etc/apt/sources to include the official archive to at least install things like ssh.

deb http://archive.debian.org/debian etch main

But that is pretty much that.

Now for the real fun part, putting it on the internet!  I’ve found that this is a simple way to redirect in port 80 on the registered address I want to use, and it allows the UML instance to NAT out the same address.

/sbin/iptables -t nat -A POSTROUTING -s 10.13.0.10/32 -o eth0 -j SNAT –to 1.2.3.4
/sbin/iptables -t nat -A PREROUTING -i eth0 -p tcp –dport 80 -d 1.2.3.4 -j DNAT –to 10.13.0.10:80
/sbin/iptables -t nat -A POSTROUTING -s 10.13.0.10/32 -o eth0:0 -j MASQUERADE

Remember that you’ll also need to enable ipv4 forwarding for this to work!  You’ll need to run:

sysctl -w net.ipv4.ip_forward=1

or add it to /etc/rc.local
And with all that done, the 32bit web server is now talking. It’s not what everyone was hoping for, but at least it’s a solution.

As an update, for those who don’t have the option of TUN/TAP bridging, there is always SLiRP, as covered here.

The client needs to access the internet!

Posted on by
Reply

But let’s not give them access to everything.

This is a common scenario I see, where someone needs to get updates to some magical software package on the internet.  Great.  And people just give them access to ANY site, which ends up being not only the internet (the intended destination) but the rest of their internal network.  Granted a good defense in the SDN world is inbound rules as well for each VM, but everything is never 100%.

RFC1918 defines our friends, the private address ranges:

     10.0.0.0        -   10.255.255.255  (10/8 prefix)
     172.16.0.0      -   172.31.255.255  (172.16/12 prefix)
     192.168.0.0     -   192.168.255.255 (192.168/16 prefix)

However the solution to this fun filled problem is to grant them HTTP/HTTPS access to the inverse of this.  Enter the netmask command.  You can give it a range, and it’ll lay out what networks to you need to add like this:

     netmask -c 0.0.0.0:9.255.255.255
     0.0.0.0/5
     8.0.0.0/7

Now I can exclude everything right up until 10.0.0.0/8 !

It’s quite the handy tool, but I didn’t see any Windows version.  So a few minutes with MinGW, and dealing with it’s weird Makefile’s way of linking things, and here you go!

This way you can permit internet access, not give them inside, access and still have a global DENY actually work.

YAY.

And if anyone is interested here are the networks:

     0.0.0.0/5
     8.0.0.0/7
    11.0.0.0/8
    12.0.0.0/6
    16.0.0.0/4
    32.0.0.0/3
    64.0.0.0/2
   128.0.0.0/3
   160.0.0.0/5
   168.0.0.0/6
  172.0.0.0/12 
 172.32.0.0/11
 172.64.0.0/10
 172.128.0.0/9
   173.0.0.0/8
   174.0.0.0/7
   176.0.0.0/4
   192.0.0.0/9
192.128.0.0/11
192.160.0.0/13
192.169.0.0/16
192.170.0.0/15
192.172.0.0/14
192.176.0.0/12
192.192.0.0/10
   193.0.0.0/8
   194.0.0.0/7
   196.0.0.0/6
   200.0.0.0/5
   208.0.0.0/4

Yes, I know it’s a LOT of typing.

One million packets served!

Posted on by
6
one million ICMP packets!

Success rate is 100 percent (1000000/1000000)!

So over in my work on porting Dynamips to MinGW, I’ve created a version of SLiRP that sends and receives data over UDP.  In retrospect, something I should have done a long time ago, as it makes troubleshooting it easier as now if it were to crash it’s a stand alone program, so it won’t crash the emulator.

The good news is that I’ve been able to copy files into the virtual router using HTTP.  I’ve even been able to access my OS/2 machine over FTP and load a file!

R1#copy ftp://10.12.0.14/README disk0:
Destination filename [README]?
Accessing ftp://10.12.0.14/README…
Loading README !
[OK – 76743/4096 bytes]

76743 bytes copied in 8.740 secs (8781 bytes/sec)

R1#dir disk0:README
Directory of disk0:/README

12 -rw- 76743 Sep 18 2015 09:01:08 +00:00 README

66875392 bytes total (41652224 bytes free)

Which is very cool!

Part of the ‘trick’ is that you should set your time out to be as long as possible to send a million packets.  I just set the maximum values.

line con 0

exec-timeout 35791 23
stopbits 1

line aux 0

stopbits 1

line vty 0 4

exec-timeout 35791 0
timeout login response 300
password cisco
login

!
end

In addition, a 7200 with idle performs MUCH better than a 1700 without idle.  There is something up with ptask, and only dispatching packets every so often.  I’m guessing it’s done that way for a reason.

Also one other cool IOS trick I learned today is that you can redirect to a file resource! Say you want that ‘show tech-support’ as a file on the disk? No problem!

show tech-support | redirect disk0:tech.txt

And of course the newer versions of IOS have a ‘do’ command that you can run from config mode to execute user commands.

R1(config)#do who
Line User Host(s) Idle Location
* 0 con 0 idle 00:00:00
2 vty 0 idle 15:24:11 10.0.2.2
3 vty 1 idle 14:59:56 10.0.2.2
4 vty 2 idle 13:43:44 10.0.2.2
5 vty 3 idle 11:23:44 10.0.2.2

 Take that Junos!

For anyone interested, the binary is included in the latest binary snapshot, and using it is pretty simple:

slirp_rdr.exe 20001 127.0.0.1 20000

This will listen on port 20001, and send traffic to 127.0.0.1 on port 20000.  Easy right?

Manually interfacing from the hypervisor can be the ‘fun’ part.  I haven’t tested with any of the tools, as I don’t know if they will let you leave something ‘listening’ that isn’t connected. For my tests I end up building something with their UI, then loading up my hypervisor that logs, and seeing what it is actually doing so I can inject stuff like this:

nio create_udp nio_udp99 20000 127.0.0.1 20001
ethsw add_nio S1 nio_udp99
ethsw set_access_port S1 nio_udp99 1

This creates a udp nio, and attaches it onto the virtual etherswitch S1, and puts it on VLAN 1.  As you can see it listens on UDP port 20000, which is where slirp_rdr is setup to send it’s data to, and it’ll send to 20001 where slirp_rdr is listening.

I’ve hard coded port 42323 to telnet into 10.0.2.15.  As always SLiRP is hard coded to have the following ip address schema:

Gateway 10.0.2.2
DNS 10.0.2.3
Netmask 255.255.255.0

Be sure to set your router to 10.0.2.15/24 for this to work, and add 10.0.2.2 as your default gateway.

The ONLY address that will respond to ping is 10.0.2.2 .  This is just the way SLiRP is.  HTTP and TCP based stuff works best, things like PPTP will not.  It’s really hit and miss, but the cool thing is that it doesn’t require any device drivers, it’s all user mode code!

Dynamips on MinGW

Posted on by
1

It’s always bugged me that the only way to build Dynamips for Windows was with Cygwin.

Well fear no more, I’ve mashed an old version (I would have tried newer, but of course Cmake fails spectacularly and with zero help as always!) and not only does it compile, but it can boot a 7200 version of IOS.

Dynamips on MinGW

Dynamips on MinGW

JIT is broken.  You have to telnet into the console.  And the console is a little wonkey as I’m sure it’s doing a lot more UNIX translation vs being a Win32 program but it does work enough to login, save the config, and reload.

But it’ll crash on reload.

I’m sure it’s full of bugs actually.

https://sourceforge.net/projects/dynamips-mingw/

I started with Dynamips 0.2.8-RC7-community and started commenting out stuff to get it to compile.  Luckily I found this ezwinports that includes mairix that includes some memory mapping functions, namely mmap and munmap ported to Win32 in an early glibc port. While I was trying to integrate libuuid, I got this fun error:

mingw “error: conflicting types” “UUID”

MinGW includes UUID support, since it’s a Microsoft thing.  Unfortunately libuuid doesn’t include unique names,  so I had to rename uuid_t to uu_uuid_t

//typedef unsigned char uuid_t[16];
typedef unsigned char uu_uuid_t[16];

in the uuid.h header file, along with all instances in Dynamips.

I also borrowed sendmsg/recvmsg along with the msg structures from VLC.  inetaton.c from WSHelper, and finally telnet.h from NetBSD.

After that it was a matter of making sure Winsock starts up, and fixing some linking breakage.

For those who want to try, the binary package is here.  I’ll have to setup git on this machine and upload all the changes.  It shouldn’t require any DLL’s, although I haven’t looked at the pcap stuff, as I mentioned it’s largely untested, so I have no idea if any of it works other than the telnet console.

PCem networking update to build 335

Posted on by
9

Well after extensive testing of various CFLAGS settings it turns out that “-O2 -flto -ffast-math -mfpmath=387” gives the best overall settings for PCem.

So yeah me and leileilol went through a dozen+ iterations to arrive at this fun conclusion.

So I’ve only included 2 executables, a debug and the -O2 build.

You can download it here.

I haven’t made any changes to the networking code, and even with a LOT of fighting got OS/2 Warp 3.0 Connect to install.

Gopher

Gopher

Plus I fixed my gopher!

**EDIT

I made a mistake, and built both exe’s as debug.  I updated the archive, those who downloaded it, will want to do so again!

PCem networking update to build 334

Posted on by
3

You can download the build here: PCem-build-334-pcap-slirp.7z

This includes more different core binaries, and mostly fixes the NE2000 to no longer panic and fault out when something dumb tries to probe it, by writing the wrong values in the wrong places.

For the curious build 334 is right here.  We didn’t make the feature cut for version 10, so hopefully it’ll make 11.  I’ll provide an unofficial build once v10 is announced, along with hopefully better networking back end modules, expanding things from pcap & SLiRP.

More progress on PCem and networking

Posted on by
1
SLiRP tcp redirects now working

SLiRP tcp redirects now working

PCem is different from other emulators in that when it starts up, reboots it’ll tear itself apart, and re-kick all the components.  Normally other emulators do this once, and as a result I never noticed that slirp_exit doesn’t actually purge the socket state.  And calling the socket teardown call causes a mbuf explosion in the code.  Sadly GDB is pretty useless trying to debug it, since it’s claiming all the structure members don’t exist.  Very strange.

Luckily I could duplicate the debug feature to go though current socket redirects, and close the sockets on the Windows side with a simple closesocket.

In this version I’ve setup the following TCP port redirects:

ExternalPORT    Internal Port
42322                 22
42323                 23
42380                 80
42443                 443

I still haven’t messed with the rc file, so there is no GUI config, instead you have to do it in the text files.  I have some notes on the whole thing on the pcem forum here.

Download the executables and source here:

http://vpsland.superglobalmegacorp.com/install/pcem/PCem-0657320820ab-pcap-slirp.7z

And for those interested, the diff against mainline 328 is here.

Adding SLiRP to PCem

Posted on by
8

So PCem is an incredible emulator for the IBM PC platform.  One thing that has been missing, and really missed has been networking.  So a while ago, SA1988 came up with a patch that incorporated the BOCHS ne2k.cc into PCem.

So as requested, I took the copy of SLiRP I’ve used in SIMH, Cockatrice and Previous, and got it working in PCem.

Telnet

Telnet from MS-DOS

This has to be one of the easier ports since PCem doesn’t use threads.  But yes, it appears to work, although I haven’t done any major testing.

For those who want to experiment, here is a binary/source blob of the project.  Right now we are just past the OMG it compiled phase to OMG it SENT and RECEIVED data phase.

If anyone wants to play, the NE2000 is set to 0x300 IRQ 10.

And you need to manually add the following to your pcem.cfg file:

netinterface = 1
netcard = 1

And you should be good to go. I think.

QuakeWorld

QuakeWorld

And yes, it’ll run QuakeWorld!