My Galaxy S6 finally got Android 5.1 today – that’s 9 months after the 5.1 release date and almost 3 months since Android 6.0 came out.
Too late. After four Galaxy Ss (1, 2, 4 and 6) I’ve already ordered a Nexus 6p which will arrive tomorrow. I’m done with the slow software updates. Nice hardware though.
In the last while I’ve spent some time learning about Docker, Kubernetes and Google Container Engine. It’s a confusing world at first but there are enough tutorials to figure out it all out if you spend the time.
While doing this I wanted to create a simple micro-service using Python 3.5’s Asyncio features. This seemed like the perfect fit for a micro-service. To have a useful goal, I ported our code that synchronizes the NightShift application with Hubspot. This works fine but after having it running for a while I discovered that the initial structure I built hid the tracebacks within tasks until the program exited. Figuring out a high level pattern that addressed this took a lot longer than I thought would. To help spare others from pain, and to hopefully create a useful tutorial, I have created a Github repo called python-asyncio-kubernetes-template.
This little repo has all the files required to create a simple Python/Asyncio micro-service and run it on your own Kubernetes cluster or on Google Container Engine. The micro-service has an HTTP endpoint to receive healthchecks from Kubernetes and it properly responds to shutdown signals. Of course, it also immediately prints exceptions which was the original pain point.
The README for the project contains a simple tutorial that shows how to use this end to end.
Hopefully this saves others time. If you know of a better way to do anything I’ve done in this repo please get in touch or submit a PR. It would be great if this repo grew to become a definitive template for micro-services built with these technologies.
python-asyncio-kubernetes-template
In the last while I’ve had to create several screencasts. After some experimentation I’ve found the following Gstreamer pipeline works well.
gst-launch-1.0 -e webmmux name=mux ! filesink location=test.webm \ pulsesrc ! queue ! audiorate ! audioconvert ! vorbisenc ! queue ! mux.audio_0 \ ximagesrc use-damage=false ! queue ! videorate ! videoconvert ! video/x-raw,framerate=5/1 ! vp8enc threads=2 ! queue ! mux.video_0
Couple of notes:
I’ve been using the Fish shell for a while now. Its auto-completion is so much better than Bash. You should try it.
I was looking at the Fish docs this morning and stumbled across this little gem.
Every configuration option in a program is a place where the program is too stupid to figure out for itself what the user really wants, and should be considered a failure of both the program and the programmer who implemented it.
From the Fish Design Documentation.
Part 1: Internet Redundancy, Or Not
Part 2: Redundant Connections to a Single Host?
In the last post I discussed how devices like your laptop and mobile phone are computing devices with multiple Internet connections not all that different from a network with multiple connections. The anecdote about Skype on a mobile phone reconnecting a call after you leave the range of Wi-Fi alludes to one key difference. That is, a device directly connected to a particular network connection can easily detect a total failure of said connection. In the example, this allowed Skype to quickly try to reconnect using the phone’s cellular connection.
Think back to our initial problem, how can a normal business get redundant Internet connections? The simplest, and at best half solution, is a router with two WAN connections and NATing out each port.
Now imagine you are using a laptop which is connected to a network with dual NATed WAN connections and you are in the middle of a Skype call. The connection associated with the Skype call will use one of the two WAN network ports and since NAT is used, the source address of the connection will be the IP address associated with the chosen WAN port. As we discussed before, this ‘binds’ the connection to the given WAN connection.
In our previous example of a phone switching to its cellular connection when the Wi-Fi connection drops, Skype was able to quickly decide to try to open another connection. This was possible because when the Wi-Fi connection dropped, Skype got a notification that its connection(s) were terminated.
In the case of a device, like our laptop, which is behind the gateway there is no such notification because no network interface attached to the local device failed. All Skype knows is that it has stopped receiving data – it has no idea why. This could be a transient error or perhaps the whole Internet died. This forces applications to rely on keep alive messages to determine when the network has failed. When a failure determination occurs, the application can try to open another connection. In the case of our dual NATed WAN connected network this new connection will succeed because the new connection will be NATed out the second WAN interface.
In the mean time, the user experienced an outage even though the network did still have an active connection to the Internet. The duration of this outage depends on how aggressive the application timeouts are. It can have short timeouts and risk flapping between connections or longer timeouts and provide a poorer experience. Of course this also assumes that the application includes this non-trivial functionality, most don’t.
Isn’t delivering packets the network’s job not the application’s?
Part 1: Internet Redundancy, Or Not
Previously I wrote about how true redundancy for Internet connections is only available to Internet providers and very large enterprises. This post continues from there.
I would guess that the fact that it’s not possible to get redundant Internet access is a big surprise to people who haven’t look into it in detail. Surprisingly though, if you have a smart phone or a laptop that you plugin to an Ethernet port, you live through the problems caused by this Internet protocol design flaw every day. These problems seem so normal that you may have never considered that reality could be otherwise.
Let’s start with the example of a laptop attached to a docking station at your desk. It’s very common to use the wired Internet connection at your desk vs. wireless because it typically offers faster, more consistent service. Consider the case of needing to transfer a large file while working at your desk. You start the transfer, it’s humming along in the background, and you switch over to another task. A few minutes later you remember that you have a meeting so you yank the laptop from its docking station and walk to the meeting room.
What just happened to the file transfer? The answer of course is that the file transfer died and you may be thinking, “Of course it died. The laptop lost its connection”. This seems normal because we’re all used to this brokenness.
Think back to the previous blog post. We were trying to get redundant Internet links to a small business and households and found that it’s not possible with the Internet protocols as they exist today. Now think about what the laptop is – it’s a computing system with two Internet connections. This really isn’t very different from a network with two connections. Ultimately, the reason the file transfer died is because of all the same limitations discussed in the last post related to network redundancy. That is, solving the problem of enabling multiple redundant connections for a network, solves the problem for individual hosts as well.
Now consider a smart phone user that starts a Skype voice or video conversation while in the office and then heads to the car to go meet a client. If you’ve accidentally tried this before you know that as you leave the range of the office Wi-Fi, the connection drops. In the particular case of Skype, it may be able to rejoin the conversation after the phone switches to the cellular data connection but most applications don’t even make an attempt at this. Like the laptop, a smart phone is just a computing device with multiple Internet connections.
One last example, your office server that runs Active Directory or performs some other important function. You probably would like network redundancy for this as well right? This also isn’t possible without low level ‘hacks’ like bonding two Ethernet ports to the same switch together.
Not only does the Internet not allow for true redundancy for networks, the lack of this functionality causes trouble for end hosts as well.
