Category Archives: General

Google gcloud and API rate limits

Leave a reply

Lately we’ve been making use of the Google Pub/Sub service at work. During development I ran into continual problems with API rate limits doing a modest number of API requests.

After some investigation with the Google support people (we have Gold support) it turns out that if you use the default authentication provided by the ‘gcloud’ command in your local development environment, you get different rate limits than a service account does.

I lost a lot of time trying to optimize for these API rate limit errors. Hopefully this little note will stop someone else from having to go through the same pain.

A couple weeks with the Nexus 6p

Leave a reply

As I wrote a couple weeks ago, I switched to a Nexus 6p from a Samsung S6 mostly because the lack of software updates was frustrating.

Here are a few random thoughts on the Nexus 6p now that I’ve used it for a couple weeks:

  • The hardware build quality is almost as good as the Galaxy S6. Certainly better than the Galaxy S4.
  • The screen is quite nice but the colors are a bit weird until you turn on the developer options and enable the sRGB mode. This really should be the default setting.
  • No un-installable carrier bloatware and Samsung apps is a big plus. Not being able to delete the Bell TV app or Microsoft OneDrive (because Samsung made a deal with them) was stupid and annoying. It also highlights how little respect Samsung has for the end user vs. trying to please the carriers.
  • The raw benchmarks on the Galaxy S6 beat the Nexus 6p but the 6p feels much faster during normal use. I don’t know the cause for certain but to whatever extent this is a result of TouchWiz, Samsung should just quit trying to do user facing software. They are horrible at it.

So far my experience is that the Nexus 6p is an upgrade from the Galaxy S6.

Python, Asyncio, Docker and Kubernetes

Leave a reply

In the last while I’ve spent some time learning about Docker, Kubernetes and Google Container Engine. It’s a confusing world at first but there are enough tutorials to figure out it all out if you spend the time.

While doing this I wanted to create a simple micro-service using Python 3.5’s Asyncio features. This seemed like the perfect fit for a micro-service. To have a useful goal, I ported our code that synchronizes the NightShift application with Hubspot. This works fine but after having it running for a while I discovered that the initial structure I built hid the tracebacks within tasks until the program exited. Figuring out a high level pattern that addressed this took a lot longer than I thought would. To help spare others from pain, and to hopefully create a useful tutorial, I have created a Github repo called python-asyncio-kubernetes-template.

This little repo has all the files required to create a simple Python/Asyncio micro-service and run it on your own Kubernetes cluster or on Google Container Engine. The micro-service has an HTTP endpoint to receive healthchecks from Kubernetes and it properly responds to shutdown signals. Of course, it also immediately prints exceptions which was the original pain point.

The README for the project contains a simple tutorial that shows how to use this end to end.

Hopefully this saves others time. If you know of a better way to do anything I’ve done in this repo please get in touch or submit a PR. It would be great if this repo grew to become a definitive template for micro-services built with these technologies.

python-asyncio-kubernetes-template

 

Creating a Screencast with Audio in Linux

Leave a reply

In the last while I’ve had to create several screencasts. After some experimentation I’ve found the following Gstreamer pipeline works well.

gst-launch-1.0 -e webmmux name=mux ! filesink location=test.webm \
  pulsesrc ! queue ! audiorate ! audioconvert ! vorbisenc ! queue ! mux.audio_0 \
  ximagesrc use-damage=false ! queue ! videorate ! videoconvert ! video/x-raw,framerate=5/1 ! vp8enc threads=2 ! queue ! mux.video_0

Couple of notes:

  • use-damage=false is important if you are using a composited desktop (eg Gnome 3). It took a discussion with a dev on #gstreamer to figure this out.
  • framerate=5/1 reduces the framerate to 5 per second. This greatly reduces the amount of video encoding required. My computer couldn’t keep up without setting this. The low framerate works quite well for a screecast and keeps the files small.

Configurability is the root of all evil

Leave a reply

I’ve been using the Fish shell for a while now. Its auto-completion is so much better than Bash. You should try it.

I was looking at the Fish docs this morning and stumbled across this little gem.

Every configuration option in a program is a place where the program is too stupid to figure out for itself what the user really wants, and should be considered a failure of both the program and the programmer who implemented it.

From the Fish Design Documentation.