Today I rebuilt one of our micro-services using Go 1.7.
1.7’s new SSA compiler produces faster code but I was surprised to see this much improvement. This isn’t a perfect comparison – a couple of library dependencies got updated as well and Kubernetes may have put the containers on different machines in the cluster.
At work we’ve started to instrument our micro-services with Prometheus. The combination of Prometheus and Grafana is pretty amazing.
One of the nice things Prometheus does is auto-discovery of services in a Kubernetes cluster. The docs for this area aren’t bad but I couldn’t find a complete example so here’s the configuration we use.
- job_name: 'kubernetes'
kubernetes_sd_configs:
- api_servers:
- 'https://kubernetes.default.svc'
in_cluster: true
relabel_configs:
- regex: "__meta_kubernetes_role"
action: labelmap
replacement: "kubernetes_role"
- regex: "__meta_kubernetes_node_label_(.*)"
action: labelmap
replacement: "kubernetes_node_label_${1}"
- regex: "__meta_kubernetes_service_namespace"
action: labelmap
replacement: "kubernetes_service_namespace"
- regex: "__meta_kubernetes_service_name"
action: labelmap
replacement: "kubernetes_service_name"
- regex: "__meta_kubernetes_service_label_(.*)"
action: labelmap
replacement: "kubernetes_service_label_${1}"
- regex: "__meta_kubernetes_service_annotation_(.*)"
action: labelmap
replacement: "__meta_kubernetes_service_annotation_${1}"
In the last while I’ve spent some time learning about Docker, Kubernetes and Google Container Engine. It’s a confusing world at first but there are enough tutorials to figure out it all out if you spend the time.
While doing this I wanted to create a simple micro-service using Python 3.5’s Asyncio features. This seemed like the perfect fit for a micro-service. To have a useful goal, I ported our code that synchronizes the NightShift application with Hubspot. This works fine but after having it running for a while I discovered that the initial structure I built hid the tracebacks within tasks until the program exited. Figuring out a high level pattern that addressed this took a lot longer than I thought would. To help spare others from pain, and to hopefully create a useful tutorial, I have created a Github repo called python-asyncio-kubernetes-template.
This little repo has all the files required to create a simple Python/Asyncio micro-service and run it on your own Kubernetes cluster or on Google Container Engine. The micro-service has an HTTP endpoint to receive healthchecks from Kubernetes and it properly responds to shutdown signals. Of course, it also immediately prints exceptions which was the original pain point.
The README for the project contains a simple tutorial that shows how to use this end to end.
Hopefully this saves others time. If you know of a better way to do anything I’ve done in this repo please get in touch or submit a PR. It would be great if this repo grew to become a definitive template for micro-services built with these technologies.
python-asyncio-kubernetes-template