uSwitch is a comparison and switching service centered around the home. Our teams are organised around the product that a customer is trying to compare (energy, broadband + TV, mobile phones, insurance and banking products). Traditionally infrastructure has been built and operated inside of these different teams. We are a cross product team working to improve consistency in how our services are built, deployed and automated.
Teams have used AWS primitives directly (via terraform, cli and older configuration management tools), we are now using Kubernetes running on AWS to help us provide a better abstraction for teams to run their services. We are using this abstraction to raise the bar for data security and improve the tooling teams have to introspect their running services.
We use a collection of open source software to support our infrastructure platform: Drone CI, ElasticSearch+Logstash+Kibana (ELK), Prometheus, Grafana, Vault.
Examples of some projects we have recently worked on:
- Short lived database credentials
Our running services previously relied on having long lived credentials to access data that were rarely, if ever, rotated. We wanted human and pod identity to be used to grant short-lived credentials based on policies. We used Vault to build a solution to this problem, creating tooling such as vault-creds to make it as easy as possible for developers to use these credentials with their services.
- Kiam: a service that integrates AWS IAM with Kubernetes
We have a lot of existing AWS resource that have their access limited using IAM. We used Kube2IAM initially but experienced race conditions that would hand different role credentials to pods. We started work on a replacement and have worked with community to get it used in other places. Kube-Aws are now incorporating it into their deployment.
- Heimdall: an alert custom resource controller
In order to manage alerts teams want to create against data in Prometheus, we created Heimdall. It aggregates Alert resources, updating Prometheus configuration. Where we have common alerts - 5xx rate alerts for example - Heimdall provides annotations that can be added to Kubernetes resources that simplify alert creation.
- Plan and work on our common infrastructure
- Work with teams to design, build and improve systems
- Help migrate existing teams onto our common infrastructure
- Debug issues across applications and levels of the stack
- Develop tooling to help our teams work better