Cloud dependencies

Join me on the Software Engineering Daily podcast. In this episode, Jeff and I discuss my personal project deps.cloud. We discuss benefits and tradeoffs to leveraging monorepos as well as challenges mainting source control at a large organization. I hope you enjoy the show!

Checking Service Dependencies in Kubernetes

Back in July, I found myself needing to better coordinate deployments of my applications to Kubernetes. After searching around, I found many ways that people where trying to solve this problem. Some used shell scripts to apply multiple YAML files with a fixed time sleep between them. Others used shell scripts and tailed the rollout using kubectl rollout status -w. Now, I manage a lot of my deployments using GitOps and Flux. So leveraging these shell scripts to manage my rollouts into clusters wasn’t really an option.

It wasn’t until I came across Alibaba Cloud’s blog post on solving service dependencies that I felt like I had something to work with. The article described two techniques. The first was inspecting dependencies within the application itself. At Indeed, we leverage our status library to do this. The second was to enable services to be checked, independent of the application.

In this post, I’ll demonstrate how to use my service-precheck initialization container (built off of the Alibaba blog post) to ensure upstream systems are up before attempting to start a downstream system.

Moving to a Service Mesh

Similar to my talk on gRPC Java but with more of a focus on the addition of the service mesh.

Using docker-buildx for Multi-architecture Containers

When you build a container image, it’s typically only built for one platform (linux) and one architecture (amd64). As the Internet of Things continues to grow, the demand for more arm images increased as well. Traditionally, in order to produce an arm image, you need an arm device to do the build on. As a result, most projects wind up missing arm support.

BuildKit provides emulation capabilities that support multi-architecture builds. With BuildKit, you build container images across multiple architectures concurrently. This core utility backs docker buildx, a multi-architecture build utility for docker. In this post, I’ll discuss why you should produce multi-architecture container images and demonstrate how to use docker buildx to do it.

Moving Licenses - Apache 2.0 to MIT

Yesterday, I decided to switch the license that I apply to my personal projects. Many open source projects use the Apache 2.0 license. After reading through it a few times, I liked the level of coverage that it provided. It was however a bit wordy in my opinion. These were often simple little side projects that I was hacking on in my free time. After some discussion with others in the community and a few podcasts, I decided to make a switch.

Raspberry Pi Cluster Monitoring

In my last few posts, I talked a bit about my at home development cluster. Due to the flexibility of my cluster, I wanted to provide a monitoring solution that was valuable across each technology I use. In this post, I discuss how monitoring is setup on my cluster. I’ll walk through setting up each node, the Prometheus server, and the Graphana UI.

Raspberry Pi Cluster Setup

Previously, I talked about the different orchestration technologies that I’ve run on my Raspberry Pi cluster. That post was rather high level and only contained details relating to k3s. In this post, we’ll take a more in depth look at my cluster setup and my management process around it.

k3s on Raspberry Pi

Over the last few days, I’ve been revisiting Kubernetes on my Raspberry Pi cluster. I hope to share what I learned in the process and some of the tooling that I discovered along the way.

Easy Steps to a 64bit Raspberry Pi 3 B/B+

I was quite surprised to see how under documented installing a 64 bit operating system onto a Raspberry Pi is. Many articles out there talk about needing to compile Linux, which sounds oh-so-pleasant. One day, I stumbled across a 64bit OpenSUSE version that was compatible, but the installation instructions required a Linux OS to be done properly. Since I primarily work on OSX, this presented yet another barrier.

After a lot of searching around, I finally found a straight forward and simple way to do it.

Moving to grpc-java

Historically, Indeed has used Boxcar (Indeed’s proprietary framework) to build distributed systems. Over the last year, we have been shifting several of our systems to use gRPC. The first question product teams often ask is "How does gRPC compare to Boxcar?" In this presentation, I put the two frameworks head to head and present the results. I show how my team established some common workloads and gathered metrics to better inform other engineers. We learned a lot about how to optimize the gRPC Java library when performing this analysis. In closing, I present the lessons that we learned performance tuning gRPC services and how you can leverage this information for your own services.