Author Archives: (zaphinath)
Karpenter vs ECK Elasticsearch
Fixing ECK Elasticsearch Cluster Issues with Karpenter and PodDisruptionBudgets Recently, we ran into an issue with our Kubernetes cluster where Karpenter was updating AMIs. While this normally helps keep nodes up-to-date, in our case it was replacing nodes faster than Elasticsearch could recover. The result was a large Elasticsearch cluster that ended up in a […]
Solar Panels For EV Cars
Solar Car Dream When I was a kid, I often dreamed of having a car that was fully electric and could run on solar power in real time. Imagine never needing to stop to recharge—just endless travel powered by the sun. A vehicle like this would allow you to roam the world without ever worrying […]
ECK – Unify Kibana and Elasticsearch Versions
Managing Elasticsearch and Kibana Versions with ECK, ArgoCD, and Kustomize We use the ECK operator to manage our Elasticsearch clusters. It does a great job and integrates well with our GitOps workflow. All of our deployments run through ArgoCD, and we use Kustomize to tie the pieces together. This works smoothly in most cases, but […]
AI Longhorn Destruction + Solution
Longhorn PVC Mounting Issue and Backup Setup Recently I ran into a persistent error with Longhorn while trying to attach a PVC. The error kept repeating and initially, I thought I needed the v2 engine—but it turns out the real culprit was multipath being created. The Longhorn KB article on troubleshooting multipath explained exactly how […]
Enabling Windows nodes inside an EKS cluster on AWS.
Description We recently went about adding windows nodes for some legacy dotnet stacks at Ancestry. As part of this we followed the AWS documentation to enable windows. We also use Karpenter to handle the scheduling and decided to make a new provisioner for just windows. After following the docs a new windows node came up […]
Birth of a Liberal
Abstract I just want to begin by expressing my gratitude for this country where I live, The United States of America. It really is a land of immense freedom and opportunity. I love the constitution and feel that it really is an inspired document. I consider myself somewhere in the middle of the political spectrum. […]
Java New Relic Agent Increases Memory Footprint
Overview We recently have been upgrading our java stacks to have the latest new relic agent 6 and 7 to incorporate distributed tracing into our environments. In testing the various versions of the new relic agent six we found that our stacks all took a performance hit. We increased our memory usage and increased our […]
HSEARCH400080: Unable to detect the Elasticsearch version running on the cluster
Description I recently decided to use the elasticsearch and hibernate bindings in java. Hibernate is already a very good ORM and has come a long ways in the last several years, but the integration with elasticsearch and lucene make it an amazing combination. It cuts down on the maintenance of keeping an elasticsearch index up […]
Move from Quay to AWS ECR Automation
Why the move We recently have decided to move all the docker repositories from a private registry called Quay to ECR. Quay uses robot tokens in order to authenticate the registry in order to push images. We have found it is more cost effective to just use ECR instead of pay for and maintain Quay […]
How to Set Master Nodes in Elasticsearch
Problem In order to horizontally scale an elasticsearch cluster in the cloud we need to make sure we don’t remove the master nodes if we scale the cluster down during times of less usage. We do this in Amazon Web Services by having two groups of compute instances. The first block is equal to the […]