#kubernetes
Tag

A beginner's guide to managing Kubernetes resources in Python with kr8s

Mar 11, 2024 5 minute read #python, #kubernetes, #tutorial, #kr8s

Managing Kubernetes resources with Python has never been easier thanks to the kr8s Kubernetes client for Python.
Comparison of kr8s vs other Python libraries for Kubernetes

Sep 4, 2023 11 minute read #kubernetes, #python, #kr8s, #lightkube, #pykube-ng, #kubernetes-asyncio

I’ve been working on kr8s for a while now and one of my core goals is to build a Python library for Kubernetes that is the most simple, readable and produces the most maintainable code. It should enable folks to write dumb code when working with Kubernetes.
Livestream notes: Replacing aiohttp with httpx in kr8s

Jul 4, 2023 1 minute read #coding, #python, #kr8s, #httpx, #asyncio, #trio, #aiohttp, #kubernetes

This post will be updated with notes from the livestream throughout the day.

Today I will be streaming some open source code refactoring. Come and join in on Twitch!. Don’t forget to say hi in the chat 😊.
Introducing kr8s, a new Kubernetes client library for Python inspired by kubectl

Jun 19, 2023 7 minute read #project, #kubernetes, #python, #kr8s, #kubectl-ng

For the last few months I’ve been tinkering with a new Kubernetes client library for Python called kr8s.
Debugging Data Science workflows at scale

May 12, 2023 15 minute read #python, #dask, #kubernetes, #apache-beam, #google-cloud, #google-kubernetes-engine

The more we scale up our workloads the more we run into bugs that only appear at scale. Reproducing these bugs can be expensive, time consuming and error prone. In order to report a bug on a GitHub repo you generally need to isolate the bug and come up with a minimal reproducer so that the maintainer can investigate. But what if a minimal reproducer requires hundreds of servers to isolate and replicate?
Running Jupyter in your Dask Kubernetes cluster

Mar 10, 2023 2 minute read #dask, #kubernetes, #jupyter

Did you know that the Dask scheduler has a --jupyter flag that will start a Jupyter server running within the Dask Dashboard?
Oversubscribing GPUs in Kubernetes

Feb 7, 2023 5 minute read #kubernetes, #gpus, #scheduling

Sometimes I want to oversubscribe the GPUs in my Kubernetes cluster. This is especially useful when I’m developing but could also be useful in light workloads where you have ample GPU memory and don’t mind the occasional failure.
Quick and dirty way to pre-pull container images on Kubernetes

Feb 2, 2023 2 minute read #kubernetes, #containers

Sometimes when I give live demos with Kubernetes clusters I want to make sure that the container images I’m going to use are already pulled onto all of the nodes in my cluster. The last thing I want is for a Pod to be created to then sit in a Pending state while an image is pulled, especially given how large containers can be in the Data Science space.
How to check your NVIDIA driver and CUDA version in Kubernetes

Aug 12, 2022 3 minute read #kubernetes, #nvidia-driver, #cuda

When using GPUs with Kubernetes it can be important to know which driver and CUDA versions are installed on the nodes.
Using Dask on KubeFlow with the Dask Kubernetes Operator

Jul 27, 2022 10 minute read #python, #dask, #kubernetes, #kubeflow, #operator, #data-science

Kubeflow is a popular Machine Learning and MLOps platform built on Kubernetes for designing and running Machine Learning pipelines for training models and providing inference services. It has a notebook service that lets you launch interactive Jupyter servers (and more) on your Kubernetes cluster as well as a pipeline service with a DSL library written in Python for designing and building repeatable workflows. It also has tools for hyperparameter tuning and running model inference servers, everything you need to build a robust ML service.
Goodbye Docker Desktop for Mac, Hello Colima

Jan 31, 2022 6 minute read #docker, #colima, #docker-desktop-for-mac, #kubernetes, #kind

Today is the deadline for the license changes to Docker Desktop for Mac and Windows. This means that if you are employed at a company with more than 250 employees or your company makes more than $10m you need to start paying a subscription to continue using Docker Desktop.
Docker Desktop for Mac alternatives for developers

Jan 28, 2022 15 minute read #docker, #kubernetes, #kind, #podman, #minikube, #docker-desktop

In a couple of days Docker will begin charging employees of companies with >250 employees to use Docker Desktop. I have no problem with paying for software that brings me value, but you wouldn’t believe how complex it can be for large companies to sign employees up to subscription services. Paperwork everywhere! To avoid this I’m evaluating alternatives for Docker Desktop to use on my MacBook.
Running Kubeflow inside Kind with GPU support

Jan 27, 2022 8 minute read #community, #open-source, #kubernetes, #kind, #kubeflow, #gpus

This week I’ve been playing around with Kubeflow as part of a larger effort to make it simpler to use Dask and RAPIDS in MLOps workflows.
Quick hack: Adding GPU support to kind

Jan 25, 2022 4 minute read #community, #open-source, #kubernetes, #kind, #golang

This post has been superseded with this tutorial that no longer requires any code changes. Please read that instead.
How to create a Helm chart repository with Chartpress, Travis CI, GitHub Pages and Jekyll

Oct 28, 2019 19 minute read #kubernetes, #helm, #github-pages, #jekyll

Helm has become a pervasive tool in the Kubernetes community for packaging, managing, upgrading and distributing applications. It uses a packaging format called charts which are a collection of templates that describe Kubernetes resources and can be configured by the user.
How to merge Kubernetes kubectl config files

Oct 15, 2019 1 minute read #kubernetes, #cli, #tips

Sometimes when working with a new Kubernetes cluster you will be given a config file to use when authenticating with the cluster. This file should be placed at ~/.kube/config. However you may already have an existing config file at that location and you need to merge them together.
Debugging Kubernetes PVCs

Aug 3, 2018 2 minute read #kubernetes, #persistent-volume-claim, #debugging Archive

Sometimes I find that something goes wrong in a container and some data stored in a persistent volume gets corrupted. This may result in me having to get my hands dirty and have a poke around in the filesystem myself.
Adaptive Dask clusters on Kubernetes and AWS

Jul 21, 2017 7 minute read #dask, #aws, #kubernetes, #autoscaling Archive

Originally published on the Met Office Informatics Lab blog on July 21st, 2017.

Introduction

This article assumes a basic understanding of Amazon Web Services (AWS), Kubernetes, Docker and Dask. If you are unfamiliar with any of these you should do some preliminary research before continuing.

#kubernetes Tag

Introduction

#kubernetes
Tag