#data-science
Tag

Using Dask on KubeFlow with the Dask Kubernetes Operator

Jul 27, 2022 10 minute read #python, #dask, #kubernetes, #kubeflow, #operator, #data-science

Kubeflow is a popular Machine Learning and MLOps platform built on Kubernetes for designing and running Machine Learning pipelines for training models and providing inference services. It has a notebook service that lets you launch interactive Jupyter servers (and more) on your Kubernetes cluster as well as a pipeline service with a DSL library written in Python for designing and building repeatable workflows. It also has tools for hyperparameter tuning and running model inference servers, everything you need to build a robust ML service.
Hypothetical datasets

May 8, 2019 14 minute read #data-science, #netcdf, #s3, #atmospheric-sciences, #big-data Archive

In Theo’s previous posts on storing high momentum data and its accompanying metadata we get some interesting insights into the future of cloud based data storage. In this post I’m going to cover how we are working with today’s NetCDF-based challenges, by making assumptions!
Intro to Earth Information Workshop

Mar 12, 2019 6 minute read #data-science, #weather-data, #pangeo, #workshop Archive

This article was originally written for the the Met Office workshop run at the Intro to Earth Information event on the 12th of March 2019.

Using Dask on KubeFlow with the Dask Kubernetes Operator