#aws
Tag

Setting up GPU Data Science Environments for Hackathons

Originally published on the RAPIDS AI blog on August 13th, 2019.

Background

In my first week working at NVIDIA, I have been spending some time with my previous colleagues at the Met Office to explore how the two organizations can collaborate.

Exploring Dask and Distributed on AWS Lambda

I spent some time this week exploring whether it would be possible to run Dask and Distributed on a function as a service platform like AWS Lambda.

Adaptive Dask clusters on Kubernetes and AWS

Originally published on the Met Office Informatics Lab blog on July 21st, 2017.

Introduction

This article assumes a basic understanding of Amazon Web Services (AWS), Kubernetes, Docker and Dask. If you are unfamiliar with any of these you should do some preliminary research before continuing.

Moving large volumes of data to S3

Originally published on the Met Office Informatics Lab blog on April 20th, 2017.

We just moved ~80TB of data to S3 (stay tuned to hear what we’re doing with it).

A note on AWS disk performance testing

Here is an interesting note on testing the disk performance of your AWS instances. Before you can acurately test the performance of your EBS disk you need to read all the sectors of the disk at least once if that disk was created from a snapshot.

Amazon S3: s3cmd put ([Errno 32] Broken pipe)

Recently I decided to use Amazon’s S3 as another location to store some of my server backups. However I found when testing that I was unable to upload my backup tarballs to S3. I ended up with the following errors.