Weather and clouds

Posted by Johan on Tuesday, May 31, 2011

Ever since I wrote my master thesis I have had an interest in weather. My thesis was about metadata updates in a real time database at the Swedish Meteorological and Hydrological institute. The thesis was mainly about weather stations that deliver observations used when running weather models.

After my thesis I ended up as a consultant but for the last 2,5 years I’m back at SMHI doing various work, mostly backend. I have worked with real time deliveries of weather along with processing, visualisation etc. I worked with Diana (diana.met.no), a weather workstation software developed by Met.no.

Weather is a good thing since it is to interest to almost everyone. Telling someone that you’re a system developer is not a good conversation starter but saying that you work with weather will at least lead to a couple complains about how bad the forecasts are. From a system engineering point of view weather is good since there are huge quantities of data to process and it is a good source for testing new technologies.

Last fall I did a couple of tests using Python + CUDA + GRIBAPI for reading grib data (gridded binary is the format used for weather) and using CUDA to run marching squares and render images. As always there are to little spare time to finish all the small projects but at least I got a feel for CUDA and I hope to be able to use it in the future.

On to the clouds. 2 years ago I went to EclipseCon and learned a lot about EC2, both the founder of Smugmug and Stackoverflow praised it so after coming home I started using it at work. Today we have a couple of machines running in EC2 together with S3 for storage and RDS for databases. My primary server at home has handed over a lot of the work to an free tier EC2 instance.

I started using Python last fall and I like it a lot, being a shell scripting ninja I like regular expressions and {ba,k,c}sh but reusing scripts is not that easy but Python is perfect for this.

Getting to the point, my last project is to learn Google App Engine and I will do this by building a weather viewer using free weather data from NOAA. Python and probably some shell scripting will download the data and upload it to GAE and then I will use Django to render a site with Google Maps and my weather on it.

I want to build a free service that display the raw weather and I will build some kind of API so that people can get forecast values for latitude and longitude points.

Why? Mostly to learn Google App Engine but also since I know that there are people in need of such an API.