ForecastAdvisor Weather Forecast Accuracy Blog
Friday, March 6, 2009
How Good Are Week Out Weather Forecasts?
I am often asked if forecast for a week or more out is worth anything. Worth is a hard thing to measure, but skill is easier. And if there is no skill, then likely the forecast would not be worth much.
At ForecastWatch we use two measures of skill. How much better is a weather forecast than climatology, and how much better is a weather forecast than persistence. Those are both unskilled forecasts, in that it doesn't take any skill to create them. The climate weather forecast says that the weather will be exactly like the 1971-2000 average. A persistence forecast says the weather however many days in the future will be exactly like today.
The easiest way to measure skill, at least for temperature forecasts, is to compare average absolute error. The difference between the temperature forecast, and the actual temperature that occurred, is the error. Take the positive difference, average it over a lot of forecasts, and that is a measure of how good a forecast is at predicting the temperature. An average absolute error of 3 degrees is generally better than an average absolute error of 6 degrees.
The chart below shows the average absolute high temperature forecast error for all of 2008, for all stations, and all providers. Lower is better. Red is the unskilled persistence forecast. Yellow is the climate average forecast. The green line is the average of all weather forecast providers. Lower is better. The climate average forecast is a straight line because it doesn't matter how many days you forecast out, the climate average forecast for a given date is always the same.
What sticks out on this graph is the intersection of the yellow climate average line with the green forecasters' line at nine days out. What that means is that the weather forecasters predicted high temperatures nine days out just as well (or just as badly) as using the 1971-2000 climate average high temperatures as the forecast prediction for each location.
What is even more interesting is that for high temperature forecasts greater than nine days out, weather forecasters (at least the ones ForecastWatch tracks) do WORSE than the climate average forecast. This means that you would do better just looking at historical average temperatures when determining the temperature more than nine days out.
The American Meteorological Society recently said as much. Every ten years, they release an information statement describing the current state of weather science. The statement released August 2007, made this observation:
"The current skill in forecasting daily weather conditions beyond eight days is relatively low."
The AMS is very wise. And in fact, for this sample, composed of nearly 18 million forecasts over the year from multiple national providers, there is no skill in daily weather forecasts beyond nine days.
On the positive side, meteorologists always did better than the persistence (tomorrow is like today) forecast. There are many other interesting things about this graph that I'll talk about later.
December 2005 January 2006 March 2006 June 2006 July 2006 August 2006 September 2006 October 2006 November 2006 December 2006 January 2007 February 2007 March 2009 September 2009 March 2010 April 2010 February 2011 April 2011 June 2011 February 2012 September 2012 June 2013 October 2013 February 2014 June 2016 Current Posts