Everybody can be a weather forecaster. In fact, my two daughters, ages seven and nine, can predict the weather a year from now.

I ask them, "What is the temperature going to be like next winter?". And they answer "Cold."

They are right. So I ask them, "What about next summer?". And they correctly answer, "Warm."

They are right because temperature tends to follow averages. My children reason that because last winter was cold, it will be cold this winter. Because temperatures tend to an average, people and businesses can use that information to plan. They use that information to make better decisions. Municipalities in the northeast don't order road salt for July. And retailers in the upper plains don't display swimsuits in November. You don't need a meteorologist on staff to make those types of decisions.

If only it were that easy. As we all know, while temperatures tend to follow averages, rarely does temperature stay average. Temperatures swing wildly from above average to below average and back again, and in sometimes unpredictable ways. It's these extremes and these changes, where temperature isn't normal, that make weather interesting and keep meteorologists employed.

An electric utility makes long term decisions about how much electricity to produce based on past averages. They know that, for example, August electricity demand is higher than May electricity demand because there are more air conditioners running in August than May. This is generally true because August is warmer than May (at least where I live). But what if a meteorologist told the electric utility in August that next week in August would be much hotter than average.

They would want to know because they would want to make sure they generated enough electricity to power all those air conditioners working overtime. Because if they didn't, the alternative is brown-outs, not enough electricity to go around.

So it is often of greater value when a forecaster can predict weather extremes. This goes not only for temperature, but also other extremes: tornados, heavy winds, flooding rain. Is there a way we can look at how well forecasters predict temperature extremes?

Normally, when a weather forecast's accuracy is calculated, you take the forecast and look forward to the actual. You see how well the forecast predicted the actual temperature. If you want to look at extreme temperatures, though, you want to take the actual temperatures and look back toward the forecasts. You want to figure out how accurate forecasts are when the actual is some amount above or below the normal average expected temperature.

For example, you can look at all forecasts made for a date where the temperature was ten degrees below normal, and see how well they predicted that ten degree below normal temperature. In fact, you can do that for all days, grouped by how different the actual temperature was from the average expected climate normal. If you graph forecast temperature error grouped by that difference, you get the chart below.

Click here for a larger version of this graph.

This graph shows average error, or bias, for high temperatures. What it shows is the tendency of a forecast to be either too high or too low. If the bias were zero, that would mean that on average, forecasts were equally too high or too low, or they all were right on. If bias was negative, it would mean that, on average, forecasts tended to under-predict temperature. That is, on average, the forecasts tended to predict a lower temperature than what actually occurred. Conversely, if bias were positive, it would mean that, on average, forecasts tended to over-predict temperature, predicting a higher temperature than what actually occurred.

The first thing that you notice about the graph is that bias is not the same for all actual temperature differences from normal. When temperatures are normal, or near normal, bias is nearly zero. There is an equal chance that a temperature forecast will be either too high or too low. But for the extremes, bias tells a different story. When the actual temperature is well below normal, forecast bias tends to be positive. Forecasts tend to be too warm when the actual temperature is colder than normal. And on the other side of the graph, bias is negative. Forecasts tend to be too cold when the actual temperature is warmer than normal.

The further out the forecast, the steeper the bias' slope. That means that forecasts tend to be more conservative than actual temperatures, and that conservatism grows as the forecast is for a time further into the future. That part is expected, since a nine day out forecast is going to need to rely more on climate normals than a one day out forecast because of our current inability to accurately model instabilities the further out in the future we are trying to predict.

But that bias doesn't tell us how well forecasts predict high temperature, only how it's trending. Two forecasts, one ten degrees too high, one ten degrees too low have a bias of zero, but average ten degrees wrong. That is called absolute error. The graph below shows high temperature absolute error plotted against how far the actual high temperature was from the average climate normal.

Click here for a larger version of this graph.

What this shows is that high temperature weather forecasts tend to be most accurate when the temperature is average, right near the normal climate. That makes sense, because if you always predicted the normal average temperature, which doesn't take any skill, you would have an error of zero for days when the temperature was exactly the climate average temperature.

But if you always predicted the climate normal, your error would always equal the difference between the actual and the climate normal. So on days when the temperature is six degrees below normal, your error would be six degrees. If you look at the error curves, you can see that forecasters do better than that. But error does significantly increase when the actual temperature is further from the climate normal.

What this ultimately means is that weather forecasters don't do an equally good job of forecasting for all temperatures. If a forecaster says they have an average absolute error of 3 degrees, you cannot assume that means the forecaster can predict temperature extremes that well. And sometimes, what you are most interested in are those extremes.

ForecastWatch helps businesses and individuals understand and place value on weather forecasts so that they can be more accurately used quantitatively in modeling and prediction. We'll talk more about these graphs and what they mean in a future post.