And the verdict is ... GFS FAIL!

By: Stu Ostro , 2:07 AM GMT on January 18, 2013

Share this Blog
13
+

Early today was the time of when this GFS model temperature departure forecast was for, which I featured in my "More Than 40 Below Average in Peoria?" blog a couple weeks ago.




This is how it turned out ...




The low temperature this morning in Peoria was 28F, 11 degrees above average, not more than 40 below average. That's a bust of >50 degrees.

This was the ECMWF ensemble mean which I included in the January 2 entry. Certainly not perfect -- even it was too aggressive with the western cold, and conversely it didn't reflect the pocket of cold air associated with the current southern storm system -- but its forecast overall was much closer to reality than that operational GFS output.




And the long-range GFS was up to its tricks again recently. Ugh. Caveat emptor.

This forecast made a few days ago for next week had an 850 millibar temperature of <-30C over the Ohio River, which is about as low as it ever gets there (such as during the great arctic outbreak of January 1985).

Not.

Reader Comments

Comments will take a few seconds to appear.

Post Your Comments

Please sign in to post comments.

Sign In or Register Sign In or Register

Not only will you be able to leave comments on this blog, but you'll also have the ability to upload and share your photos in our Wunder Photos section.

Display: 0, 50, 100, 200 Sort: Newest First - Order Posted

Viewing: 16 - 1

Page: 1 — Blog Index

16. thomascuy
12:37 AM GMT on September 13, 2013
The forecast surface state (snow covered or not) has a lot to do with the forecast lows. If the model calls the snow cover wrong then large errors generally ensue. To call a 15 day forecast a fail, though, is like shooting fish in a barrel. They're generally all wrong past day 10 if not earlier. Having said that, there's pretty good agreement on a dominant Hudson's Bay upper low for the next few weeks...lookin' cold.

Download Soal CPNS 2013 dan Pendaftaran CPNS 2013 | Lowongan CPNS 2013 | soal cpns 2013 | soal cpns 2013 pdf
Member Since: September 13, 2013 Posts: 0 Comments: 0
14. WunderAlertBot (Admin)
10:27 PM GMT on February 02, 2013
stuostro has created a new entry.
13. georgevandenberghe
9:54 PM GMT on January 25, 2013
The fact that we're even arguing about the skill of a deterministic integration 16 days out (still near zero but tantalizingly believable sometimes) and that the deterministic forecast actually has some skill out to day 7 is amazing compared with what we thought would be possible 30 years ago when I was a student.
Member Since: February 1, 2012 Posts: 19 Comments: 2817
12. Progster
3:09 AM GMT on January 25, 2013
Quoting stuostro:
Progster, I know what you're saying, and in some situations that's the case, but not this one. Snowcover or lack thereof does not account for a >50 degree error, and the GFS busted synoptically - not nearly as deep a trough - as it predicted a 500 mb height over Peoria of ~529 dm and the actual height was ~555, what the ECMWF ensemble mean had.

And while yes, models are usually wrong in the details at 10+ days, it's not like the GFS repeatedly shows too much warmth. Following a recent FB post in a broadcast meteorology group about the GFS's apparent cold bias in including in that market (RDU) in NC, I plotted the GFS's forecast temps for New Year's Day there (below). That's just one snapshot, not a scientific analysis of systemic biases, but is consistent with the anecdotal impression that I and others have had in this and recent winters. (Also note that while overall there was less error when the forecast time frame shifted to the GFS's higher resolution, interestingly the largest deviation was within that time frame, at 180 hrs.)

And what's also happening is that there's then a lot of reaction on social media to each model run, in fact a major market TV meteorologist posted a 300 hr GFS forecast and said that this could be one of the coldest Christmases on record in the East (temps there were actually above average on Christmas Day), and even the natural gas market can react to operational long-range runs of the GFS.

So it's not just a random or snowcover thing, it's a bigger problem involving meteorology and communication, further fueled in this and last winter by sensationalistic claims about things such as statospheric warming (which I might blog about at some point!).



It would be interesting if you could plot forecast snowcover from the long range GFS in the same way you plotted forecast versus observed temperatures - although I agree in this case it was a long wave problem, not a surface problem. In fact I'm pretty impressed - the GFS was well behaved out to hour 156 so not terrible for 7.5 days followed of course the big Rossby error. Those kinds of departures from reality are of course the reason we prefer ensembles for forecasts really just beyond a few days. I have wondered about the ECMWF's superior skill...I think It may have alot to do with its superior analysis method whereby the initial states for ensemble members are created using an ensemble of perturbed analyses. Helps to smooth out the observation error.

Speaking of cold bias, the 850 temp forecast for NYC from today's 12Z ECMWF ensemble mean valid 00Z on the 30th is about 5.5C lower that the GFS. 2.5C vs 8. The Euro has the long wave trof swinging east much faster than the GFS even by day-6. Strangely enough, they're virtually identical by day-10.
Member Since: September 4, 2007 Posts: 0 Comments: 493
11. Astrometeor
4:50 AM GMT on January 23, 2013
Stu Ostro:

In response to your question yesterday, I have offered a response.

Hope this answers you

Member Since: July 2, 2012 Posts: 103 Comments: 10816
10. Stu Ostro , Senior Meteorologist
8:28 PM GMT on January 21, 2013
STLweatherjunkie, like with Progster's comment, I know what you're saying, and nevertheless what I just posted applies to your point too. While you're right, it should be no surprise that the ECMWF ensemble provides more valuable information than the GFS operational runs, alas, there's too much stuff out there that suggests that it is one, and so it becomes a necessary and fair comparison. And it's not just that it's ensemble methodology, it's the ECMWF's superior initialization and resolution.
Member Since: August 24, 2012 Posts: 32 Comments: 24
9. Stu Ostro , Senior Meteorologist
8:23 PM GMT on January 21, 2013
Progster, I know what you're saying, and in some situations that's the case, but not this one. Snowcover or lack thereof does not account for a >50 degree error, and the GFS busted synoptically - not nearly as deep a trough - as it predicted a 500 mb height over Peoria of ~529 dm and the actual height was ~555, what the ECMWF ensemble mean had.

And while yes, models are usually wrong in the details at 10+ days, it's not like the GFS repeatedly shows too much warmth. Following a recent FB post in a broadcast meteorology group about the GFS's apparent cold bias in including in that market (RDU) in NC, I plotted the GFS's forecast temps for New Year's Day there (below). That's just one snapshot, not a scientific analysis of systemic biases, but is consistent with the anecdotal impression that I and others have had in this and recent winters. (Also note that while overall there was less error when the forecast time frame shifted to the GFS's higher resolution, interestingly the largest deviation was within that time frame, at 180 hrs.)

And what's also happening is that there's then a lot of reaction on social media to each model run, in fact a major market TV meteorologist posted a 300 hr GFS forecast and said that this could be one of the coldest Christmases on record in the East (temps there were actually above average on Christmas Day), and even the natural gas market can react to operational long-range runs of the GFS.

So it's not just a random or snowcover thing, it's a bigger problem involving meteorology and communication, further fueled in this and last winter by sensationalistic claims about things such as statospheric warming (which I might blog about at some point!).

Member Since: August 24, 2012 Posts: 32 Comments: 24
8. STLweatherjunkie
7:41 AM GMT on January 21, 2013
While I appreciate your verification of the shenanigans the long range GFS plays, because no model is ever right they are just approximations of reality from the beginning ... Additionally, it really is not fair to compare a deterministic GFS solution to the ensemble mean of a suite of 52 individual members of the already superior European model. Chaos theory applies to meteorology so it should be no surprise that the ensemble mean is closer to reality on a 10 day forecast than one deterministic solution.
Member Since: September 9, 2001 Posts: 0 Comments: 1042
7. georgevandenberghe
4:09 PM GMT on January 20, 2013
I think the GFS was correct that we would transition to a below normal temperature pattern in the East the second half of January. It was very fast on when, at least a week. I would agree with Stu that the arctic outbreak forecast for PIA was a complete bust and the one for the East this week is not the same one seven days late. But the idea that the cold air would eventually come down rather than staying in Canada all month (which does happen sometimes) was correct.

I'm not a professional forecaster (not good enough,no question and no regrets) but do follow the weather and use my Met degree for NWP improvement in the computer trenches.
Member Since: February 1, 2012 Posts: 19 Comments: 2817
6. juslivn
3:51 AM GMT on January 20, 2013
This may be a dumb question, but:
It is supposed to be low of 7 degree temps in Chicago tomorrow night (Sun) and windchills of -8. Peoria will be low of 9 degrees tomorrow and wind chills -4. It is dropping rapidly now. Monday in Minneapolis/St. Paul they are calling for -32 degree wind chills.

Was it that far off?
Member Since: August 20, 2009 Posts: 91 Comments: 10392
5. Progster
4:43 PM GMT on January 18, 2013
The forecast surface state (snow covered or not) has a lot to do with the forecast lows. If the model calls the snow cover wrong then large errors generally ensue. To call a 15 day forecast a fail, though, is like shooting fish in a barrel. They're generally all wrong past day 10 if not earlier. Having said that, there's pretty good agreement on a dominant Hudson's Bay upper low for the next few weeks...lookin' cold.
Member Since: September 4, 2007 Posts: 0 Comments: 493
4. georgevandenberghe
2:59 PM GMT on January 18, 2013
I cynically observed as a student at PSU that if we got all the
snow that was forecast with the "wishful approximation" we'd
start a new ice age.

That said, the Arctic air was there and it was within the envelope
of possibilities it could have come down the way the GFS said. The GFS did
a similar thing for the DC area about a week ago. It now looks like we will get an arctic outbreak of typical seasonal severity with highs just below freezing and lows near 10. This is respectable over bare ground
but it has a return period here of a year, not several years or decades.


I haven't taken the time but individual ensemble members (NCEP runs 21) will probably also do this if you want to fish for the solution you want to fantasize about rather than one that is likely to actually happen.

If you want the solution that's likely to actually happen wait until the ensembles are either consistent or a series of deterministic forecasts
are. This often happens when we are still 5-7 days out.

Thanks Stu for the followup.
Member Since: February 1, 2012 Posts: 19 Comments: 2817
3. whitewabit (Mod)
3:20 AM GMT on January 18, 2013
Living in Peoria I have noticed that all the models seem to predict cooler temperatures then we eventually get in the winter ..

Precipitation forecasts are always too high also .. if we would have gotten all the rain that was forecast last year we wouldn't have been in drought !!
Member Since: August 17, 2005 Posts: 372 Comments: 34251
2. Barefootontherocks
2:43 AM GMT on January 18, 2013
Thanks for following up on this, stuostro.
Beautiful winter day in central Oklahoma today with a high of 55, 3 or 4 degrees above average. Better than that -1 or so would have been with a repeat of the January 1985 temps.
Member Since: April 29, 2006 Posts: 162 Comments: 20241
1. Astrometeor
2:11 AM GMT on January 18, 2013
And people say that computers don't have fun....
Thanks Stu!

Actually was kinda hoping for that to come true, would've been interesting for a 16 year old to say the least.
Member Since: July 2, 2012 Posts: 103 Comments: 10816

Viewing: 16 - 1

Page: 1 — Blog Index

Top of Page

About stuostro

Senior meteorologist at The Weather Channel. Proud to be a weather-obsessed weather geek. Would be a DJ if not a meteorologist.

Local Weather

Clear
78 °F
Clear