We're changing our WunderBlogs. Learn more about this important update on our FAQ page.

Challenging Bill Proenza's QuikSCAT numbers

By: Dr. Jeff Masters , 5:04 PM GMT on July 04, 2007

A political storm engulfed the National Hurricane Center this week, with a majority of the senior hurricane forecasters calling for Bill Proenza's removal as director. The most visible issue revolved around the extraordinary focus on the aging QuikSCAT satellite. The public argument put forth by Mr. Proenza was that QuikSCAT data was so vital to hurricane track forecasting that without it, track forecast errors would increase significantly, leading to larger warning areas and increased costs for evacuation and emergency planning.

Focus on QuikSCAT--out of proportion?
On March 16th Proenza went public with the QuikSCAT concerns and associated statistics for the first time, stating that "two- and three-day forecasts of a storm's path would be affected. The two-day forecast could be 10 percent worse while the three-day one could be affected up to 16 percent," with the conclusion," that would mean longer stretches of coastline would have to be placed under warnings, and more people than necessary would have to evacuate." As a result of these comments, a perception arose in the public and among lawmakers that without QuikSCAT, NHC would not be able to provide accurate hurricane forecasts. Legislation was hastily introduced into both the House and Senate to provide an immediate replacement for the $375 million satellite.

Proenza's statements raised several questions: 1) Why the focus on track forecast errors in landfalling situations, when QuikSCAT was widely known to be used in intensity forecasting and for tropical cyclones too far at sea to be accessed by the Hurricane Hunters? 2) Could such specific and significant gains in track forecast error truly be attributed to QuikSCAT? Where did these numbers come from, and why was no uncertainty being attached to them?

Since QuikSCAT data became available, starting in 1999, average track errors for 48-hour and 72-hour forecasts have been reduced by 43 miles and 62 miles respectively. Fully one quarter of this improvement was being attributed by Proenza to QuikSCAT. This was an extraordinary performance increase to attribute to one satellite, and seemed doubtful.

We find out where the QuikSCAT numbers came from
In mid-June, Margie Kieper and I asked Proenza to comment on how he got his QuikSCAT numbers. He cited an unpublished study, "A Two Season Impact Study of Four Satellite Data Types and Rawinsonde Data in the NCEP Global Data Assimilation System", by Tom H. Zapotocny, James A. Jung, John F. LeMarshall and Russ E. Treadon. I contacted one of the authors, who informed me that the study was submitted for publication on January 26, 2007, and accepted for publication in the journal Weather and Forecasting on May 23, 2007. It will probably appear in the October-November time frame, according to the publisher. This raises an immediate problem, since only a privileged few are able to read unpublished research. This limits the possibilities for an informed debate on the issue, and basing important policy decisions on unpublished research is thus normally to be avoided. However, making accurate hurricane forecasts is important enough that such considerations can be excused. Proenza didn't give me any details on the study, other than the fact that QuikSCAT data improved 72-hour and 48-hour hurricane track forecasts by 16% and 10% respectively, for a select group of storms from the 2003 hurricane season. One of the authors graciously sent me a copy of the study, though, and after reading it, I had these observations:

1). The study looked at a very limited number of cases over a six-week period during 2003--only 19 cases were available for 72 hour forecasts. The 19 cases were not 19 storms, just 19 separate forecasts from the 4 hurricanes and 2 tropical storms that occurred during the 6-week study period. This sample is too small to draw definitive conclusions about the impact of the QuikSCAT on tropical cyclone forecasts. The two longest-lived storms during the test period were Fabian and Isabel, storms that spent the majority of their lifetimes far away from land. Since the quality of the observing network increases close to land, particulary when reconnaissance data from the Hurricane Hunters is available, it is reasonable to conclude that the impact of the QuikSCAT data for storms within 72 hours of landfall would be less than for the sample as a whole. The study was not primarily designed to study tropical cyclone track accuracy, so there was no separation out of the cases we really care about--storms 72 hours or less from landfall.

2). The study was done with only one model, the GFS. NHC official forecasts make use of several models, including the GFDL, UKMET, NOGAPS and ECMWF. Consequently, a change in the accuracy of a single model will have only a partial effect on NHC official forecast accuracy. As far as I know, there have not been studies done of the impact of QuikSCAT on tropical cyclone forecasts in the GFDL, UKMET or the ECMWF models. Past studies on the impact of dropsonde data from the Hurricane Hunters, however, show that the GFDL is less sensitive to these data than the GFS is.

3). When I attended the AMS hurricane conference in May 2006 in Monterey, I came across a poster presentation by Dr. Jim Goerss that evaluated the impact of QuikSCAT on the NOGAPS model. His study was far more comprehensive, and included 12 hurricanes, 5 typhoons, and 7 tropical storms from a 6-week period in 2004. The number of cases was 212 at 72 hours, eleven times as many as the study Proenza cites. Dr. Goerss found that QuikSCAT probably improved 24-hour track forecasts by 2.5% (90% confidence of this), but at all other forecast times (48, 72, 96, and 120 hours), QuikSCAT had no statistically significant effect (i.e., zero effect).

It is hard to compare the results from these two studies, since they used two different data assimilation systems. We do not know if they used all the data, or how they treated the vertical impact of the data. The uncertainties are high, and Proenza's simple statement that QuikSCAT data improves hurricane tracks forecasts by 10% and 16% is unreasonable, without at least making mention that these numbers are highly uncertain.

I believe that NHC official forecasts for landfalling storms in the Atlantic would not be significantly affected by the loss of the QuikSCAT satellite. I can't think of a hurricane scientist out there who would defend using a study with only 19 cases that didn't focus on landfalling storms, to make the case Proenza is making--particularly in light of the data from the unpublished Goerss study showing no effect of QuikSCAT data on NOGAPS model tropical cyclone track errors. Proenza should have at least attached some measure of uncertainty to his numbers, which he did not.

One could argue that the study cited by Proenza has undergone peer review, and is thus the only scientific study one can use to make arguments on QuikSCAT's effectiveness. The Goerss study has not been published in a journal, and has not undergone peer review. However, Proenza was making his QuikSCAT accuracy arguments in March, two months before the Zapotocny study he cited had been accepted for publication.

QuikSCAT misconceptions
The numbers pushed by Proenza have led to some potentially serious misconceptions about QuikSCAT. The Congressional Record has this to say about QuikSCAT:

"A single plane gathering data is like a tiny fishing line collecting data only along the single strand of the line. The satellite, on the other hand, provides rich, detailed data horizontally from one side of the storm to the other side, and vertically, from the ocean surface to the top of the storms swirling winds. The QuikSCAT is like a detailed MRI."

Well, QuikSCAT is not like an MRI, it just measures the ocean surface winds. In a letter written by Representatives Melancon and Klein in support of H.R. 2531, there are comments that data from the reconnaissance aircraft are inferior to the data from the QuikSCAT:

"Short-term options for replacing QuikSCAT include hurricane hunter aircraft, buoys, and foreign satellites--all of which will collectively produce inferior data."

There is not a hurricane forecaster anywhere that would trade hurricane hunter data for QuikSCAT. Lawmakers may start cutting aircraft reconnaissance with misconceptions like this. That would be a disaster.

I would hate to lose the QuikSCAT satellite, and have been calling for a replacement since before Mr. Proenza came on the job. QuickSCAT data is invaluable in identifying weak systems and in defining storm structure, particularly of outer wind radii of 34 knots and 50 knots. This is particularly true outside of the Atlantic, where there are no Hurricane Hunter flights, and in the Atlantic beyond where the Hurricane Hunters can reach. Track forecasts for tropical cyclones in the Pacific and Indian Oceans may benefit from QuikSCAT data, since Hurricane Hunter information is not available. QuikSCAT also helps identify when a tropical depression or tropical storm is intensifying.

Besides hurricanes, the QuickSCAT data is invaluable to the Ocean Prediction Center, which now issues hurricane force wind warnings for extratropical storms in the Atlantic and Pacific. Search and rescue missions, and the U.S. Navy also greatly benefit from QuikSCAT. QuikSCAT should be replaced, but not due to a rush knee-jerk reaction that will get us a replacement with old technology. NHC needs a "next-generation" scatterometer, one that has greatly improved capabilities to help tackle the structure and intensity problem. We should take our time, and deal with a gap in coverage, if it gets us an instrument that has higher resolution, higher saturation speed, and is not adversely affected by rain. Such a gap would not put the public at risk.

It greatly troubles me that the most visible and admired member of my profession has failed to use good science in his arguments for funding a replacement of the QuikSCAT satellite. The Director of the National Hurricane Center needs to be an able politician and good communicator, but being truthful with the science is a fundamental requirement of the job as well. Mr. Proenza has misrepresented the science on the QuikSCAT issue, and no longer has my support as director of the National Hurricane Center.

Other critical concerns--lost in the hubbub?
We strongly support many of the valid concerns Proenza has raised. Of particular concern are the slashing of critical research funding for the Joint Hurricane Testbed (JHT) from $1.7 million to $1 million, and the lack of adequate yearly increases to the National Hurricane Center budget. Both of these important concerns still remain to be addressed; they were quickly overshadowed by a frantic campaign by lawmakers to fund a new QuikSCAT satellite. The JHT provides the means for promising research to be tested in the NHC operational environment, usually resulting in a successful transition to an operational product at NHC. This program has been extremely successful, and its budget should have been increased, not slashed. As hurricane activity has increased dramatically over the last twelve years, NHC's budget should have increased accordingly, but it did not.

Proenza also raised legitimate concerns about NOAA's effort to promote their "Corporate Identity" by renaming the National Hurricane Center and National Weather Service. The new organizations would be called the "NOAA Hurricane Center" and the "NOAA Weather Service". He also justly complained about NOAA's plan to spend between $1.5 million and $4 million on a "bogus" 200-year NOAA anniversary celebration.

While wanting to take a neutral stand as to whether to call for Proenza's dismissal, Senior NHC Hurricane Specialist Lixion Avila clearly shares the concerns that have been put forth by the other senior forecasters Richard Pasch, James Franklin, and Richard Knabb, and former director Max Mayfield. Avila noted, "If I [was] the director of the hurricane center, I would not spend my time fighting for QuikSCAT--I would be fighting to make sure that the reconnaissance planes are always there." That leaves a vacationing Jack Beven as the only senior hurricane forecaster to not comment publicly on the issue. Max Mayfield has refrained from making public comments on the deteriorating situation these past months, but all of his comments in the Miami Herald article lend support for the hurricane forecast staff. Given his previous experience in the position of NHC Director and his successful tenure, his feedback counts tremendously.

With the busiest part of hurricane season just a few weeks away, expect a decision on Bill Proenza's tenure to be made soon.

Jeff Masters and Margie Kieper

Having lost the support of most of his senior forecasters, and having misrepresented the science on the importance of the QuikSCAT satellite on hurricane forecasts, it would be best for Mr. Proenza to step down as director of the National Hurricane Center.

--Jeff Masters


The views of the author are his/her own and do not necessarily represent the position of The Weather Company or its parent, IBM.

Reader Comments

Comments will take a few seconds to appear.

Post Your Comments

Please sign in to post comments.

Log In or Join

You be able to leave comments on this blog.

Display: 0, 50, 100, 200 Sort: Newest First - Order Posted

Viewing: 1764 - 1714

Page: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44Blog Index

1763. NorthxCakalaky
2:19 AM GMT on July 06, 2007
Posted By: HurricaneGeek at 2:19 AM GMT on July 06, 2007.

Stormy2day, 37?
ouch....lol

LOL, He might not have known what the required wind speed for a tropical storm was.
Member Since: December 31, 1969 Posts: Comments:
1762. HurricaneGeek
10:18 PM EDT on July 05, 2007
Stormy2day, 37?
ouch....lol
Member Since: Posts: Comments:
1761. Thundercloud01221991
2:17 AM GMT on July 06, 2007
you guys are funny
Member Since: Posts: Comments:
1759. NorthxCakalaky
2:17 AM GMT on July 06, 2007
tornadofan is that a team name?Or do you like tornadows alot?
Member Since: December 31, 1969 Posts: Comments:
1758. NorthxCakalaky
2:15 AM GMT on July 06, 2007
Posted By: tornadofan at 2:14 AM GMT on July 06, 2007.

Posted By: NorthxCakalaky at 2:10 AM GMT on July 06, 2007.

LOL, What a turn around..


You should see my turn around jump shot in basketball>.
" You Cant See Me " lol
Member Since: December 31, 1969 Posts: Comments:
1757. Stormy2day
2:16 AM GMT on July 06, 2007
hurry HG ...take it again before TC changes the questions and you will get a 100! (I started with a 37 ...LOL)
Member Since: Posts: Comments:
1756. Stormy2day
2:14 AM GMT on July 06, 2007
I got my 100! ::::::::putting my WUBA shirt back on:::::::
Member Since: Posts: Comments:
1755. HurricaneGeek
10:14 PM EDT on July 05, 2007
I got an 88% whoo hooo
Member Since: Posts: Comments:
1754. NorthxCakalaky
2:13 AM GMT on July 06, 2007
Posted By: Stormy2day at 2:12 AM GMT on July 06, 2007.

TC ...exactly! :-) But, I'm learning!

Im learning with you. :)
Member Since: December 31, 1969 Posts: Comments:
1753. tornadofan
2:12 AM GMT on July 06, 2007
Posted By: NorthxCakalaky at 2:10 AM GMT on July 06, 2007.

LOL, What a turn around..


You should see my turn around jump shot in basketball>.
Member Since: Posts: Comments:
1752. Stormy2day
2:12 AM GMT on July 06, 2007
TC ...exactly! :-) But, I'm learning!
Member Since: Posts: Comments:
1750. Thundercloud01221991
2:09 AM GMT on July 06, 2007
Posted By: Stormy2day at 2:09 AM GMT on July 06, 2007.

TC - okay, second time around - I got an 88%! See, I'm learning! Now, if you would stop changing the questions, I might be able to get a 100!


That is not fair you would memorize the answers
Member Since: Posts: Comments:
1749. NorthxCakalaky
2:09 AM GMT on July 06, 2007
Posted By: tornadofan at 2:09 AM GMT on July 06, 2007.

Posted By: StormW at 2:06 AM GMT on July 06, 2007.

Actually, it's not misspelled...it's a short way us Met's name a trough...you'll see it sometimes on the NHC surface analysis map, as well as the HPC surfcae maps at times.


Thanks for the education W. I enjoy your daily updates on your blog as well.


LOL, What a turn around..
Member Since: December 31, 1969 Posts: Comments:
1748. tornadofan
2:07 AM GMT on July 06, 2007
Posted By: StormW at 2:06 AM GMT on July 06, 2007.

Actually, it's not misspelled...it's a short way us Met's name a trough...you'll see it sometimes on the NHC surface analysis map, as well as the HPC surfcae maps at times.


Thanks for the education W. I enjoy your daily updates on your blog as well.
Member Since: Posts: Comments:
1747. Stormy2day
2:07 AM GMT on July 06, 2007
TC - okay, second time around - I got an 88%! See, I'm learning! Now, if you would stop changing the questions, I might be able to get a 100!
Member Since: Posts: Comments:
1746. weatherboykris
2:02 AM GMT on July 06, 2007
Posted By: WPBHurricane05 at 12:26 AM GMT on July 06, 2007.

I think Bill Proenza can stay. He isn't the one doing the forecasting is he? As long as he tells NOAA what there doing wrong and the Hurricane forecasters forecast hurricanes, I have no problem with what hes doing.
Posted By: WPBHurricane05 at 12:27 AM GMT on July 06, 2007.

IMVHO The forecasters are just trying to look good so they don't get fired.


IMO Proenza should get the ax.Not so much because he's wrong,but for 2 reasons;for one,he's a bad fit.He was never a hurricane forecaster(the first NHC director to have that distinction),and that will naturally lead to some...separation between him and his employees.Second,the damage has been done.Even if NOAA launched a new Quikscat tomorrow,if the Miami Herald article is to be believed,there is too much internal strife being caused;it even said there were shouting matches going on between forecasters,and now that some of the senior forecasters have called for his firing,how will they ever have a cordial,functioning relationship?This needs resolved now,and the best way to do that is to get rid of him.
Member Since: Posts: Comments:
1745. NorthxCakalaky
2:06 AM GMT on July 06, 2007
Not realy many islands but the coast is ragged.
Member Since: December 31, 1969 Posts: Comments:
1742. NorthxCakalaky
2:01 AM GMT on July 06, 2007
Everybody asks what if a cat 5 hit bla bla bla... What if a cat 5 in eastern n.c islands.Look at this link to see what I mean.I know some people live there, but there is also swamps.Link
Member Since: December 31, 1969 Posts: Comments:
1741. TheCaneWhisperer
10:01 PM EDT on July 05, 2007
Just a little note on 96L! There isn't upper level anything associated with this area. Mid-Level shear map may put things in perspective, the circulation to the NE of the LLC is the sheared mid-level. Kind of like a weak tornado that snakes down to the surface. IMO upper level shear maps are useless for 96L right now, mid-levels will not allow it to produce the high cloud tops.
Member Since: Posts: Comments:
1739. Thundercloud01221991
2:00 AM GMT on July 06, 2007
Posted By: Stormy2day at 1:49 AM GMT on July 06, 2007.

you all would let me get a met degree even though I got a 37% on TCs quiz?

How did you get a 37 on it, go try again
Member Since: Posts: Comments:
1738. tornadofan
2:00 AM GMT on July 06, 2007
Posted By: Arough209ownage at 1:57 AM GMT on July 06, 2007.
Trough, not trof. It's idiotic when people spell it like that.


Chill. wat's rong with misspellin wurds?
Member Since: Posts: Comments:
1737. weathermanwannabe
9:58 PM EDT on July 05, 2007
Just came on for a few minutes to see what is happening, make one observation, miss-spell a word,and, someone makes a derogatory remark...Good Night..
Member Since: Posts: Comments:
1736. KoritheMan
1:53 AM GMT on July 06, 2007
I think it's a non-tropical low, nola. I doubt anything will happen with it. Besides, it's over the Florida Peninsula. Can't get development from something overland.
Member Since: Posts: Comments:
1735. nolasoci
1:56 AM GMT on July 06, 2007
Thanks Weatherman. I always like to be in touch with these systems. Thanks for your info.
Member Since: Posts: Comments:
1734. WeatherfanPR
1:58 AM GMT on July 06, 2007
m
Member Since: Posts: Comments:
1731. weathermanwannabe
9:46 PM EDT on July 05, 2007
The most notable thing in the tropics for the last several days (other than you know who) has been the persistant storms off the E coast of Florida...I'm a bit surprised that a depression or organization has not formed in that region in spite of several lows hanging around right over/around the State...But the rain has been a good thing (next best thing to a tropical soaker)....Maybe in a few days if a part of this "blob' can break off from the frontal trof.........
Member Since: Posts: Comments:
1730. nolasoci
1:48 AM GMT on July 06, 2007
whats happening with the low by Florida????
Member Since: Posts: Comments:
1729. Stormy2day
1:48 AM GMT on July 06, 2007
you all would let me get a met degree even though I got a 37% on TCs quiz?
Member Since: Posts: Comments:
1728. WeatherfanPR
1:48 AM GMT on July 06, 2007
Interesting, new models run for 96L, GOM maybe down the road!!! Link
Member Since: Posts: Comments:
1727. Thunderstorm2
9:47 PM EDT on July 05, 2007
I know JP..i just edited my comment.
Member Since: Posts: Comments:
1725. Thunderstorm2
9:44 PM EDT on July 05, 2007
That Looked Good at that Latitutde STL!
Member Since: Posts: Comments:
1724. Bamatracker
1:44 AM GMT on July 06, 2007
im for that!! woohoo for met degrees all around!!
Member Since: Posts: Comments:
1721. tornadofan
1:38 AM GMT on July 06, 2007
Weatherunderground is associated with the University of Michigan, right? They ought to provide online courses in meterology, so that we can get an online Bachelors of Science.
Member Since: Posts: Comments:
1719. NorthxCakalaky
1:35 AM GMT on July 06, 2007
Posted By: nolasoci at 1:35 AM GMT on July 06, 2007.

Everybody remember Katrina. She was falling apart and everyone wrote her off. She got to Florida and it fired up. Then got in the gulf and blew up and hit my home of NOLA. I think this may still have some odd chance

Katrina will be told through history.
Member Since: December 31, 1969 Posts: Comments:
1717. Bamatracker
1:35 AM GMT on July 06, 2007
RL3AO...i agree that if it keeps its circulation into the caribbean it has more hope. But not for a few days.
Member Since: Posts: Comments:
1716. nolasoci
1:34 AM GMT on July 06, 2007
Everybody remember Katrina. She was falling apart and everyone wrote her off. She got to Florida and it fired up. Then got in the gulf and blew up and hit my home of NOLA. I think this may still have some odd chance.
Member Since: Posts: Comments:
1715. Thundercloud01221991
1:33 AM GMT on July 06, 2007
Posted By: NorthxCakalaky at 1:32 AM GMT on July 06, 2007.
Posted By: Arough209ownage at 1:29 AM GMT on July 06, 2007.

Of course, it always bites the dust this time of day...It needs to have the diurnal max thing before it can get more convection back...

" 3:00 p.m to 4:00 " LOL, I studied the test.



LOL LOL LOL LOL
Member Since: Posts: Comments:
1714. RL3AO
8:33 PM CDT on July 05, 2007
Even if it can keep a mid-level low, it might be worth watching if it makes it to the Caribbean.
Member Since: Posts: Comments:

Viewing: 1764 - 1714

Page: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44Blog Index

Top of Page
Ad Blocker Enabled

Category 6™

About

Cat 6 lead authors: WU cofounder Dr. Jeff Masters (right), who flew w/NOAA Hurricane Hunters 1986-1990, & WU meteorologist Bob Henson, @bhensonweather

Local Weather

Light Rain
43 °F
Light Rain

JeffMasters's Recent Photos

Mountain wave clouds over Labrador
Mountain wave clouds over Labrador
Mountain wave clouds over Labrador
Labrador ice