[June 28th, 2006 - Email sent to the tropical storms email list by Prof. Judith Curry]

Dear Colleagues:

We have begun scrutinizing the datasets for NATL TS, and I would appreciate any comments/advice that you have on discrepencies that we have found in the HURDAT data set. The attached file noting discrepancies and the message below was prepared by two Georgia Tech students, James Belanger and Mark Jelinek. We have read the documentation and understand that this is a work in progress. We note only errors of internal inconsistency within the data sets.

These internal inconsistencies need to be cleaned up. Ideally this would be done by the group that generated the data set. If there are insufficient resources to do this in the short term, we will either fix the discrepancies based upon any advice that we receive, and/or assign error bars to particular storms associated with the discrepancies in the data set. We prefer that others modify the data set so that we can use an "off the shelf" data set (so that we are not accused of "cooking" the data set in case we come up with any controversial findings :)

Comments from the best tracks committee and/or other users of the data set would be greatly appreciated. Thanks.

Judy

Notes regarding the attached xcel spreadsheet:

"The purpose of this evaluation was to identify both obvious and potential errors related to the three versions of the HURDAT data files available at

http://www.aoml.noaa.gov/hrd/hurdat/Data_Storm.html
1. The analysis first identified the discrepancies between the 'Tab Delimited' version and the 'Easy to Read' version then between those two versions and the 'Original' version 2. Then apparent discrepancies between the 'Best Track' portion of the data and the landfall designation information were identified"

Each worksheet title is an indicator as to the analysis contained within "The analysis did not review and/or attempt to clarify:

  1. Any potential discrepancies with the storm speed/direction or maximum winds expressed in miles and kilometers per hour that are calculated and provided in both the 'Easy to Read' and 'Tab Delimited' versions from the components in the 'Original' version of the data
  2. Any discrepancies between the 'Center Fix' and 'Metadata' portion of the reanalysis and the 'Original' data file
  3. Any concerns about terminology used in making the landfall designations contained in the 'Original' data file"

[6 July 2006 - Letter from the NHC Best Track Change Committee Chair Colin McAdie to Prof. Judith Curry] Dear Professor Curry,

Thank you for pointing out some issues with HURDAT. I should first say that for scientific purposes I recommend that you use only the official version of HURDAT as found on our website

http://www.nhc.noaa.gov/tracks1851to2005_atl.txt.
It appears that a number of comments attached to your note deal with differences found between HURDAT and two other derived versions that appear on the AOML/HRD website.

While clearly they should be consistent, it appears that a programming oversight within that organization created most of the inconsistencies, hence our recommendation that only HURDAT be used for analysis. It is our understanding that these derived versions were created for the purpose of making the dataset more ‘readable’ rather as an alternative source for scientific investigation. Chris Landsea has informed me that these errors have now been been corrected and reposted to the AOML/HRD website.

This is not to say that HURDAT itself is without flaws, and we appreciate your efforts in uncovering inconsistencies. We continue to find and correct errors. I believe you are aware of the ongoing re-analysis effort being conducted by Chris Landsea that was specifically designed to screen for errors and apply current understanding to the dataset. As of this writing, the NHC Best-Track Change Committee (BTCC) is evaluating the submission of Landsea et al. for 1915-1923. I can attest that this is quite a labor-intensive process both for the re-analysis team and the BTCC. In addition, any changes made to U.S. landfalling category 4 or 5 hurricanes must be submitted for outside review. The BTCC strives to consider all available sources before making a decision. During the reevaluation of H. Andrew, for example, the BTCC hosted a multi-day workshop to hear a variety of opinion. Individual cases can be very difficult to decide. However, progress is being made and the official version of HURDAT now reflects changes accepted by the BTCC for the period 1851-1914, and H. Andrew, 1992.

As part of this process, we require that data, discussion, and rationale be provided for any changes. The resulting 'metadata' appears on the re-analysis webpage. Please note that inspection of the metadata, in particular with regard to the winds at the time of landfall, can resolve many of these apparent discrepancies.

With regard to the "Best Track vs. Landfall" comments, you have indeed found some important inconsistencies for the period of 1915 to 2002. We are aware of many of these, and in fact they have provided the impetus for the reanalysis. As such, they will be dealt with sequentially within the reanalysis. Again, if you have raw observations, suggested HURDAT changes, and a discussion on how to resolve these inconsistencies, please feel free to submit these directly to the reanalysis committee.

However, out-and-out errors are a different matter and can be corrected fairly quickly. Concerning the comments for 1851 to 1914, four of these were indeed correct and changes have been to made accordingly to HURDAT for these systems: 1874/07, 1878/05, 1880/11, and 1893/04. However, for the remainder no changes were needed. We would encourage your students to read through both the Metadata documentation on these individual systems:

http://www.aoml.noaa.gov/hrd/hurdat/metadata_5114.html

as well as the U.S. hurricane table:
http://www.aoml.noaa.gov/hrd/hurdat/ushurrlist.htm

to see why the tropical storms and hurricanes are categorized as they currently are. Also note that what appear to be errors in the landfall designation are very frequently artifacts of the 6-hourly reporting interval. That is, the 6-hourly data does not capture the estimated winds at landfall. These landfall designations (as they appear in HURDAT ) are supported by data and discussion found in the Tropical Cyclone Reports rather than winds given for the nearest 6-hourly interval.These reports are also found on our website at
http://www.nhc.noaa.gov/pastall.shtml.

Sincerely,
Colin McAdie
Chair, BTCC