Home > GHCN, GSOD > Now For a Few Words from Our Sponsor

Now For a Few Words from Our Sponsor

2010 June 28

Browsing through the NOAA/NCDC data set definitions, I thought I would post some of those that are relevant to surface record efforts. I concentrated on global data, so if you are looking for US data, you will have to turn to the source pages:

NOAA Dataset Documentation: Surface Metadata

NCDC Dataset Documentation & Metadata


Dataset 3210: Summary of the Day – First Order

Abstract: This Summary of the Day First Order data file contains daily selected elements of observations taken by certified observers. The stations are located worldwide and are operated by the National Weather Service (NWS), U.S. Air Force (Air Force Weather), U.S. Navy (Navy Weather Service), and the Federal Aviation Administration (FAA). A very small portion of this data dates back to 1890. Data from the late 1940’s and onward comprise the bulk of this data set. These First Order Stations are usually fully instrumented and therefore record a complete range of meteorological parameters. The observations are generally recorded for the 24-HR period midnight to midnight.

Through the years, approximately 1,380 principle stations have recorded observations in the program. Stations have varying periods of record and varying types of data elements. In the early years of aviation most stations were located in major cities. As aviation became more sophisticated, stations shifted to airports. Areal coverage includes the contiguous United States, Caribbean Islands, Pacific Islands, and other overseas stations of the NWS, FAA, U.S. Navy, and U.S. Air Force.
Field elevations of fixed surface locations for the data set are mostly below 1,000 meters above sea level. The minimum elevation is 1 meter and the maximum is 2,300 meters.

The primary source of surface observational data in the United States is the Automated Surface Observing System (DS-3211). Users must be aware that this data set contains flagged data that have been quality controlled by personnel at the NCDC. DS-3211 contains only automated quality controlled data. The ASOS users guide covers all essential aspects of system operation, including data acquisition, processing, and report formatting and dissemination. Elements, such as soil data, not found in DS-3210 can be found in DS-3200.

It must be noted that NCDC has the observations from the time the station opened, but the NWS has the current data. Official surface weather observation standards can be found in the Federal Meteorological Handbook.

Dataset 3500: Monthly Climatic Data for the World

Abstract: The National Climatic Data Center processes international electronic transmissions in CLIMAT (surface land station format), CLIMAT SHIP (ocean ship format) and CLIMAT TEMP (upper air format), for the purpose of building a surface/upper air database and publishing the MCDW bulletin. Approximately 1200 surface and 500 upper air stations are processed. These data are used in agricultural and energy assessment activities, in crop yield model development, and in the analysis of global atmospheric and regional climatic variations. The National Center for Atmospheric Research (NCAR) produces the data sets DSI-9645 and TD9648 (obsolete), which are also archived at NCDC, from the MCDW annual. (This data set contains data from 1986 onward. However, the data sets listed above contain other data from as early as 1731.

Dataset 3505: Global Integrated Surface Hourly Data

Abstract: The Integrated Surface Hourly (ISH) database is composed of worldwide surface weather observations from about 20,000 stations, collected and stored from sources such as the Automated Weather Network (AWN), the Global Telecommunications System (GTS), the Automated Surface Observing System (ASOS), and data keyed from paper forms. Most digital observations are decoded either at operational centers and forwarded to the Federal Climate Complex (FCC) in Asheville, NC, or decoded at the FCC. The US Air Force Combat Climatology Center (AFCCC), the National Climatic Data Center (NCDC), and the US Navy’s Fleet Numerical Meteorological and Oceanographical Command Detachment (FNMOD), make up the FCC in Asheville. Each agency is responsible for data ingest, quality control, and customer support for surface climatological data. All data are now stored in a single ASCII format. Numerous DOD and civilian customers use this database in climatological applications.

Abstract: The Integrated Surface Database(ISD) is composed of worldwide surface weather observations from about 20,000 stations, collected and stored from sources such as the Automated Weather Network (AWN), the Global Telecommunications System (GTS), the Automated Surface Observing System (ASOS), and data keyed from paper forms. Most digital observations are decoded either at operational centers and forwarded to the Federal Climate Complex (FCC) in Asheville, NC, or decoded at the FCC. The US Air Force Combat Climatology Center (AFCCC), the National Climatic Data Center (NCDC), and the US Navy’s Fleet Numerical Meteorological and Oceanographical Command Detachment (FNMOD), make up the FCC in Asheville, NC. Each agency is responsible for data ingest, quality control, and customer support for surface climatological data. All data are now stored in a single ASCII format. The database is used in climatological applications by numerous DoD and civilian customers.

ISD (identical to ISH) refers to the digital database and format in which hourly, synoptic (3-hourly), and various other weather/climate observations are stored. The format conforms to Federal Information Processing Standards (FIPS). The database includes data originating from various codes such as synoptic, airways, METAR (Meteorological Routine Weather Report), and SMARS (Supplementary Marine Reporting Station), as well as observations from automatic weather stations. The data are sorted by station-year-month-day-hour-minute. This document provides complete documentation for the database and its format.

Dataset 3850: Surface Weather Observation Forms 1001

Abstract: In order to fill in the observation gap prior to the time when commercial aviation began in the U.S., NCDC’s Climate Data Modernization Project (CDMP) retrieved surface observations taken two to four times daily beginning as early as 1893 at the City Weather Bureau Offices. Through 1936 observations were taken twice daily; then in 1937 the general practice was to record four observations per day. Since this data overlaps with the pre-1948 hourly dataset DSI-3851 (surface airways observations), data for each station were not keyed once their observations overlapped with the DSI-3851 data which generally contains more observations per day.

The data elements are as follows: station pressure, sea level pressure, dry and wet bulb temperature, dew point, maximum and minimum temperature, wind direction and speed, precipitation, cloud amount and type, ceiling, state of weather and visibility. It should be noted that not all elements are present for all stations in this dataset, and that ceiling and visibility observations did not begin at the city office’s until the 1930’s.
Official surface weather observation standards can be found in the Circular N manuals. The images are available on a web based system, Web Search Store Retrieve Display (WSSRD), and will eventually be accessible through NCDC’s On-Line store.

Dataset 9100: Global Historical Climatology Network

Abstract: The Global Historical Climatology Network Version 2 temperature database was released in May 1997. This century-scale data set consists of monthly surface observations from ~7,000 stations from around the world. This archive breaks considerable new ground in the field of global climate databases. The enhancements include: (1) data for additional stations to improve regional-scale analyses, particularly in previously data-sparse areas; (2) the addition of maximum/minimum temperature data, to provide climate information not available in mean temperature data alone; (3) detailed assessments of data quality to increase the confidence in research results; (4) rigorous and objective homogeneity adjustments to decrease the effect of non-climatic factors on the time series; (5) detailed metadata (e.g., population, vegetation, topography) that allows more detailed analyses to be conducted; and (6) an infrastructure for updating the archive at regular intervals, so that current climatic conditions can constantly be put into historical perspective.

Dataset 9101: Global Daily Climatology Network

Abstract: The Global Daily Climatology Network (GDCN) represents a compilation of global daily timescale data into a single and consistent format. The data set serves the needs of researchers, weather-sensitive businesses, agriculture, and policy makers whose are dependent upon complete and accurate analysis of daily temperature and precipitation. Data within the GDCN have been extensively checked through a series of quality control procedures to ensure erroneous values have been removed and/or identified. The GDCN currently has over 800 million days of weather data and that number is expected to increase as the GDCN expands in the future. If you are using a recent version of Microsoft Internet Explorer, you can view the 9101 complete document

Dataset 9290a: Global Synoptic Climatology Network – Canada

Abstract: These files are a compilation of in situ hourly meteorological observations for Canada initially obtained within the framework of joint efforts to create Global Synoptic Climatology Network among the Meteorological Service of Canada (Downsview, Ontario and Vancouver, British Columbia), Research Institute for Hydrometeorological Information, of the Russian State Committee for Hydrometeorology, Obninsk, Russian Federation, and NOAA National Climatic Data Center. The data were then pre-processed according to a standard quality control routine and generation of derived variables (e.g., some humidity characteristics were derived from others). Currently, there are approximately 170 active stations, which can be operationally updated from the entire list of 768 locations, and another 350 stations are updatable with a delay. The maximum period of the data span is from January 1, 1953 to February 21, 2005. Variables in the data set include sea level and station pressure, surface air temperature, water vapor pressure, relative humidity, wind speed and direction, several characteristics of cloudiness, and present weather.

Dataset 9290c: Global Synoptic Climatology Network – USSR

Abstract: These files are a compilation of in situ hourly meteorological observations for the former USSR initially obtained within the framework of joint efforts to create Global Synoptic Climatology Network among the Meteorological Service of Canada (Downsview, Ontario and Vancouver, British Columbia), All-Russian Research Institute for Hydrometeorological Information-World Data Center of the Federal Service for Hydrometeorology and Environmental Monitoring, Obninsk, Russian Federation, and NOAA National Climatic Data Center, Asheville, North Carolina. The data in this archive have been compiled from several sources that include information delivered via the Global Telecommunication System, international exchange, and random data acquisitions. This information was verified versus the original data holdings at the national hydrometeorological archive of the Russian Federation and appended when it was necessary. These data were then pre-processed according to a standard quality control routine and generation of derived variables (e.g., some humidity characteristics were derived from others). Currently, there are mostly data for Russia but prior to 1991 the entire former USSR was covered by 2095 stations (Figure 1). The maximum period of the data span is from January 1, 1871 to January 1, 2001. Variables in the data set include sea level and station pressure, surface air temperature, water vapor pressure,relative humidity, wind speed and direction, several characteristics of cloudiness, and present weather.

Dataset 9618: Global Summary of the Day

Four files are included in this product. These files contain summarized data
which have been extracted from surface synoptic weather observations,
exchanged on the Global Telecommunications Systems (GTS). The National
Meteorological Center (NMC) of NOAA maintains an archive file of the complete
surface synoptic reports which are received from the GTS. The Climate Analysis
Center (CAC) extracts portions of these NMC archive files, performs an
automated decode of extreme temperatures and accumulated precipitation
according to WMO code manuals, and performs limited automated validation of
the parameters. The data for all reporting stations are summarized on a daily
basis to satisfy current operational requirements related to the assessment of
crop and energy production.

FILE #1. Header file which explains the contents of the four files.

FILE #2 Daily summarized values of maximum temperature, minimum temperature
and precipitation for each reporting station for one month.

FILE #3 Monthly summary of up to 31 daily summary files. Average
temperatures have been computed from the daily maxima and minima for each
station. The highest and lowest temperatures reported during lowest
temperatures reported during the month and their dates, the total accumulated
precipitation for the month, and the number of reports with present or past
weather codes indicating the occurrence of precipitation are also saved.

FILE #4 Station library consisting of the location identifier, latitude,
longitude, elevation, country number, region number, quality indicator, call
letters, station name and country name

Dataset 9618a: Global Summary of the Day, 1977-1986

Abstract: This dataset contains three files of summarized data which have been extracted from surface synoptic weather observations, exchanged on the Global Telecommunications Systems (GTS). The National Meteorological Center (NMC) of NOAA contains an archive file of the complete surface synoptic reports which are received from the GTS. The Climatic Analysis Center (CAC) extracts portions of these NMC archive files, performs an automated decode of extreme temperatures and accumulated precipitation according to WMO code manuals, and performs limited automated validation of the parameters. The data for all reporting stations are summarized on a daily basis to satisfy current operational requirements related to the assessment of crop and energy production.

Dataset 9618b: Global Summary of the Day, 1987-Present

Abstract: Four files are included in this product. These files contain summarized data which have been extracted from surface synoptic weather observations, exchanged on the Global Telecommunications Systems (GTS). The National Meteorological Center (NMC) of NOAA maintains an archive file of the complete surface synoptic reports which are received from the GTS. The Climate Analysis Center (CAC) extracts portions of these NMC archive files, performs an automated decode of extreme temperatures and accumulated precipitation according to WMO code manuals, and performs limited automated validation of the parameters. The data for all reporting stations are summarized on a daily basis to satisfy current operational requirements related to the assessment of crop and energy production.

Dataset 9641a: WMO Global Standard Normals

Abstract: This data set contains 1961-1990 global standard climate normals for over 4000 stations worldwide computed by more than 135 countries and territories. The major parameters that make up this data set consist of: maximum temperature, minimum temperature, mean temperature, soil temperature, precipitation, snowfall, snow depth, wet bulb temperature, dew point temperature, relative humidity, sea level pressure, vapor pressure, wind speed and direction, cloud cover, sunshine duration, solar radiation, evaporation, number of days with various weather elements (occurrence/nonoccurrence), and number of days with weather parameters beyond various threshold values. The statistics include: mean, median, quarterlies, extremes, frequency distribution, standard deviation, and number of years with non-missing data. The normals data were computed by the Member countries and territories of the World Meteorological Organization (WMO) and submitted to the National Oceanic and Atmospheric Administration’s National Climatic Data Center (World Data Center-A for Meteorology) for collection and processing. The quality assurance process consisted of extreme limits checks and consistency checks. The global standard normals are prepared worldwide once every 30 years.

Dataset 9644: World Weather Records

Abstract: This data set has been collected under the name “World Weather Records” (WWR) since the first series was published in a single volume in 1927. The 1981-1990 decadal series is the eighth series of published data. As of November 1996, only three of these series: 1951-1960, 1961-1970 and 1971-1980 have been digitally archived in DSI-9644. Other data sets at the National Climatic Data Center (NCDC), most notably the Global Historical Climate Network (GHCN: DSI-9100) have historical surface climatological data derived from the earlier, non-digital series of WWR. Elements from 1951-70 include monthly means of station and sea level pressure, temperature, as well as monthly total amounts of precipitation. Beginning with the 1981-90 series, values of mean maximum and minimum monthly temperatures were additionally included in DSI-9644.

Dataset 9645: World Weather Records NCAR Surface

Abstract: The National Center for Atmospheric Research (NCAR) has world monthly surface station climatological data for over 3900 different stations (2500 in more recent years) up through 1998. Data for some stations goes as far back as the mid-1700’s. The standard parameters available are sea level pressure, station pressure, temperature, and precipitation. After 1960, additional available parameters include moisture and percent of possible sunshine.

Dataset 9685: Global Daily Synoptic Surface Data

Abstract: The National Center for Atmospheric Research (NCAR) offers NCDC and United States Air Force DATSAV TD9685 global daily surface observation data sets. These are Global surface observations for many stations in modified SYNOP format. Parameters included in this dataset are: surface pressure, temperature, wind direction, wind speed, dewpoint, cloud cover, cloud type, height of cloud base, descriptive weather, visibility, snowfall, precipitation, station elevation, sea level pressure, sea surface temperature and wave height. The period of record is January 1967 – December 1980 and the area of coverage is global.

Dataset 9950: DATSAV2 Global Surface Data

Abstract: DATSAV2 is the official climatological database for surface observations. The database is composed of worldwide surface weather observations from about 10,000 currently active stations, collected and stored from sources such as the US Air Force’s Automated Weather Network (AWN) and the WMO’s Global Telecommunications System (GTS). Most collected observations are decoded at the Air Force Weather Agency (AFWA) formerly known as the Air Force Global Weather Central (AFGWC) at Offutt AFB, Nebraska, and then sent electronically to the USAF Combat Climatology Center (AFCCC). DATSAV2 refers to the digital tape format in which decoded weather observations are stored. The DATSAV2 format conforms to Federal Information Processing Standards (FIPS). The DATSAV2 database includes data originating from various “codes” such as synoptic, airways, METAR (Meteorological Routine Weather Report), and SMARS (Supplementary Marine Reporting Station), as well as observations from automatic weather stations.

Dataset 9956: DATSAV3 Global Surface Hourly Data

Abstract: The DATSAV3 Surface Database is composed of worldwide surface weather observations from about 10,000 currently active stations, collected and stored from sources such as the Automated Weather Network (AWN) and the Global Telecommunications System (GTS). Most collected observations are decoded at the Air Force Weather Agency (AFWA) at Offutt AFB, Nebraska, and then sent electronically to the USAF Combat Climatology Center (AFCCC). AFCCC builds the final database through decode, validation, and quality control software. All data are stored in a single ASCII format. The database is used in climatological applications by numerous DoD and civilian customers.

DATSAV3 refers to the digital tape format in which decoded weather observations are stored. (Two older, discontinued formats were DATSAV and DATSAV2.) The DATSAV3 format conforms to Federal Information Processing Standards (FIPS). The DATSAV3 database includes data originating from various codes such as synoptic, airways, METAR (Meteorological Routine Weather Report), and SMARS (Supplementary Marine Reporting Station), as well as observations from automatic weather stations. The users handbook provides complete documentation for the database and its format.

AFCCC sorts the observations into station-date-time order, validates each station number against the Air Weather Service Master Station Catalog (AWSMSC), runs several quality control programs, and then merges and sorts the data further into monthly and yearly station-ordered files. AFCCC then provides the data to the co-located National Climatic Data Center (NCDC).

Advertisements