ACRONYM LIST | |
---|---|
ASCII | American Standard Code for Information Interchange |
ATLAS | Automated Temperature Line Acquisition System |
BODC | British Oceanographic Data Centre |
CMAN | Coastal Marine Automated Network |
COADS | Comprehensive Ocean-Atmosphere Data Set |
COAPS | Center for Ocean Atmospheric Prediction Studies |
COARE | Coupled Ocean Atmosphere Response Experiment |
DADC | Data Archive and Distribution Center |
DPC | Data Processing Center |
DQE | Data Quality Evaluator |
FSU | Florida State University |
GTS | Global Telecommunication System |
IDL | Interactive Data Language |
IMET | Improved METeorology ship and buoy system |
IOP | Intensive Observing Period |
ISS | Integrated Sounding System |
NCAR | National Center for Atmospheric Research (USA) |
NCDC | National Climate Data Center (USA) |
NGDC | National Geophysical Data Center (USA) |
NOAA | National Oceanic and Atmospheric Administration (USA) |
netCDF | network Common Data Format |
PROTEUS | PROfile Telemetry of Upper ocean currentS |
PSS | Priority Sounding Station |
QC | quality control |
R/V | research vessel |
SLP | sea-level pressure |
TCIPO | TOGA/COARE International Project Office |
TOGA | Tropical Ocean Global Atmosphere |
UTC | universal time coordinated |
VAWR | Vector Averaging Wind Recorder |
VIDAT | VIsual Data Assessment Tool |
VISIT | VIsual Station Intercomparison Tool |
WMO | World Meteorological Organization |
Once the data were collected, the DPC verified the accuracy of the measurements using a series of data quality evaluation procedures. The focus of the DPC Handbook of Quality Control Procedures and Methods is to describe the methodology and detail these quality control (QC) procedures. The DPC intends for this document to be used by the COARE scientific community as well as other interested researchers. The flagging procedures contained in this report may also be of general interest to data evaluators from the international data centers
The overall goal of the DPC's QC is to provide a well-documented, reliable, and consistent data set to the scientific community. The DPC evaluates the data to the best level possible with the QC results depending primarily on the level of knowledge of the incoming data as well as examination and analysis of the data. Information provided about how the data were collected, i.e. metadata, is an essential component of any data set, particularly in light of the historical context of climate variability. Metadata will be included with all DPC quality controlled data sets either as additions to the data or as a part of the QC reports. Ideally, the metadata contains information on the instrument setup, calibrations, and formats for the meteorological data.
Data quality flagging, not value replacement, is the method used to denote suspect or erroneous data. The choice to flag questionable data at the DPC, as opposed to data correction, was made to insure the data remain in the original form that was sent to the DPC. For scientific reasons, the end user must determine appropriate utilization of the flags provided. This will allow additional processing/decisions to be made at later times.
The QC procedure for all COARE data sets is a five-step process schematically outlined in Fig. 1. The process begins with the acquisition of a COARE meteorological data set and accompanying metadata (e.g. README file). The first step of the QC is converting the meteorological data to a standard format for internal use at the DPC. The format includes metadata and is described in Section 2. Once converted, the data are sent through an automated preprocessing program that checks for values outside of a realistic range, statistical outliers, unrealistic ship movements, etc. The preprocessing procedures are outlined in Section 3. During the third step of the QC, the preprocessed meteorological data are visually inspected by a trained Data Quality Evaluator (DQE) using the VIsual Data Assessment Tool (VIDAT). VIDAT is an interactive, graphically-based tool developed at the FSU DPC, that allows the DQE to add or remove QC flags from time series of meteorological data. VIDAT is described in detail in Section 4. In the fourth step of the QC, the DQE compares neighboring stations using the VIsual Station Intercomparison Tool (VISIT). VISIT is another interactive graphically-based tool developed at FSU and is described in section 5. When the meteorological data have passed all final inspections, the data are converted to a standard format suitable for public distribution and further analysis. At this time a QC report is written combining flag information from the preprocessor and the DQE's VIDAT and VISIT evaluation. More information, i.e. calibrations done prior to the data's arrival at the DPC, may be extracted from the metadata and added to the QC report. The resulting quality controlled data and report are then made available for the COARE community.
Figure 1. Data flow (arrows) through the FSU DPC. The data that arrive from a provider (*individual investigators, chief scientists, data centers, etc.) usually have two components; the data values and the documentation. The data values are converted to a standard format and combined with metadata extracted from the documentation. Automated preprocessing and visual inspection (VIDAT, VISIT) of the data follow. In the final stages, the data are converted to a user friendly public format and QC reports are written. Both ASCII and netCDF formats of the data and the reports are then distributed to the COARE community.
An additional important global attribute, the Florida State University (FSU) version number (Table 1), was developed to track the data files as they passed through the various stages of the QC process. Intermediate numbers were used at times as some steps in the QC process were completed in multiple stages. Version numbering is automated, i.e. each new file created during the QC process is automatically assigned appropriate version numbers.
The FSU version number is also an integral part of the file names assigned to all data arriving at the DPC. The file naming convention created for FSU COARE surface meteorological data is based on the ship's call sign or a World Meteorological Organization (WMO) identification number for land stations and buoys. This convention takes the form:
ID: | Site identifier containing up to 8 alphanumeric values. Normally a ship's call sign or a WMO number for a buoy or land station. A "." always marks the end of the ID. |
YY: | Two digit year of the first record in the file. |
MM: | Two digit month of the first record in the file. |
sD: | Two digit date of the first record in the file. |
nD: | Three digit value indicating number of consecutive days of data in file. |
v###: | FSU version number (without decimal points). |
.nc: | Identifier for netCDF files. ASCII files will be identified using .asc. |
7JDU.930202017v200 <----> <----><-><--> ID 1st # FSU# Day DaysThe file name reveals that the Natsushima cruise began on 2 February 1993 and ran for 17 days. The version number shows that the data has completed all levels of QC and is ready for distribution.
Table 1: Version numbers for data files at the FSU DPC. Multiple versions are indicated by a '?'. | |
FSU Version | Definition |
0.0.0 | Data in original format (digital or hard copy) |
0.0.1 | Digitized original data (from hard copy) |
0.0.9 | Data converted to internal FSU DPC netCDF format |
0.2.0 | Preprocessing completed, ready for visual inspection |
0.2.? | VIDAT inspection completed |
1.0.0 | First public release to community |
1.0.? | Updated public version (metadata added/minor errors corrected) |
1.5.0 | Data ready for level 2 visual processing |
1.5.? | VISIT inspection completed |
2.0.0 | Second public release to community |
2.0.? | Updated second release versions |
Most of the data that arrive at the FSU Surface Meteorology DPC are converted directly to netCDF using standard FORTRAN or C language calls. Normally the only change made to the data is a conversion to a standard set of units (Table 2) using common conversion tables. The original units are noted and retained in a variable attribute. However, if data arrive with no units attached, and only after every effort to determine their units is exhausted, the data are discarded. Furthermore, any data arriving without an accurate time stamp are discarded only after all attempts to obtain correct time information were exhausted. The data also must have platform position information; thus data lacking latitude and longitude values are discarded only after all attempts to correctly position the data have failed. Data collected without a record of time, position, or units are useless to the scientific community. All original data contributions are archived in the event additional information is discovered later that would allow the inclusion of discarded data.
Table 2: DPC standard units. | ||
Variable | Units | Convention |
time | minutes | since 01-01-1980 00:00 UTC |
latitude | degrees | + North, - South |
longitude | degrees | 0-359 degrees East |
platform heading | degrees | clockwise from true North |
platform speed | m s-1 | |
wind direction | degrees | direction wind is blowing from (meteorological convention) |
wind speed | m s-1 | |
atmos. pressure | mb | |
air temperature | o Celsius | |
wet-bulb temp. | o Celsius | |
dew point temp. | o Celsius | |
sea temperature | o Celsius | |
relative humidity | percent | |
specific humidity | g kg-1 | |
precipitation | mm | if a rain rate - mm hr-1 |
radiation | W m-2 |
In some cases, additional modifications are made to the original (Version 0.0.1) data. These include ordering the data in a correct time sequence and removing duplicate records whenever possible. As time permits and with adequate metadata, obvious typographical errors can be corrected. A simple example of a typographical error is a single time stamp reporting the year to be 1990 when the remainder of the cruise data are from a line measured in 1993. All modifications made to the original data are carefully documented in the QC report and all original data are retained in the FSU DPC archive.
A final modification to the original data structure may be necessary if there are multiple measurements of the same parameter in one data set. For example, if a ship has three different instruments to measure sea temperature (i.e. bucket, intake, and a thermosalinograph) the netCDF file will contain three sea temperature variables (TS, TS2, and TS3) each with its own set of attributes. No modification of the original data is required if the three sea temperatures are all recorded at the same rate; however, if the time resolution varies, the time stamp from the sea temperature recorded at the finest temporal resolution will be used as the base time for the netCDF file. For example (Table 3), if the SST is recorded every 5 minutes by thermosalinograph (TS), every 15 minutes by intake (TS2), and approximately every hour by bucket (TS3), the second two sea temperature time series have their times matched to the nearest time of the first sea temperature (TS). Severe time mismatches are handled on a case-by-case basis and notes are provided in the QC report.
A final note on the conversion of incoming data to the FSU DPC netCDF format: the netCDF files are limited to 1440 records, which corresponds to the number of minutes in a day. The 1440 record limit was chosen primarily for convenience and to simplify the operation of the preprocessor and visual inspection software. Thus incoming data files with more than 1440 records were broken into smaller files. Not all files contain a single day's data, but the limit of 1440 records still applies.
Table 3: Example of matching times for multiple measurements of the same variable (e.g. sea temperature) from one platform. | ||
Time for TS | Time for TS2 | Time for TS3 |
00:00 | 00:01 | 23:58 |
00:05 | ||
00:10 | ||
00:15 | 00:16 | |
00:20 | ||
00:25 | ||
00:30 | 00:31 | |
00:35 | ||
00:40 | ||
00:45 | 00:46 | |
00:50 | ||
00:55 | 00:56 | |
01:00 | 01:01 | |
01:05 |
One of the primary objectives of the FSU DPC for surface meteorology is to provide the community with quality controlled data (i.e., the data will be evaluated and assigned a flag (or flags) which relates information concerning the quality of the data). In the past, data quality information was provided through a series of elaborate and sometimes complicated numerical flags which often were difficult to use and at times ignored. Another problem with some older flagging techniques is that they correspond to entire data records instead of the individual data values. For example, if a station records 20 variables, and only the pressure value is in question, the entire record should not be flagged as erroneous.
At the FSU DPC a very simple approach to QC flags was taken. The quality control flags are single alphabetic characters for each data value in a record that indicate either problems or notable features. The quality control flags for multiple variables in each record at a single time are combined in a single character string and stored in the flag variable for that record. The length of the flag string is equal to the number of variables that underwent quality control. As an example, assume that a FSU DPC netCDF file contains only these five variables: time, latitude, longitude, atmospheric pressure, and air temperature data along with a flag variable. If the values for the first record of all five variables pass all quality control checks, then the first record of the flag variable will be ZZZZZ (a Z indicates acceptable data - see Appendix C) i.e. one flag for each of the five data values. If the second record contains a non-sequential time (flagged by C), but good latitude and longitude values, a pressure value of 1190 mb (too high - flagged by B), and a temperature that is six standard deviations from the climatology (flagged by G), then the second record flag string will be CZZBG. The non-Z flags are values that failed one or more of our QC tests (outlined in sections 3 and 4).
In order to gain easy, quick access to flags which correspond to specific variables, each quality controlled variable in a FSU COARE netCDF file has a variable attribute qcindex. The qcindex is an integer index that points to the position in the flag string for that variable. In the above example the qcindex for pressure is four, thus the fourth character in the flag string is the QC flag for pressure. The flagging method described here allows access to the flag for any variable using the qcindex variable attribute.
Prior to all QC processing, all acceptable data values in a FSU DPC netCDF file are assigned a Z flag. Three flags may be added during the conversion of the data to the FSU format. The A flag is assigned to any data value where the original units are not known or explicitly provided, and the units were determined using supporting information (i.e. neighboring stations, country unit conventions, synoptic maps, etc.). The supporting information used will be outlined in a QC report for the data.
In a few cases, meteorological data arrived at the DPC with separate platform position and movement files. Efforts were made to match the position data accurately to the meteorological data; however, due to varying time stamps, a perfect match was not always possible. As a result some of the platform positions and movements are uncertain. In other cases, positions are known to be uncertain according to information from the data provider. The P flag denotes uncertain platform position/movement information.
The third possible flag added during the conversion process, Q, is for data that arrives already flagged as questionable. The DPC attempts to match any quality control flags on the incoming data to a similar flag in our system. When data arrives with flags already assigned, a section is added to the QC report explaining how the original flags were converted to the FSU system.
Table 4: Flag, flag purpose, and flag type issued by preprocessor. | ||
Flag | Purpose | Type |
B | Value out of realistic range | Univariate |
C | Time not sequential | Univariate |
D | Failed TTwTd test | Multivariate |
E | Failed resultant wind recomputation test | Multivariate |
F | Platform velocity unrealistic | Multivariate |
G | Value > 4 standard deviations from climatology | Univariate |
L | Oceanographic platform crosses land | Multivariate |
T | Time duplicate | Univariate |
The first of the univariate tests flags any observation where the time is not sequential or is duplicated. The time is flagged with a C when the next record does not have a time later than the current record. Often when multiple out of sequence times occur in a file, efforts are made to reorder the records. If the original data order is modified, a note will be included in the QC report.
When two consecutive the times are identical, the time duplicate flag T is added to both times. The time duplicate check indicates only that the times are identical. Visual inspection of the remainder of the data record is used to determine if they are exactly identical records. If both entire records are identical, one is discarded. However, if any discrepancies occur in the two suspect records with duplicate times, both are retained and assigned T flags.
An obvious limitation of the time tests is that the first time value is assumed to be correct. The major advantage to the time tests is that they allow the correction, in most cases, of the time sequence and the removal of duplicate records.
The statistical test is applied as follows: 1) the data value (x) to be compared to the climatology is mapped to the nearest climatology box using the data value's latitude and longitude, 2) if x falls outside the range defined by the (Xc) ± 4 s.d., x is flagged with a G. The statistical test was designed primarily to focus the DQE's attention on extreme values. The DQE may change the G upon visual inspection, e.g.when a physical explanation for the extreme values can be determined.
For example, atmospheric pressure from the island station Nauru clearly has values less than 4 s.d. (dashed line) from the climatological mean (solid line) Fig. 2. All values below the dashed line were flagged with a G. In the example shown, no reason for the low pressures could be determined, so the G flags were unchanged to caution other users of the Nauru pressure data.
The main limitation to the statistical test is the test can only be applied to variables that are in both the climatology and the netCDF file. In the case of the da Silva et al. (1994) climatology, the statistical check is limited to the wind speed, sea
level pressure, air temperature, sea surface temperature, and relative humidity data. Another limitation of the comparison occurs due to varying observational systems. For example, the sea temperature can be measured from instruments located anywhere from the surface to approximately four meters below sea level. These measured sea temperatures are compared to a climatological SST based primarily on bucket and intake temperatures. The same instrument related errors can apply to all the variables since ship instruments are often not at standard heights or in standard shelters.
Figure 2. Atmospheric pressure data from the island of Nauru for December 1992 showing values that are greater than four standard deviations (dashed line) from the da Silva et al. (1994) mean (grey line).
Table 5: Range bounds used by DPC. | ||||
Variable | Lower Bound | Upper Bound | Units | Comments |
time | 1-1-1980 | 12-31-1999 | ||
latitude | -90 | 90 | degrees | |
longitude | 0 | 359 | degrees | |
platform heading | 0 | 359 | degrees | |
platform speed | 0 | 15 | m s-1 | research vessels |
0 | 0 | m s-1 | anchored buoys | |
0 | 2 | m s-1 | drifters | |
0 | 0 | m s-1 | land station | |
plat. wind direction | 0 | 360 | degrees | |
plat. wind speed | 0 | 40 | m s-1 | |
wind direction | 0 | 360 | degrees | |
wind speed | 0 | 40 | m s-1 | |
pressure | 950 | 1050 | mb | sea level |
air temperature | -10 | 40 | o Celsius | |
wet bulb temperature | -10 | 40 | o Celsius | |
dew point temperature | -10 | 40 | o Celsius | |
sea temperature | 0 | 35 | o Celsius | |
relative humidity | 0 | 100 | percent | |
specific humidity | 0 | 48 | g kg-1 | |
rain rate | 0 | 150 | mm hr-1 |
The bounds test also verifies that coded data are within expected ranges. For example, cloud type codes should be in a range of 1 to 10. If a value is not in that range, the bounds routine changes the out of range data value to the special value (-8888) and notes the change to the assessment file. The assessment file is then checked by the DQE to search for systematic errors in the codes. The change in the data is made to eliminate erroneous codes that are of no use to the end user, and the check is done only for coded variables that do not have quality control flags assigned to them.
The bounds test highlights extreme events in all variables, not just the five variables covered by the statistical test. The bounds test has been quite reliable in noting realistic extreme events like typhoons and arctic outbreaks. One disadvantage is that the routine currently uses the same bounds over the entire globe. A better scheme would vary the range of values for different climatic regions of the globe.
Figure 3. Relative humidity data from Manus island for February 1993. The B flags indicate values that exceed the upper bound for relative humidity (100%).
Table 6: Threshold speeds used to assign unrealistic platform speed flag (F). | |
Platform Type | Maximum Speed (m s-1) |
Research Vessel | 15.0 |
Drifting Buoy | 2.0 |
Anchored Buoy | 0.0 |
Land Station | 0.0 |
The dot product of the two radii can be found using
where 1 represents the angle between the two vectors in radians. Another definition of the dot product takes the sum of the products of the coefficients for each unit vector,
Equating (2) and (3) gives
Dividing (4) by r2 and taking the arccosine results in
The distance, d, between the two positions on the globe is then found by
The speed is then found by dividing the distance by the number of minutes elapsed between the two platform positions.
For all data with a time step of at least three minutes, velocities are calculated using sequential latitude and longitude values. Empirical tests showed that three minutes was the smallest useable increment for the latitude and longitude data. When the time step is one minute, too many positions are flagged because the distances traveled by the platform are so small that dividing the distance by the time often results in speeds nearing infinity. Thus one-minute data were tested using three-minute intervals.
For example, latitude and longitude plots for the R/V Knorr, Fig. 4 indicate a region of unrealistic ship speeds. The data show rapid one-minute changes in latitude from 53oS to 50.4oS near 1200 UTC on 23 February 1993. Since the data on the R/V Knorr are recorded every minute, the three-minute average speed centered near the sudden latitude jump would be nearly 1584 m s-1. This far exceeds the maximum speed for a research vessel, Table 6. As a result, the preprocessor flagged both latitude and longitude positions used to calculate the speed. Again, the preprocessor highlights a suspect portion of the data and the DQE will make the final determination on the data values to flag using the visual editor.
A major limitation of the current speed test is that the accuracy of the calculations is limited by the accuracy of the latitude and longitude data. Normally latitude and longitude are stored to the hundredth of a degree, which is equivalent to 1100 m. To determine a ship speed within a threshold of 15 m s-1, more that one- minute spacing between sequential positions is needed; thus the three-minute intervals mentioned previously.
Figure 4. Rapid changes in latitude and longitude values, like these for the R/V Knorr, are flagged for failing the platform speed test. All values within a region of rapid change are flagged.
The more common occurrence, however, is simply erroneous latitude and longitude data. For example, latitude and longitude positions for the R/V Kexue1 on 1 January 1993, Fig. 5 indicates many of the ship positions are over Australia. The most plausible explanation for the erroneous positions is that the latitude positions should be in the Northern Hemisphere, not the southern. Incorrect negative signs are a common problem with position data.
The test for movement over land is similar to the statistical test. The first step was to create a land/ocean map of the entire globe. Using the El Topo5 five-minute global topography (NOAA-NGDC), we created a land mask and assigned all grid boxes with elevations greater than or equal to zero meters to a value of one. All negative elevations were set to zero. The result was a five-minute binary grid. The land test maps the platform's latitude and longitude positions to the appropriate five-minute box in the land mask, and the land mask is checked for indications of land. When the position occurs over land, the latitude and longitude values are both flagged with an L.
The five-minute (~9 km) resolution of the land mask is the main limitation to the land test. When a research vessel enters port or passes through narrow channels (i.e. Panama Canal, Straights of Magellan, etc.), land flags are often assigned by the preprocessor. The DQE always verifies the land flags and removes any that are the result of passage through small bodies of water.
The land test often points out possible sign errors in the latitude and longitude data. The erroneous position data can then be modified after verifying the actual cruise position (via e-mail or other contacts). No changes to position data due to probable sign errors have currently been applied; however, any changes to the position data will be documented in the QC reports.
Figure 5. Position data for the R/V Kexue1 which resulted in land flags at all the positions over Australia (filled diamonds). The open diamonds are positions the land test found to be over water.
Care must be taken when using either approach as the orientation of the ship with respect to true north must be taken into account. The orientation of the ship is normally represented by the heading of the vessel, which in most cases does not equal the course of the vessel over the ground. At low ship speeds, currents and the wind will push a ship sideways through the water. As a result, the vector motion of the ship over the fixed earth rarely lies in the direction of the bow of the vessel.
A second key point regards ship winds, as it is critical to know a reference point from which the winds are measured. One common practice is to make the bow zero degrees. Other vessels, however, may orient the winds with the stern being zero. Thus all vessels should report the zero reference line for their ship winds. It is worth noting that this may or may not be the direction of the zero line on the anemometer itself (Leslie Hartten, personal communication, 1996).
As a result, the COARE DPC has determined that six parameters are necessary to accurately calculate the true wind from any vessel. These parameters include the ship wind direction and speed, the course and speed of the vessel, the heading of the vessel, and the zero reference for the ship wind direction. If provided, these six variables can be used in the following numerical calculation of the true wind.
Prior to discussing the true wind equations it is necessary to define the two coordinate systems that are used. The first is the earth coordinate system, which has zero degrees on the positive y-axis and degree values increasing in a clockwise direction. The second is the cartesian coordinate system that has zero degrees defined on the positive x-axis with degree values increasing in the counter clockwise direction. For this discussion, lower case variables are used to denote the earth coordinates and upper case variables for cartesian coordinates.
Figure 6b is a schematic representation of a research vessel in the earth coordinate system. The vessel has a heading (h) of
Figure 6. Schematic diagrams showing a) vector subtraction method of calculating the true wind and b) the six vectors and angles required to compute the true wind (T). In a) the roman numerals mark the quadrants in the cartesian coordinate system. (see text for more details)
To begin the calculation, the ship wind is referenced to true north in the earth coordinate system. Referencing the ship wind is accomplished by summing the heading, zero line angle, and ship wind direction. The resulting angle is then converted to cartesian coordinates using
S = 90o - (h + z + d)(7)
where S is the ship wind direction in cartesian coordinates (i.e. referenced to true north). Next the earth relative course (c) of the ship must be expressed in cartesian coordinates using
C = 90o - c (8)
where C is the course angle in cartesian coordinates. The true wind is then computed by summing the components of the ship relative wind and course
Tu = ws cos S + ss cos C (9a)
Tv = ws sin S + ss sin C (9b)
where Tu and Tv are the north-south and east-west components of the true wind. The true wind speed (tspd) and direction (tdir) in the earth coordinate system can then be calculated from 9a and 9b, with the true wind speed being
tspd=(T2u+T2v) (10)
and the true wind direction being
tdir 270o - atan ( Tv / Tu ) . (11)
The 270o in (11) converts the value of atan ( Tv / Tu ) to a direction from which the wind is blowing (meteorological convention) in the earth coordinate system. Care must be taken in this calculation due to the nature of the arctangent function. If (11) is computed using the FORTRAN function 'atan2 (Tv , Tu)', the correct value of the true wind direction will be returned. However, if the arctangent function "atan" is used, 180o must be subtracted from the value of (11) when the true wind vector falls in quadrant two or three in the cartesian coordinate system. (Quadrants are noted as roman numerals in Fig. 6a.)
Returning to the example outlined in Fig. 6b, the conversion to cartesian coordinates using (7) and (8) results in values of -190o and 60o for S and C respectively. Computing the wind components using (9a) and (9b) gives an
For COARE quality control, the above calculation is used to compute true wind values received at the DPC. After the calculation is complete, a comparison between the true winds reported by the research vessel and the DPC computed winds is made. If the directions differ by more than 10o, the true wind direction reported by the vessel is flagged with an E. For wind speed, an E flag is applied when differences of more than
Overall, the wind test was never applied to any COARE data sets received at the DPC due to a lack of one or more of the necessary six calculation parameters. Even the more advanced automated systems, i.e. ISS, which were designed to measure the parameters necessary for determining the true wind, failed to provide the necessary information due to instrument failures.
An example time series, Fig. 7, shows data flagged by the TTwTd test for a cruise of the R/V Vidal Gormaz. Both times flags were assigned, the data values for air temperature and wet-bulb temperature failed the TTw test. In the first case the T=1.5oC and the Tw=2.5oC and for the second case T=6.0oC and Tw=7.5oC. These flags were reviewed by the DQE, and since no supporting information for typographical or other errors was available, the flags were retained.
Figure 7. Dry air, wet-bulb, and dew point temperature data from the R/VVidal Gormaz. The values flagged with a D failed the TTwTd test (see text).
The advantages of the TTwTd test are that it highlights physically unrealistic data and can identify typographical errors. The major disadvantage is that the DQE must still review and verify all D flags.
The preprocessor assigns up to eight different flags (Table 4) to a COARE netCDF surface meteorology file. Since any data value can be assigned only one alphabetic flag, an order of precedence was created for the flags. In general, multivariate flags will overwrite univariate flags.
More specifically, the time variable may be assigned two different univariate preprocessing flags, time not sequential (C) and time duplicate (T). The methodology of the time test gives the time duplicates flag precedence over non-sequential time flag, i.e. the duplicate test is executed after the sequential test, thus any C flags are overwritten with a T when a duplicate is found. If the duplicates are exact, one is removed, and a second run of the preprocessor will then identify the non-sequential times. Of the two univariate flags not related to time, the bounds (B) flag takes precedence over the statistical (G) flag. This order arose primarily because more variable types can be checked by the bounds test than by the statistical test.
In the multivariate tests, only the platform speed and land tests apply to the same variables (latitude and longitude), with the land flag (L) having priority over the platform velocity (F). A platform moving at an unrealistic speed is irrelevant if the platform location is over dry land.
The primary design criterion for VIDAT was to provide the DQE a simple yet extensible interface to flag data displayed in time series format. A graphical user interface is common throughout VIDAT, and file access is achieved using a point and click file browser. The DQE selects the directory and file (up to five files at a time) to be edited. Having multiple files displayed allows the DQE to intercompare data from sequential dates or nearby locations. Once a file is opened, VIDAT lists the variables within the file for the DQE to select for viewing and editing (Fig. 8).
VIDAT was also designed to record all changes made to the file, create automated reports, and update file versions whenever an edited file is saved. All changes made to the opened files are stored in memory until the DQE saves the new version of the netCDF file. Storing changes in memory allows the DQE to modify the same flag more than once if desired without modifying the original file. When saved, VIDAT combines the input file with the changes stored in memory, thus creating an updated file with an incremental version number.
Three different windows for viewing data with VIDAT are presented: the map plot, the multi-file plot, and the editor. Each offers the DQE new insight into the data undergoing QC. By synthesizing the information provided in the three windows, the DQE can make an informed decision about the quality of any value in the data set.
Figure 8. Screen shot of the file access window for the VIsual Data Assessment Tool (VIDAT). The buttons across the top have drop down menu options. Three consecutive daily files from the R/V Knorr have been selected for review.
The map plot window was designed to map the position of the platform in time on the global ocean. Positions are mapped using the latitude and longitude data from the platform. There is an option to zoom in on a portion of the global map to obtain more detailed information on the platform's movement.
Another feature of the map plot window is the overlay display of the da Silva et al. (1994) climatology over the oceans. Climatologies for wind speed, air temperature, sea surface temperature, precipitation, SLP, relative humidity, and total cloud amount are available. Plotting the climatology under the platform's motion track offers the DQE guidance for flagging data in regions of the globe that may be unfamiliar.
A global map plot of the cruise track for the R/V Vidal Gormaz during a 1993 cruise, Fig. 9a, shows the cruise track clearly marked between South America and the Antarctic Peninsula. The SLP climatology is mapped over the oceans and shows the R/V Vidal Gormaz crossing the sharp pressure gradient associated with the circumpolar trough. A zoomed view of the cruise track, Fig. 9b, with numbers (1-4) identifies the direction the vessel moved along the cruise track. Note also that the zoomed cruise track clearly displays an erroneous position along the southbound leg of the cruise.
Figure 9. VIDAT screen shot of (a) the full globe cruise track map displaying a 1993 cruise of the R/V Vidal Gormaz and (b) a zoom of the same cruise track showing an erroneous ship position value. Both panels display the atmospheric pressure climatology for November (da Silva et al. 1994).
It is often necessary to look at time series of one-minute data over several days which, due to the file size limitation of 1440 records, can extend over several files. The multiple file plot allows the DQE to select a variable and plot the time series of that variable using the data from all open files. A multi-plot example of temperature data for five days from the R/V Knorr, 00:00 UTC on 21 February 1994 through 00:00 UTC on 26 February 1994, shows the trends in the temperature data on time scales greater than one day (Fig. 10).
The multiple plot window also allows an initial investigation of entire data records from a specific platform, identifying major problems, before beginning the QC flagging process. Furthermore, the climatology time series corresponding to the appropriate month and location for wind speed, air temperature (Fig. 10 - dotted line), sea surface temperature, precipitation, SLP, relative humidity, and total cloud amount can be plotted over the data time series. The climatology assists in determining whether the data values are representative for the location of the platform.
Figure 10. Screen shot of the VIDAT multiple file display window. The window shows multiple files (days) of one-minute air temperature data collected by the IMET system on the R/V Knorr. The dotted line represents the February air temperature climatology along the same cruise line.
The VIDAT editor allows the DQE to modify and add flags to FSU netCDF files. The editor was designed as a point and click interface for highlighting and flagging data values for multiple variables from up to five files, Fig. 11. This example shows the time series plots for wind speed, air temperature, and sea temperature recorded by the R/V Vidal Gormaz from 13 to 23 November 1993. Up to six selected variables may be viewed at one time. The variables available for editing appear on the buttons to the right of the plot button. The editing window allows only three of the six plots to be viewed at one time, although the other three variables can be viewed by scrolling.
Other features of the editor include zooming on specific time windows, y-axis modification, and overlaying climatology. Similar to the multiple plot window, climatology time series for wind speed, air temperature, sea surface temperature, precipitation, SLP, relative humidity, and total cloud amount can be displayed automatically for the correct month, location, and variable on any of the six graphs.
The time series plots in the editor can be viewed either with (Fig. 11) or without (Fig. 12) flags. The modification or addition of flags is limited to only one variable of the six at a time. First the variable to edit is selected, and the DQE highlights one or more data values for flagging by clicking and dragging. The selected values are highlighted and a list of possible flags appears (Fig. 12) to allow the DQE to select the appropriate flag(s). All flags are available for the DQE to add, including the preprocessor flags, but most flags added in this stage of the QC process are from the list in Table 7. In the following sections, the criterion for the use of each of the flags in Table 7 will be outlined.
Table 7: Additional QC flags assigned by DQE. | |
Purpose | Flag |
Discontinuity in data | H |
Interesting feature in data | I |
Data are erroneous - DO NOT USE | J |
Data are suspect - USE WITH CAUTION | K |
Known instrument malfunction | M |
Spike in data | S |
Data passed evaluation | Z |
Figure 12. Screen shot of R/V Vidal Gormaz data without flags displayed. The overlying window is a point and click flag selector used to flag the highlighted values.
Discontinuity
A discontinuity is defined as a sudden and dramatic shift in the data time series. The discontinuity often takes the form of a stair step. In the relative humidity data at Kapingamarangi on 17 November 1992, two discontinuities are noted (Fig. 13). For most of the day, and many of the preceding days, the relative humidity at Kapingamarangi held fairly steady between 70 and 80% (reasonable values for an island in the SW tropical Pacific Ocean). Around 1630 UTC on 17 November, the relative humidity dropped abruptly to near 30% and held near that value until 2100 UTC when the relative humidity abruptly returned to near 80%. The two sudden shifts in the relative humidity data are flagged using the H flag. Flags are placed at the beginning and ending points of the discontinuity (Fig. 13).
Discontinuities can occur for various reasons. A shift in sensor location or the replacement of one sensor with another can both cause discontinuities. However, a change of sensors or location may not result in a return of the values to their previous levels as exemplified in Fig. 13. Instead a stair step up or down in the data would occur. The cause of the discontinuity in Fig. 13 was a known sensor malfunction, which is discussed in the next section.
Figure 13. Discontinuity in the one-minute relative humidity data at the island of Kapingamarangi. The H flags are positioned at the starting and ending values of the discontinuity. In this case the discontinuity was caused by a sensor malfunction and all the values near 30% were flagged with an M.
Sensor malfunction
Sensor malfunctions are, as the name implies, a failure of the deployed sensor or data logging equipment. Malfunctions can occur for any number of reasons with breakage, poor calibration, improper maintenance, improper installation, power surges, and extreme weather events being just a few of the causes. The malfunction flag M is used only when supporting evidence of a sensor malfunction is available. Supporting information, i.e. metadata, is usually in the form of README files or personal communication with the scientists that collected the meteorological data.
In the case of the relative humidity sensor at Kapingamarangi (Fig. 13), the data source informed the DPC that the sensor failed on 17 November 1992. The sensor functioned intermittently for the next few weeks, but on the 17th returned to normal operation at 2100 UTC. For the Kapingamarangi humidity data, all values between 1630 and 2100 UTC were flagged as a malfunction.
Interesting feature
In the past, most flagging strategies only pointed out suspicious data within a set of meteorological data. Using a slightly different philosophy, the interesting feature flag was designed to mark unique features of meteorological data that have passed all other quality assurance tests. Values marked with the I flag should be considered valid, though they are often extreme. The interesting features and the variables that may be flagged in association with such an event, Table 8, are only flagged when either 1) there is evidence in more than one variable of an event or 2) metadata or independent data (e.g. satellite imagery) confirm the presence of such an event.
Table 8: Examples of variables that are often flagged for interesting features. | |
Feature | Variable Flagged |
Hurricanes/Typhoons | Pressure |
Wind Speed | |
Convective Events | Temperature |
Relative Humidity | |
Pressure | |
Wind Speed | |
Frontal Passage | Temperature |
Wind Direction and Speed |
Figure 14. Typhoons Joni, Kina, and Oli passed the island of Nandi and were noted with the interesting feature flag I. Usually the I flag is applied to multiple variables, in this case atmospheric pressure and wind speed.
Erroneous data
Values that are obviously in error are identified and assigned the erroneous data flag, J. The J flag marks data that are highly suspicious and SHOULD NOT BE USED. For example, Fig. 15 shows a four-month period at Mili Atoll where the wind speed and direction were zero. That would imply that the wind did not blow for the entire four months! A truly unlikely occurrence at any location. Since we have no evidence to corroborate a sensor malfunction, these wind speed and direction data were flagged as erroneous.
In other cases, the J flag was used when metadata indicated the data were erroneous. When other information supports that the data are incorrect, the DQE flags the data so the end user can easily identify erroneous data.
Figure 15. Wind speed and direction data from Mili Atoll were reported as zero from 01 November 1992 through 28 February 1993. All these highly unrealistic data were flagged as erroneous (J).
Suspect data
During the QC process, data values are often found that simply look suspicious or that do not fit the general trend of the remainder of the time series. Often there is no clear reason or supporting metadata to define these values as erroneous (J). These questionable data fall into a grey area; they are neither correct nor erroneous, just suspect, and are thus assigned a K flag to mark the data as suspect and urge that they be USED WITH CAUTION.
Some examples of suspect data are found on two different dates (14 February and 20 February 1993) on the island of Kapingamarangi, Fig. 16. Two different variables (atmospheric pressure and air temperature) show similar groups of suspect data values. The suspect regions in the data both follow gaps in the time series: the pressure (Fig. 16a) having lower than average values and the temperature (Fig. 16b) having higher than average values. No reason for the anomalies could be determined, and they were not flagged as discontinuities because they lack a stair step profile. As a result the data are flagged as suspect.
Figure 16. Examples of suspect pressure (a) and temperature data (b) on the island of Kapingamarangi in 1993. Regions between the arrows contain unlikely values that lacked supporting evidence for an instrument malfunction; thus they are flagged as suspect.
Spike in data
One commonality in any digitized meteorological data is the presence of spikes. Spikes are usually one or more data values that are singularly out of the general trend of the time series. For example, a large spike in the wind speed data at Minamitorishima, Fig 17a, can be seen; the values increase from a normal of less than 15 m s-1 to a peak of 50 m s-1. With no changes in the other meteorological parameters to support such an increase, the value was flagged to indicate a spike (S).
Spikes are not necessarily large in amplitude. The Kapingamarangi pressure data contained a series of rhythmic, one-millibar spikes, Fig. 17b. Though the actual cause of these spikes is unknown, speculation is they were caused by electronic interference from a neighboring instrument system. The DQE attempts to flag all spikes, regardless of magnitude.
Figure 17. Examples of (a) a single large wind speed spike at Minamitorishima and (b) rhythmic one-millibar atmospheric pressure spikes at Kapingamarangi. Both are flagged with an S.
Once all desired flag changes have been made in VIDAT by the DQE, the DQE has the option of saving the file or reverting to previous versions. If the DQE selects to save the file, a new netCDF file is written combining the structure and data from the older netCDF file with the added flag changes. The FSU version number (Table 1) is automatically incremented and a new record is added to the history file. The history record contains the number of flag changes made for each variable, the date of the changes, and the identity of the DQE.
When a netCDF file is saved and closed, a report file is produced. The report file contains another listing of the number of flags added to each variable. The DQE can add commentary to the report file. For example, the DQE places a comment line in the report that explains the "interesting feature" flags. Comments are also made whenever the DQE feels that his choice of flags will be unclear. The report files were designed to be used internally; however, DQE comments are reviewed and many are transferred to the QC reports.
Many facets of VISIT are similar to VIDAT. A graphical user interface was utilized throughout VISIT and all data access and flagging is achieved using a series of buttons, pull down menus, and user input. The use of color to highlight value differences is one of the strengths of VISIT's design. VISIT was also designed to record all changes made to the files, create automated reports, and update the file upon closing.
Two data windows are available for the DQE in VISIT: the map window and the editor. The map window displays the desired meteorological value at a specified time for all the available stations. At the time of implementation for COARE, 208 independent ship, buoy, and land stations were available for VISIT
Table 9: List of independent meteorological data sets reviewed within the VISIT analysis for COARE. | ||||
Station Mounting | Platform Name/Type | Number of Independent Data Sets | ||
Moored Buoy | ATLAS | 15 | ||
PROTEUS | 2 | |||
VAWR/IMET | 1 | |||
Drifting Bouy | Marisonde | 4 | ||
Niiler | 83 | |||
Land | CMAN | 7 | ||
GTS | 79 | |||
ISS | 4 | |||
Research Vessel | ||||
Franklin | AWS, Bridge | 2 | ||
Hakuho Maru | Bridge | 1 | ||
Kexue #1 | ISS | 1 | ||
Le Noroit | AWS | 1 | ||
Malaita | Bridge | 1 | ||
Moana Wave | AWS | 1 | ||
Natsushima | Bridge | 1 | ||
Shiyan #3 | ISS, Bridge | 2 | ||
Vickers | AWS | 1 | ||
Wecoma | AWS | 1 | ||
Xiang Yang Hong #5 | Bridge | 1 | ||
Total | 208 | |||
Note: No surface meteorological data for the R/V Alis, Kaiyo, and Keifu Maru were ever made available to the DPC. Also, due to multiple data problems, the PSS data were not reviewed in VISIT. |
Colored boxes are used to identify the values at each station (identified with WMO number or ship call sign next to box) for the selected variable. Before plotting the data, VISIT checks the flags previously added to the data by the preprocessor and VIDAT, and if the data value has been flagged as either erroneous (J) or an instrument malfunction (M), the box for that station is magenta colored. The magenta colored box is also assigned if the data for the station are missing. Two magenta boxes occur in Fig. 18. By highlighting erroneous values, instrument malfunctions, or missing data, the DQE can easily ignore these extreme values during the analysis.
Due to the cyclic nature of the wind direction, the wind data are displayed as vectors (Fig. 19). Again, before plotting, the previous flag values are checked by VISIT; however for the wind vector plot, missing values, erroneous data, or instrument malfunctions are not plotted. Any remaining data values with non - Z flags (good data) are plotted with magenta vectors. This again aids the DQE in identifying problems with the wind data.
Other features of the map window include zooming on a specified region of the map, adjusting the maximum and minimum values of the color bar, creating report files, and autosaving features. The DQE can also overlay a monthly climatology from da Silva et al. (1994) on the selected variable.
Figure 18. Screen shot of VISIT window showing dry air temperature values for 27 November 1992 at 0300 UTC. The dry air temperature was chosen by highlighting "T" in the variable list and selecting the "Map" button. The color bar identifies the value of temperature at each station and the da Silva et al. (1994) November climatology is plotted using color contours.
Figure 19. Screen shot of VISIT window displaying wind data from the COARE research vessels on 25 December 1992 at 1200 UTC. A scale vector is provided at the top of the map.
For a few special cases, direct intercomparisons were made of the data from independent platforms collocated at one observing site. These intercomparisons were not made using VISIT, but instead by creating difference plots and histograms of the meteorological values. Direct intercomparisons where made for some vessels and land stations with multiple data recording system. As an example, direct comparisons were completed for four CMAN stations that had collocated GTS observing sites. Also, intercomparisons of the wind data from multiple instruments on the R/V Franklin were completed to identify inconsistencies in the reported wind directions.
When two collocated recording stations report different values of a meteorological parameter, a V flag can be assigned to the data. The V flag is only used when no reason for the difference between the collocated stations values can be determined. Results of these intercomparisons are presented in the QC reports and technical reports from the DPC.
Once all desired flag changes have been made in VISIT, the DQE has the option of saving the file or reverting to previous versions. If the DQE selects to save the file, a new netCDF file is written combining the structure and data from the older netCDF file with the added flag changes. The FSU version number (Table 1) is automatically incremented and a new record is added to the history file. The history record contains the number of flag changes made for each variable, the date of the changes, and identity of the DQE.
The primary method to access the data will be via the world wide web. The address of the COARE DPC home page is
All reports, netCDF, and ASCII data are available along with cruise track plots and data availability lists. Feel free to browse around.
The data will also be available via anonymous ftp from
login as anonymous and use your e-mail as a password. The data are located in the "/pub/COARE/" directory. All data and reports, except cruise track plots, are available via ftp.
Users interested in having the data mailed or who have special requests/ needs may send us a message via e-mail at
or via regular mail at
NetCDF files are self-describing, network-transparent, directly accessible, and extendible. 'Self-describing' means that a netCDF file includes information about the data it contains. 'Network-transparent' means that a netCDF file is represented in a form that can be accessed by computers with different ways of storing integers, characters, and floating-point numbers. 'Direct-access' means that a small subset of a large data set may be accessed efficiently, without first reading through all the preceding data. 'Extendible' means that data can be appended to a netCDF data set without copying it or redefining its structure.
NetCDF is useful for supporting access to diverse kinds of scientific data in heterogeneous networking environments and for writing application software that does not depend on application-specific formats. A variety of analysis and display packages have been developed to analyze and display data in netCDF form.
You can obtain a copy of the latest released version of netCDF software using a WWW browser or anonymous FTP from
ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf.tar.Z
Included in this distribution are: the C source for the netCDF data access library, sources for the FORTRAN jacket library for various systems, documentation for the netCDF library and utilities in the form of a netCDF User's Guide, source for the netCDF utilities ncdump and ncgen, a directory of test programs to verify the correct implementation of the netCDF library in new environments, and a directory of XDR (eXternal Data Representation) source code for environments that do not
support XDR.
Other files about netCDF are available from the URL
http://www.unidata.ucar.edu/packages/netcdf/
and include:
README | general information about netCDF. |
FAQ | Frequently Asked Questions (with answers) about netCDF. |
utilities.txt | a list of software packages currently available or under development for manipulating and displaying netCDF data. |
guide.ps.Z | a compressed PostScript file of the NetCDF User's Guide. This is included in the netcdf.tar.Z distribution, so you don't need both. |
ncprogs.ps | a draft PostScript document describing an initial set of netCDF operator and utility programs under development. |
ncprogs.txt | an ASCII version of ncprogs.ps. |
conventions.info | a draft document of some proposed netCDF conventions. |
cdl/ | a directory containing some examples of CDL files (an ASCII representation for netCDF files). |
msdos/ | a directory containing executables and binaries for netCDF under MSDOS 5.0. These can also be built from the sources in netcdf.tar.Z, if you have the necessary Microsoft compilers. |
mac/ | a directory containing notes and Macintosh MPW makefiles for porting netCDF to an Apple Macintosh. These were contributed by Chuck Denham, U.S. Geological Survey. |
http://www.unidata.ucar.edu/packages/netcdf/mailing-lists.html
An archive of past postings to the netcdfgroup mailing list is available for searching from the netCDF home page.
A recent paper that provides a good introduction to the use of netCDF appeared in
host: crusty.er.usgs.gov
file: pub/netcdf.asce.ps
Specific questions about netCDF that are not of interest to the netcdfgroup mailing list may be sent to support@unidata.ucar.edu.
netcdf 7JDU.930202017v100 { dimensions: rec = 132 ; f_string = 10 ; ctc_string = 9 ;variables: char ctc(rec, ctc_string) ; ctc:long_name = "cruise track code" ; ctc:FORTRAN_format = "A9" ; long time(rec) ; time:long_name = "time" ; time:units = "minutes from 1-1-1980 00:00" ; time:type = 0 ; time:qcindex = 1 ; time:FORTRAN_format = "I12" ; float lat(rec) ; lat:long_name = "latitude" ; lat:units = "degrees" ; lat:convers_units = 4 ; lat:qcindex = 2 ; lat:FORTRAN_format = "F9.2" ; float lon(rec) ; lon:long_name = "longitude" ; lon:units = "degrees east" ; lon:convers_units = 4 ; lon:qcindex = 3 ; lon:FORTRAN_format = "F9.2" ; float DIR(rec) ; DIR:long_name = "earth relative wind direction (meteorological)" ; DIR:units = "degrees true" ; DIR:convers_units = 0 ; DIR:ht = -999.9 ; DIR:inst = "propeller anemometer" ; DIR:qcindex = 4 ; DIR:FORTRAN_format = "F9.0" ; float SPD(rec) ; SPD:long_name = "earth relative wind speed" ; SPD:units = "meters/second" ; SPD:convers_units = 2 ; SPD:ht = -999.9 ; SPD:inst = "propeller anemometer" ; SPD:qcindex = 5 ; SPD:FORTRAN_format = "F9.1" ; float P(rec) ; P:long_name = "atmospheric pressure" ; P:units = "millibars" ; P:convers_units = 0 ; P:ht = -999.9 ; P:type = 0 ; P:inst = "unknown" ; P:qcindex = 6 ; P:FORTRAN_format = "F9.1" ; float T(rec) ; T:long_name = "air temperature" ; T:units = "Celsius" ; T:convers_units = 0 ; T:ht = -999.9 ; T:inst = "Aspirated psychrometer" ; T:qcindex = 7 ; T:FORTRAN_format = "F9.1" ; float TD(rec) ; TD:long_name = "dewpoint temperature" ; TD:units = "Celsius" ; TD:convers_units = 0 ; TD:ht = -999.9 ; TD:inst = "Aspirated psychrometer" ; TD:qcindex = 8 ; TD:FORTRAN_format = "F9.1" ; float RH(rec) ; RH:long_name = "relative humidity" ; RH:units = "percent" ; RH:ht = -999.9 ; RH:inst = "unknown" ; RH:qcindex = 9 ; RH:FORTRAN_format = "F9.0" ; float TS(rec) ; TS:long_name = "sea temperature" ; TS:units = "Celsius" ; TS:convers_units = 0 ; TS:depth = -999.9 ; TS:type = 8 ; TS:inst = "mercury in glass thermometer" ; TS:qcindex = 10 ; TS:FORTRAN_format = "F9.1" ; short LCT(rec) ; LCT:long_name = "low cloud type" ; LCT:FORTRAN_format = "I6" ; short MCT(rec) ; MCT:long_name = "middle cloud type" ; MCT:FORTRAN_format = "I6" ; short HCT(rec) ; HCT:long_name = "high cloud type" ; HCT:FORTRAN_format = "I6" ; char flag(rec, f_string) ; flag:long_name = "quality control flags" ; flag:FORTRAN_format = "A10" ;
// global attributes: :title = "Natsushima Surface Met. Data - TOGA/COARE" ; :site = "R/V Natsushima" ; :elev = 0 ; :ID = "7JDU" ; :platform = "General surface meteorological data" ; :facility = "JAMSTEC" ; :fsu_version = "100" ; :missing_value = -9999 ; :startdate = "02 FEB 1993" ; :enddate = "18 FEB 1993" ; :Release_Date = " 1 AUG 1995" ; data:
ctc = "T/CLEG001", "T/CLEG001", "T/CLEG001", "T/CLEG001", "T/CLEG001", . . . "T/CLEG001", "T/CLEG001", "T/CLEG001", "T/CLEG001", "T/CLEG001" ; time = 6884640, 6884820, 6885000, 6885180, 6885360, 6885540, 6885720, 6885900, 6886080, 6886260, 6886440, 6886620, 6886800, 6886980, 6887160, 6887340, 6887520, 6887700, 6887880, 6888060, 6888240, 6888420, 6888600, 6888780, 6888960, 6889140, 6889320, 6889500, 6889680, 6889860, 6890040, 6890220, 6890400, 6890580, 6890760, 6890940, 6891120, 6891300, 6891480, 6891660, 6891840, 6892020, 6892200, 6892380, 6892560, 6892740, 6892920, 6893100, 6893280, 6893460, 6893640, 6893820, 6894000, 6894180, 6894360, 6894540, 6894720, 6894900, 6895080, 6895260, 6895440, 6895620, 6895800, 6895980, 6896160, 6896340, 6896520, 6896700, 6896880, 6897060, 6897240, 6897420, 6897600, 6897780, 6897960, 6898140, 6898320, 6898500, 6898680, 6898860, 6899040, 6899220, 6899400, 6899580, 6899760, 6899940, 6900120, 6900300, 6900480, 6900660, 6900840, 6901020, 6901200, 6901380, 6901560, 6901740, 6901920, 6902100, 6902280, 6902460, 6902640, 6902820, 6903000, 6903180, 6903360, 6903540, 6903720, 6903900, 6904080, 6904260, 6904440, 6904620, 6904800, 6904980, 6905160, 6905340, 6905520, 6905700, 6905880, 6906060, 6906240, 6906420, 6906600, 6906780, 6906960, 6907140, 6907320, 6907500, 6907680, 6907860, 6908040, 6908220 ; lat = 3.99 , 3.35 , 2.78 , 2.2 , 1.65 , 1.07 , 0.48 , -0.1 , -0.68 , -1.27 , -1.73 , -1.67 , -1.7 , -1.62 , -1.63 , -1.75 , -1.7, -1.67 , -1.72 , -1.65 , -1.23 , -0.65 , -0.05 , 0 , 0 , 0 , -0.02 , -0.12 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0.02 , 0 , -0.02 , -0.02 , 0.02, 0 , 0 , 0 , -0.02 , 0 , -0.02 , -0.02 , 0.02 , 0 , 0 , 0 , 0.15 , 0.02 , 0.13 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0, 0.03, 0 , 0.02 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0.02 , 0 , 0 , 0 , 0 , 0 , -0.02 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0.42 , 0.95 , 1.45 , 1.95 , 2 , 2 , 1.73 , 1.37 , 1 , 0.63 , 0.25 , 0.03 , 0 , -0.02 , -0.38 , -0.75 , -1.13 , -1.52 , -1.84 , -1.95 , -1.99 , -2, -2.5 , -3 , -3.43 , -3.92 , -4.22 , -4.6 , -5 , -5 , -5 , -4.8 , -4.35 ; lon = 157.22 , 157.07 , 156.95 , 156.85 , 156.77 , 156.65 , 156.52 , 156.38 , 156.27 , 156.15 , 156 , 156.02 , 156.03 , 156 , 156.02 , 155.99 , 155.95 , 155.94 , 155.92 , 155.87 , 156 , 156 , 156 , 156 , 156 , 156.02 , 156.03 , 156.02 , 156 , 156 , 156 , 155.99 , 156 , 156 , 156 , 156 , 156 , 156.03 , 156 , 155.94 , 156 , 156 , 156 , 156 , 156 , 156 , 156 , 155.94 , 155.94 , 155.94 , 155.94 , 155.97 , 155.94 , 156 , 156 , 155.95 , 155.99 , 155.99 , 155.99 , 155.99 , 155.99 , 155.99 , 155.99 , 155.99 , 155.99 , 155.99 , 155.99 , 155.95 , 155.99 , 155.9 , 155.99 , 155.95 , 155.99 , 155.99 , 155.99 , 155.99 , 155.99 , 155.99 , 155.99 , 155.97 , 155.99 , 155.99 , 155.99 , 155.99 , 155.99 , 155.97 , 155.97 , 155.95 , 155.99 , 155.99 , 155.97 , 155.97 , 155.99 , 155.97 , 155.99 , 155.99 , 155.99 , 155.99 , 155.99 , 155.97 , 156 , 156 , 156 , 156 , 155.95 , 155.65 , 155.33 , 155 , 154.65 , 154.3 , 154.15 , 154.12 , 154.1 , 154.43 , 154.77 , 155.08 , 155.48 , 155.84 , 155.95 , 156 , 156 , 155.73 , 155.47 , 155.25 , 154.95 , 154.72 , 154.37 , 154 , 153.55 , 153.15 , 152.73 , 152.5 ; DIR = 30 , 30 , 40 , 30 , 10 , 340 , 340 , 350 , 350 , 340 , 340 , 350 , 350 , 350 , 340 , 350 , 310 , 340 , 320 , 310 , 280 , 330 , 340 , 20 , 10 , 310 , 270 , 180 , 260 , 340 , 360 , 320 , 330 , 340 , 330 , 340 , 40 , 60 , 20 , 340 , 10 , 340 , 360 , 360 , 350 , 360 , 10 , 10 , 30 , 320 , 320 , 350 , 320 , 300 , 320 , 340 , 340 , 340 , 300 , 320 , 320 , 260 , 350 , 300 , 310, 290 , 260 , 270 , 290 , 280 , 290 , 310 , 320 , 340 , 340 , 20 , 50 , 40 , 90 , 30 , 40 , 20 , 10 , 30 , 10 , 40 , 40 , 70 , 60 , 50, 20 , 50 , 60 , 70 , 60 , 60 , 70 , 50 , 360 , 40 , 50 , 50 , 360 , 20 , 40 , 50 , 340 , 40 , 320 , 320 , 10 , 130 , 70 , 150 , 200 , 220 , 270 , 230 , 360 , 330 , 230 , 320 , 340 , 310 , 320 , 290 , 300 , 340 , 10 , 120 , 350 , 350 ;
SPD = 6.1 , 7.8 , 4.4 , 2.3 , 5 , 6.9 , 6.9 , 8.3 , 7.7 , 5.7 , 7 , 7.5 , 7 , 4.4 , 5 , 6.5 , 2.6 , 4.5 , 4.5 , 6 , 3.8 , 2.8 , 1.8 , 1.8, 2 , 2 , 3 , 1.3 , 0.5 , 3 , 7.5 , 10 , 8.5 , 4.5 , 2 , 0.5 , 0 , 2.5 , 2.8 , 3 , 5 , 2.6 , 4 , 4.5 , 4.5 , 7 , 6 , 6 , 4 , 3 , 4 , 4.7 , 2 , 4.2 , 5 , 3.5 , 3 , 2.5 , 4 , 4 , 3 , 5.5 , 2 , 4 , 3.5 , 5 , 8 , 4.2 , 4 , 8 , 6 , 6 , 6.5 , 5.5 , 9 , 7.5 , 5.5 , 3 , 3 , 3 , 2 , 4.5 , 2.5 , 3.5 , 2.5 , 6 , 3.5 , 6 , 5 , 4.5 , 4 , 3 , 3.5 , 3 , 2 , 3 , 1 , 3 , 4 , 5.7 , 5.8 , 4.9 , 4.5 , 3.5 , 4.5 , 2.2 , 1.4 , 1.6 , 1.3 , 0.9 , 1 , 2.5 , 4 , 3.4 , 4.4 , 4.5 , 2.7 , 0.9 , 4.5 , 4 , 4.9 , 5.5 , 2.5 , 5.3 , 3 , 4 , 4.9 , 4.8 , 3 , 0.8 , 6.5 , 7.1 ;
P = 1011.7 , 1008.6 , 1008.8 , 1011.3 , 1012.9 , 1011.9 , 1010.6 , 1011.7 , 1012.4 , 1010.5 , 1009.9 , 1011.9 , 1012.8, 1010.4 , 1010.4 , 1012 , 1012.8 , 1009.4 , 1008.1 , 1010.5 , 1011.6 , 1009.2 , 1008.3 , 1010 , 1010.9 , 1008.1, 1007.4 , 1009.4 , 1009.9 , 1008.1 , 1007.8 , 1010.4 , 1010.3 , 1008 , 1007.3 , 1008.9 , 1010.8 , 1008.6 , 1007.7 , 1010.2 , 1010.4 , 1008.8 , 1007.7 , 1009.7 , 1011 , 1009.7 , 1008.4 , 1010.6 , 1011.4 , 1009.4 , 1008.7 , 1010.4 , 1011.1 , 1009.3 , 1008.8 , 1011.5 , 1011.6 , 1008.9 , 1008 , 1010.4 , 1010.6 , 1009.1 , 1007.5 , 1009.8 , 1010.9 , 1008, 1008.3 , 1010.4 , 1011 , 1008.5 , 1007.6 , 1009.6 , 1010.6 , 1008.3 , 1007.7 , 1009.6 , 1010.4 , 1008.2 , 1007.6, 1009.5 , 1010.4 , 1007.9 , 1007.9 , 1010 , 1010.8 , 1008.8 , 1008.1 , 1009.8 , 1010.4 , 1007.8 , 1006.9 , 1008.8 , 1010.5 , 1008.9 , 1007.9 , 1008.8 , 1010 , 1007.6 , 1007 , 1008 , 1009.3 , 1007.3 , 1006.2 , 1008.1 , 1008.7 , 1007.4 , 1006.4 , 1007.9 , 1009.4 , 1007.9 , 1007.7 , 1008.6 , 1008.9 , 1006.7 , 1006 , 1007.9 , 1010.1 , 1008.6 , 1008.3 , 1009.8 , 1010.7 , 1009 , 1008.1 , 1010.4 , 1011.2 , 1009.1 , 1008.5 , 1010.7 , 1011.2 , 1008.1 , 1006.1 , 1008.8 ; T = 28.2 , 28.5 , 28.3 , 28.1 , 28.5 , 27.9 , 27.8 , 28 , 28.5 , 28.7 , 27.3 , 28.5 , 28.2 , 27.9 , 27.7 , 28.1 , 28.5 , 28.3 , 28.7 , 28.4 , 28.3 , 27.9 , 27.5 , 27.7 , 28.2 , 28 , 28.1 , 27.9 , 27.7 , 27.8 , 26.4 , 25.4 , 26.9 , 25.2 , 27.4 , 27.7 , 27.8 , 27.4 , 27.4 , 28.1 , 28.5 , 28.1 , 27.5 , 28 , 28 , 27.9 , 27.2 , 27.5 , 28.1 , 28 , 28.3 , 27.8 , 28 , 27.9 , 27.9 , 27.8 , 27.6 , 27.8 , 28.5 , 28 , 27.9 , 28 , 27.6 , 28 , 28.7 , 27.5 , 26 , 26.3 , 27.4 , 26.9 , 27.4 , 28.2 , 28.4 , 27 , 27.5 , 28.1 , 27.1 , 27.9 , 27.9 , 28 , 28.3 , 28.5 , 28.5 , 28.4 , 27.9 , 27.9 , 27.9 , 28 , 28.5 , 28.5 , 28.5 , 28.4 , 28.3 , 28.3 , 28.2 , 28.2 , 28.5 , 27.8 , 27.9 , 28.4 , 27.9 , 27.9 , 27.8 , 27.5 , 28.1 , 28.5 , 28.9 , 28.7 , 28.5 , 28 , 26.8 , 27.5 , 27.9 , 28.5 , 28.5 , 28.5 , 28.5 , 27.8 , 27.4 , 27.4 , 27 , 27.4 , 27.8 , 28.2 , 28.2 , 27.9 , 28.1 , 24.4 , 26.6 , 27.9 , 28.8 , 28.3 ; TD = 22.4 , 21.9 , 22.2 , 22.8 , 23.7 , 23.1 , 21.9 , 22.7 , 23.1 , 23.3 , 23.6 , 24.9 , 23.2 , 22.9 , 22.9 , 23 , 24 , 22.9 , 22.5 , 22.7 , 22.5 , 23.1 , 22.2 , 22.6 , 22.7 , 23.3 , 23 , 23.1 , 24.3 , 23.7 , 24.5 , 23.2 , 21.5 , 21.7 , 21.5 , 21 , 22.2 , 23.6 , 24.1 , 24.1 , 24.3 , 24.4 , 24.5 , 24.5 , 24.7 , 23.9 , 23.9 , 23.8 , 23.6 , 23.5 , 23.8 , 24.5 , 24.5 , 24.1 , 23.5 , 23.7, 23.2 , 23.7 , 24.3 , 24.5 , 24.2 , 24.2 , 23.5 , 24.5 , 23.6 , 24.1 , 23.5 , 23.8 , 25.2 , 22.6 , 24.4 , 24.4 , 25.2 , 24.7, 24.1 , 24.4 , 24.5 , 24.9 , 24.5 , 24.5 , 24.1 , 24.3 , 24.3 , 24.3 , 22.8 , 24.4 , 24.4 , 24.5 , 24.6 , 24.6 , 24 , 23.9 , 24.2 , 24.1 , 24.1 , 24.4 , 24.3 , 24.5 , 24.5 , 24 , 23.6 , 23.8 , 24 , 23.6 , 23.7 , 25 , 24.1 , 23.9 , 24.1 , 22.8 , 23.5 , 23.8 , 24.5 , 24 , 23.7 , 23.7 , 24 , 24.1 , 24.4 , 24.4 , 23.1 , 22.4 , 23.4 , 24.1 , 24.4 , 23.1 , 24 , 21.9 , 24.1 , 24.5 , 23.9 , 24.5 ; RH = 71 , 68 , 69 , 73 , 75 , 75 , 70 , 73 , 73 , 73 , 80 , 81 , 75 , 74 , 75 , 74 , 77 , 73 , 69 , 71 , 71 , 75 , 73 , 74 , 72 , 76 , 74 , 75 , 82 , 78 , 89 , 88 , 73 , 81 , 70 , 67 , 72 , 80 , 82 , 79 , 78 , 81 , 84 , 81 , 83 , 79 , 82 , 80 , 77 , 76 , 77 , 83 , 81 , 80 , 77 , 78 , 77 , 78 , 78 , 81 , 80 , 80 , 78 , 81 , 74 , 82 , 86 , 86 , 88 , 77 , 84 , 80 , 83 , 87 , 82 , 81 , 86 , 84 , 82 , 81 , 78 , 78 , 78 , 79 , 74 , 81 , 81 , 81 , 79 , 79 , 77 , 77 , 79 , 78 , 79 , 80 , 78 , 83 , 82 , 77 , 78 , 78 , 80 , 80 , 77 , 81 , 76 , 75 , 77 , 73 , 82 , 80 , 82 , 77 , 75 , 75 , 77 , 80 , 84 , 84 , 79 , 74 , 77 , 79 , 80 , 75 , 79 , 86 , 86 , 82 , 75 , 80 ; TS = 28.8 , 29 , 29.1 , 28.9 , 28.8 , 28.8 , 28.8 , 28.9 , 29 , 29.5 , 29.2 , 29 , 29 , 28.9 , 29 , 29 , 29.2 , 29.4 , 29.6 , 29.3 , 29 , 29 , 28.8 , 28.9 , 29.3 , 29.8 , 29.9 , 29.3 , 29.2 , 29 , 29.1 , 28.6 , 28.8 , 29 , 28.8 , 28.8 , 28.8 , 28.8 , 28.8 , 28.8 , 29 , 29.3 , 29.6 , 29.1 , 29 , 28.8 , 29 , 29 , 29 , 29.8 , 29.5 , 29.4 , 29 , 29 , 28.9 , 28.9 , 29.2 , 29.5 , 29.4 , 29.2, 29.1 , 29 , 28.9 , 29 , 29.4 , 29.4 , 29 , 28.6 , 29 , 29 , 29 , 29 , 29.2 , 29 , 28.6 , 29 , 29 , 29 , 28.8 , 29 , 29.1 , 29.3 , 29, 29.2 , 29.1 , 29 , 29 , 29 , 29.4 , 29.4 , 29.6 , 29.4 , 29.2 , 29.2 , 29 , 29 , 29.1 , 29.7 , 29.7 , 28.8 , 29.2 , 29 , 28.8 , 28.8 , 29.1 , 29.7 , 29.8 , 29.6 , 29.6 , 29.2 , 29 , 29.2 , 29.2 , 29.9 , 29.6 , 29.4 , 29.2 , 29.2 , 29 , 29 , 29.1 , 29.1 , 29.4 , 29 , 29.1 , 29.2 , 29.2 , 29.2 , 29.2 , 28.9 , 29 , 29.1 ; LCT = 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 3, 3, 3, 3, 3, 2, 2, 2, 3, 9, 9, 2, 2, 2, 2, 3, 3, 9, 3, 3, 3, 3, 3, 2, 2, 3, 2, 2, 2, 2, 2, 3, 9, 9, 9, 2, 2, 2, 3, 3, 3, 9, 9, 9, 3, 3, 3, 3, 9, 3, 9, 9, 9, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 9, 9, 9, 9, 3, 3, 9, 9, 9, 3, 2, 2, 3, 3, 3, 3, 9, 9, 3, 9, 9, 9, 3, 3, 3, 3, 3, 3, 9, 9, 9, 9, 9, 9, 9, 3, 9, 9, 3, 3, 3 ; MCT = 0, 0, 0, 0, 8, 4, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 6, 6, 6, 6, 0, 0, 6, -9999, 6, 2, 2, 2, 6, 3, 5, 5, 0, 3, 6, 6, 6, 6, 0, 0, 0, 0, 3, 3, 3, 0, 0, 6, 6, 6, 6, 0, 0, 1, 7, 7, 3, -9999, -9999, -9999, -9999, -9999, -9999, 3, -9999, -9999, 2, 2, 2, 0, 0, 6, 8, 0, 2, 2, 2, 6, 3, 3, 0, 3, 3, 6, 6, 6, 6, 0, 0, 6, 6, 6, 0, 3, 3, 5, 3, 0, 0, 0, 6, 6, 6, 6, 6, 0, 1, 7, 7, 7, -9999, 6, 6, 6, 6, 6, 0, 0, 0, -9999, 6, 6, 0, 6 ; HCT = 2, 2, 6, 6, 0, 3, 1, 2, 6, 5, 2, 2, 1, 2, 2, 2, 2, 2, 2, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, -9999, -9999, 3, 3, -9999, 7, 7, -9999, -9999, 1, 2, 2, 2, 2, 2, 2, 2, 7, 7, 7, 7, 7, 8, 2, 7, 3, 3, 3, 7, 7, -9999, 8, 8, 8, -9999, -9999, -9999, -9999, -9999, -9999, 9, -9999, -9999, -9999, -9999, -9999, 9, 9, 7, 7, 7, -9999, -9999, -9999, 7, 9, 6, 6, 7, 2, 8, 8, 8, 6, 7, 7, 7, 7, 7, 7, 8, 9, 2, 2, 2, 7, 2, 2, 2, 2, 6, 8, 7, 2, 7, 8, 7, -9999, 6, 9, 3, 3, 3, 3, 2, 2, -9999, 3, 3, 7, 9 ; flag = "ZZZZZZZZZZ", "ZZZZZZZZZZ", "ZZZZZZZZZZ", "ZZZZZZZZZZ", "ZZZZZZZZZZ", . . . "ZZZZZZZZZZ", "ZZZZZZZZZZ", "ZZZZZZZZZZ", "ZZZZZZZZZZ", "ZZZZZZZZZZ" ; }
Definitions of COARE quality control flags. | |
---|---|
Flag | Definition |
A | Original data had unknown units. The units shown were determined using a climatology or some other method. |
B | Original data were out of the range bounds outlined (Table 5). |
C | Time data are not sequential or date/time not valid. |
D | Data failed TTwTd test. In the free atmosphere, the temperature is always greater than or equal to the wet-bulb temperature, which in turn is always greater than or equal to the dew point temperature. |
E | Data failed resultant wind recomputation check. When the data set includes all variables required, a program recomputes the earth relative wind speed and direction and compares the computed values to the reported earth relative wind speed and direction. A failed test occurs when the wind direction difference is > 10o or the wind speed difference is > 5 m/s. |
F | Platform velocity unrealistic. Determined with platform position data. |
G | Data are > 4 standard deviations from the COADS climatological means (da Silva et al. 1994). Test applied only to pressure, temperature, sea temperature, relative humidity, and wind speed. |
H | Discontinuity found in data. |
I | Interesting feature found in data. More specific information on the feature is contained in the QC reports. Examples include: hurricanes, sharp sea water temperature gradients, strong convective events, etc. |
J | Data are of poor quality by visual inspection, DO NOT USE. |
K | Data suspect - USE WITH CAUTION - flag applied when the data look to have obvious errors, but no specific reason for the error can be determined. |
L | Oceanographic platform passes over land. |
M | Known instrument malfunction. |
N | Inconsistent with neighboring station. |
P | Position of platform or its movement are uncertain. Data should be used with caution. |
Q | Data arrived at the DPC already flagged as questionable. |
S | Spike in the data. Usually one or two sequential data values (sometimes up to four values) that are drastically out of the current data trend. Spikes occur for many reasons, including power surges, typographical errors, data logging problems, lightning strikes, etc. |
T | Time duplicate. |
V | Collocated data differ - use with caution. |
Z | Data passed evaluation. |