Knorr IMET Data Quality Control Report
Jennifer Lovell and Shawn R. Smith
World Ocean Circulation Experiment
WOCE Surface Meteorology Data Center
Center for Ocean Atmospheric Prediction Studies
Florida State University
December 22, 2000
Report WOCEMET 01-02
Version 1.0
<>
INTRODUCTION:
This report summarizes the quality of surface meteorological data collected by the research vessel Knorr (identifier: KCEJ) during seven WOCE cruises completed in 1995, 1996 and 1997. The data were provided to the Florida State University Data Assembly Center (DAC) in electronic format by M. Lamont (WHOI) and were converted to standard DAC netCDF format. The data were then processed using an automated screening program, which added quality control flags to the data, highlighting potential problems. Finally, the Data Quality Evaluator (DQE) reviewed the data and current flags, whereby flags was added, removed, or modified according to the judgment of the DQE and other DAC personnel. Details of the WOCE quality control procedures can be found in Smith et al. (1996). The data quality control report summarizes the flags for the Knorr meteorological data, including those added by both the preprocessor and the DQE.
DATA VARIABLES:
The Knorr data are expected to include observations averaged once every minute on these WOCE cruises. Values for the following variables were collected:
Time |
(TIME) |
Latitude |
(LAT) |
Longitude |
(LON) |
Platform Heading (Gyrocompass) |
(PL_HD) |
Platform Course |
(PL_CRS) |
Platform Speed Over Ground |
(PL_SPD) |
Platform Speed Over Water |
(PL_SPD2) |
Platform Relative Wind Direction (IMET) |
(PL_WDIR) |
Platform Relative Wind Speed (IMET) |
(PL_WSPD) |
Earth Relative Wind Direction (IMET) |
(DIR) |
Earth Relative Wind Speed (IMET) |
(SPD) |
Atmospheric Pressure |
(P) |
Air Temperature |
(T) |
Sea Temperature |
(TS) |
Relative Humidity |
(RH) |
Atmospheric Radiation |
(RAD) |
Precipitation |
(PRECIP) |
1995 FLAG SUMMARY Statistical Information: Details of each 1995 cruise are listed in Table 1 and include the cruise date, number of records, number of values, number of flags, and total percentage of data flagged. A total of 1,047,931 values were evaluated with 40,429 flags added by both the preprocessor and the DQE resulting in a total of 3.86% of the values being flagged. Table 1: Statistical Cruise Information
Cruise Identifier |
Cruise Dates |
Number of Records |
Number of Values |
Number of Flags |
Percent Flagged |
I__01W/00 I__01E/00 |
08/29/95 – 09/28/95 09/30/95 – 10/13/95 |
42,974 18,669 |
730,558 317,373 |
34,316 6,113 |
4.70 1.93 |
Summary:
The 1995 IMET data from the Knorr proves to be of good quality with 3.86% of the reported values flagged for potential problems. The distribution of flags for the remaining variables are detailed in Table 2.
Table 2: Number of Flags and Percentage Flagged for Each Variable
Variable |
B |
G |
K |
S |
Total Number of Flags |
Percentage of Variable Flagged |
||
LAT LON PL_HD PL_CRS PL_SPD PL_SPD2 PL_WDIR PL_WSPD DIR SPD P T TS RH PRECIP RAD |
8,862
1
16,944 |
4 632 |
6,414 4,573 143 1,311
863 |
17 9 5 7 1
464 62 5 11 29 5 67 |
0 0 17 9 5 8,869 1 0 6,878 4,635 148 1,326 661 6 67 17,807 |
0.00 0.00 0.03 0.01 0.01 14.39 0.00* 0.00 11.16 7.52 0.24 2.15 1.07 0.01 0.11 28.89 |
||
Total Number Of Flags |
25,807 |
636 |
13,304 |
682 |
40,429 |
|||
Percent Of All Values Flagged |
2.46 |
0.06 |
1.27 |
0.07 |
3.86 |
*Percentages < 0.01
B-Flags:
Platform speed over water (PL_SPD2) was assessed 8,862 bounds flags due to negative data values recorded. The sensor on this vessel will record negative values when the ship is moving astern relative to the water surface. B-flags retained to mark negative PL_SPD2.
One B-flag was given to relative humidity (RH) for the value 100.1%.
Radiation (RAD) received 16,944 B-flags during two different WOCE cruises. I__01W/00 had 13,209 B-flags and I__01E/00 had 3,736 B-flags. The values were between zero and negative one Wm-2. These physically unrealistic negative radiation values are likely the result of the instrument not tuned to low radiation values.
G-flags:
During the I__01E/00 cruise, temperature (T) was assessed four G-flags by the preprocessor. The DQE felt these flagged values were realistic, as they were approximately ½ degree Celsius lower than the climatological data value and were left in place to highlight extreme values.
Sea temperature (TS) received 632 G-flags during the I__01W/00 cruise. These flagged values were approximately three or four degrees Celsius lower than the climatological value; therefore, the DQE feels these are realistic, though extreme, sea temperatures.
All G-flags were left in place to highlight values that are greater than four standard deviations from the climatological mean (da Silva et al. 1994).
K-flags:
The K-flag represents suspect data and should be used with caution. The most significant use of the K-flag was to reveal signatures of ship motion in the variables. The variables, earth relative wind direction (DIR), earth relative wind speed (SPD), atmospheric pressure (P), temperature (T), and atmospheric radiation (RAD) showed stair steps in the data. These stair steps are related to a change in platform course (PL_CRS), heading (PL_HD), and/or platform speed (PL_SPD) and should not exist in earth relative data. Subsequently, the data were flagged as suspect.
The earth relative wind direction (DIR) and earth relative wind speed (SPD) had stair steps in the data, which were caused by flow distortion. Flow distortion is the disturbance of airflow from other objects or instruments upstream from the anemometer.
Pressure (P) had stair steps in the data that were a result of a change in either forward speed or direction. These stair steps were associated with approximately a ½ millibar (mb) increase. The exact cause of the increase is unknown at present.
Temperature (T) was assessed K-flags due to radiational heating of the ship. When the platform relative wind speed was low, ~3 ms -1 or less, significant increases in temperature occurred during daylight hours.
During the I__01W/00 cruise, radiation (RAD) was assessed 863 K-flags. When the ship's heading ranged from 90 to 210 degrees, the radiation values would decrease. This decrease was due to a potential shadowing problem associated with the ship's position. The time of the year, September, along with the ship's position at sea, the Indian Ocean, reveal the potential for a shadowing problem. Note: The DQE is not certain exactly where on the bow tower the radiation sensor is located. Knowing the exact location is essential to verify a shadowing problem.
Spikes:
Isolated spikes occurred in most of the variables throughout the data. Spikes are a relatively common occurrence with automated data, caused by various factors (e.g. electrical interference, ship movement, etc.). These individual points were assigned the S-flag.
1996 FLAG SUMMARY
Statistical Information:
Details of the 1996 cruise are listed in Table 3 and include cruise dates, number of records, number of values, number of flags, and total percentage of data flagged. A total of 900,600 values were evaluated with 132,343 flags added by the preprocessor and the DQE resulting in 14.69% of the values being flagged.
Table 3: Statistical Cruise Information
Cruise Identifier |
Cruise Dates |
Number of Records |
Number of Values |
Number of Flags |
Percent Flagged |
AR_24_/02 |
11/02/96 – 12/05/96 |
47,400 |
900,600 |
132,343 |
14.69 |
Summary:
The 1996 IMET data from the Knorr proves to be of poor quality with 14.69% of the reported values flagged for potential problems. The distribution of flags for the remaining variables are detailed in Table 4.
Table 4: Number of Flags and Percentage Flagged for Each Variable
Variable |
B |
G |
J |
K |
S |
Total Number of Flags |
Percentage of Variable Flagged |
LAT LON PL_HD PL_CRS PL_SPD PL_SPD2 PL_WDIR PL_WSPD DIR SPD DIR2 SPD2 P T TS RH PRECIP RAD |
1 5,168
15,350 |
13 |
449 113 111
22 23 561 82 1,191 84 127 85 |
3,088
47,201 47,171 3,932 5,027 1,154 |
2 2 15 13 46 26 56 554 56 24 34 422 29 87 24 |
0 0 0 2 3 8,720 126 157 47,227 47,227 4,508 5,106 1,739 116 1,613 126 214 15,459 |
0 0 0 0.00* 0.01 18.40 0.27 0.33 99.64 99.64 9.51 10.77 3.67 0.24 3.40 0.27 0.45 32.61 |
Total Number Of Flags |
20,519 |
13 |
2,848 |
107,573 |
1,390 |
132,343 |
|
Percent Of All Values Flagged |
2.28 |
0.00* |
0.32 |
11.94 |
0.15 |
14.69 |
*Percentages < 0.01
B-flags:
Platform speed over ground (PL_SPD) had one B-flag during the AR_24_/02 cruise. The Knorr was at slow ship speed and recorded a value just below zero. Negative ship speed is highly possible when the vessel is at a slow ship speed. If a strong current is present, the ship would appear to be drifting in the opposite direction of the ship's forward motion.
Platform speed over water (PL_SPD2) received 5,168 B-flags for negative data values. The sensor on this vessel will record negative values when the ship is moving astern relative to the water surface.
Radiation (RAD) received 15,350 B-flags on the AR_24_/02 cruise. These values were between zero and negative one Wm-2. These physically unrealistic negative radiation values are likely the result of the instrument not tuned to low radiation values.
G-flags:
During the AR_24_/02 cruise, relative humidity (RH) received 13 G-flags. The DQE felt these flagged values were realistic, even though they were approximately four degrees Celsius lower than the climatological value. G-Flags were left in place to highlight extreme values.
J-flags:
Platform speed over water (PL_SPD2) received 449 J-flags during the AR_24_/02 cruise. The J-flags were placed on data values that flat-lined on one value for a period of time.
Platform relative wind direction (PL_WDIR) and platform relative wind speed (PL_SPD) received a total of 224 J-flags. The variables flat-lined on zero many times and should not be used.
During the AR_24_/02 cruise, PL_WDIR and PL_WSPD flat-lined on zero. Platform relative wind direction and platform relative wind speed are used to calculate the DIR2 and SPD2. When these values hold constant at zero (see J-flags PL_WDIR and PL_WSPD) the earth relative winds are then incorrectly calculated. Consequently, the DIR2 and SPD2 were J-flagged and should not be used.
During the AR_24_/02 cruise, J-flags were given to pressure (P), temperature (T), relative humidity (RH), precipitation (P) and atmospheric radiation (RAD) since the data values flat-lined on one value for a period of time.
Sea temperature (TS) was assessed 1,191 J-flags during AR_24_/02 cruise. Occasionally, when the ship speed was very low, the sea temperature would increase very rapidly, approximately four to nine degrees Celsius in one minute. The problem may be related to poor water flow in the seawater intake. The sea temperature would resume to previous data trend when the ship's speed increased.
K-flags:
The platform speed over water (PL_SPD2) received 3,088 K-flags during the AR_24_/02 cruise. These K-flags highlight values that were extremely noisy compared to the surrounding data.
The ocean relative wind direction (DIR) calculated by Knorr, ocean relative wind speed (SPD) calculated by Knorr, earth relative wind direction (DIR2) calculated by DAC, and the earth relative wind speed (SPD2) calculated by DAC had stair steps occurring throughout the data sets. The cause was likely due to flow distortion. Flow distortion is the disturbance of airflow from other objects or instruments upstream from the anemometer. The ocean relative winds displayed flow distortion more than the earth relative winds. The significance of the stair stepping varied throughout the data set; therefore, the earth relative winds should be used with caution.
Pressure (P) had stair steps throughout the data sets. There were some stair steps in the pressure data that were a result of a change in either forward speed or direction. These stair steps were associated with approximately a ½ millibar (mb) increase in pressure relative to both the forward speed and direction change of the ship. Other K-flags were given to highlight pressure values that increased approximately ½ millibar but did not have any meteorological or ship relative data to prove or disprove the increase. These data should be used with caution.
Spikes:
Isolated spikes occurred in all of the variables in the data. Spikes are a relatively common occurrence with automated data, caused by various factors (e.g. electrical interference, ship movement, etc.). These individual points were assigned the S-flag.
1997 FLAG SUMMARY
Statistical Information:
Details of each cruise are listed in Table 5 and include cruise dates, number of records, number of values, number of flags, and total percentage of data flagged. A total of 2,838,534 values were evaluated with 157,451 flags added by the preprocessor and the DQE resulting in 5.55% of the values being flagged.
Table 5: Statistical Cruise Information
Cruise Identifier |
Cruise Dates |
Number of Records |
Number of Values |
Number of Flags |
Percent Flagged |
A__24_/01 A__20_/00 A__22_/00 AR_24_/01 |
05/30/97 – 07/04/97 07/14/97 – 08/11/97 08/13/97 – 09/04/97 10/05/97 – 11/20/97 |
51,076 39,502 30,063 64,877 |
817,216 632,032 481,008 908,278 |
37,864 33,877 29,961 55,749 |
4.63 5.36 6.23 6.14 |
Summary:
The 1997 IMET data from the Knorr proves to be of fair quality with 5.55% of the reported values flagged for potential problems. The distribution of flags for the remaining variables are detailed in Table 6.
Table 6: Number of Flags and Percentage Flagged for Each Variable
Variable |
B |
F |
G |
J |
K |
L |
S |
Total Number of Flags |
Percentage of Variable Flagged |
LAT LON PL_HD PL_CRS PL_SPD PL_SPD2 PL_WDIR PL_WSPD DIR SPD DIR2 SPD2 P T TS RH PRECIP |
4,215
86 3,874 |
1,252 273 |
422
27 |
339 1,385 1,694 6,933 |
27,487 52,677 17,689 30,215 437 927
22 |
111 6 |
125 675 19 106 98 136 4 9 3,741 301 621 80 40 439 215 240 531 |
1,488 954 358 1,491 1,792 11,284 4 9 31,228 52,978 18,310 30,295 899 1,366 328 4,114 553 |
0.95 0.61 0.22 0.95 1.14 7.17 0.00* 0.01 19.83 33.65 11.46 18.97 0.57 0.87 0.21 2.61 0.35 |
Total Number Of Flags |
8,175 |
1,525 |
449 |
10,351 |
129,454 |
117 |
7,380 |
157,451 |
|
Percent Of All Values Flagged |
0.29 |
0.05 |
0.02 |
0.36 |
4.56 |
0.00* |
0.26 |
5.55 |
*Percentages < 0.01
Note: The first three cruises of the 1997 WOCE data have a different set of quality control variables from the fourth cruise, AR_24_/01. AR_24_/01 does not contain platform relative wind direction (PL_WDIR), platform relative wind speed (PL_WSPD), earth relative wind direction (DIR2), calculated by DAC, or earth relative wind speed (SPD2), calculated by DAC. AR_24_/01 does contain platform speed over water (PL_SPD2) and precipitation (PRECIP), where the first three cruises do not contain these variables.
B-flags:
Platform speed over water (PL_SPD2) was assessed 4,215 bounds flags since negative data values were recorded. The sensor on this vessel will record negative values when the ship is moving astern relative to the water surface. B-flags retained to mark negative PL_SPD2.
Sea temperature (TS) received 86 B-flags by the preprocessor during the A__24_/01 cruise. The B-flags were given to values of negative water temperature, zero to –1.5 degrees Celsius. This event is possible as the ship was near the coast of Greenland and may have been near a region of sea ice formation. As sea ice forms, salt is released into the ocean, increasing the salinity and lowering the freezing point.
Relative humidity (RH) was assessed 3,874 B-flags during the A__20_/00 and AR_24_/01 cruises. The flagged values were above 100%, but below 102%.
F-flags:
During the AR_24_/01 cruise, latitude (LAT) and longitude (LON) received 1,525 F-flags. These flags show that the platform speed computed by the preprocessor exceeds a realistic speed for a research vessel (15 meters per second). This may have been caused by uncertainties or truncation error in the navigation data.
G-flags:
The G-flag represents a value that is greater than four standard deviations from the climatological mean (da Silva et al. 1994). Pressure (P) was assessed 422 G-flags during the A__24_/01 cruise. These flags were left in place to highlight extreme values that were approximately four millibars lower than the climatological data value.
Sea temperature (TS) received 27 G-flags during the A__22_/00cruise. The DQE felt these flags were realistic, as they were approximately ½ degree Celsius less than the climatological value.
J-flags:
All 10,351 J-flags assessed to platform heading (PL_HD), platform course (PL_CRS), platform relative speed over ground (PL_SPD), and platform relative speed over water (PL_SPD2) occurred during the AR_24_/01 cruise. These data values were erroneous because they flat-lined on a particular value over an extended period.
K-flags:
The K-flag represents suspect data and should be used with caution. The most significant use of the K-flag was to reveal signatures of ship motion in the variables. The variables, earth relative wind direction (DIR) calculated by Knorr, earth relative wind speed (SPD) calculated by Knorr, earth relative wind direction (DIR2) calculated by DAC, earth relative wind speed (SPD2) calculated by DAC, atmospheric pressure (P), and temperature (T) showed stair steps in the data. These stair steps are related to a change in platform course (PL_CRS), heading (PL_HD), and/or platform speed (PL_SPD) and should not exist in earth relative data. Subsequently, the data were flagged as suspect.
Temperature (T) had a ventilation problem, which occurred when the platform wind direction (PL_WDIR) was from around 100 degrees or from around 180 degrees. This likely affected the flow of the air before reaching the bow-mounted thermometer. In these instances, significant increases in temperature were flagged as cautionary.
Precipitation (PRECIP) was assessed 22 K-flags during the AR_24_/01 cruise when the data values dropped and then resumed back to the prior data trend. (See Fig. 1)
Fig. 1 Precipitation data from the Knorr that shows a discontinuity in the data that was K-flagged. (Data were K-flagged from approximately 14:00 to 15:00 UTC)
L-flags:
During the AR_24_/01 cruise, the position data had serious discontinuity problems. There were many spikes in the data, which in turn gave way to many data points residing over land. Most of these data values were treated as spikes and were given the S-flag. Latitude (LAT) and longitude (LON) received 117 L-flags, land flags, to highlight erroneous data that were not a result of a spike.
Spikes:
Isolated spikes occurred in all of the variables in the data. Spikes are a relatively common occurrence with automated data, caused by various factors (e.g. electrical interference, ship movement, etc.). These individual points were assigned the S-flag.
FINAL DISSCUSSIONS: Precipitation (PRECIP) was very reliable during the 97 WOCE cruises. Other than a few spikes and the stated K-flags, the DQE felt that the 97 WOCE precipitation data is a good source of data.
REFERENCES:
Smith, S.R., C. Harvey, and D.M. Legler, 1996: Handbook of Quality Control Procedures
da Silva, A.M., C.C. Young and S. Levitus, 1994: Atlas of Surface Marine Data 1994, Volume 1:
Slutz, R.J., S.J. Lubker, J.D. Hiscox, S.D. Woodruff, R.L. Jenne, D.H. Joseph, P.M. Seurer and
and Methods for Surface Meteorology Data.WOCE Report No. 141/96,Report WOCEMET
96-1, Center for Ocean-Atmospheric Prediction Studies Florida State University,
Tallahassee FL 32306-2840
Algorithms and Procedures. NOAA Atlas Series.
J.D. Elms, 1985: COADS - Comprehensive Ocean Atmosphere Data Set,
CIRES/ERL/NCAR/NCDC, Boulder, Colorado.