Knorr IMET Data Quality Control Report (1995)
David F. Zierden and Shawn R. Smith
World Ocean Circulation Experiment (WOCE)
Surface Meteorological Data Assembly Center
Center for Ocean Atmospheric Prediction Studies
Florida State University
June 6, 1997
Report WOCEMET 97-14
Version 1.0
Introduction:
This report summarizes the quality of surface meteorological data collected by the Knorr (identifier: KCEJ) IMET
system during 9 WOCE cruises made in 1995. The data was provided to the Florida State University Data Assembly Center (DAC)
in electronic format by B. Walden at the Woods Hole Oceanographic Institute. They were converted to standard DAC netCDF format
and then processed using an automated data screening program which adds quality control flags to the data, highlighting
potential problems. Finally, the Data Quality Evaluator reviews all the data and the preprocessor flags. Flags are then added,
modified, and deleted according to the judgement of the Data Quality Evaluator and other DAC personnel. An in depth description
of the WOCE quality control procedures can be found in Smith et al. (1996). The data quality control report summarizes all flags
for the Knorr IMET data and explains the reasons why these flags were assigned.
Statistical Information:
The Knorr data were expected to include observations taken every minute on each of the WOCE cruises. Values for the following variables
were collected on all cruises:
| Time | (TIME) |
| Latitude | (LAT) |
| Longitude | (LON) |
| Platform Heading | (PL_HD) |
| Platform Speed Over Water | (PL_SPD2) |
| Atmospheric Pressure | (P) |
| Air Temperature | (T) |
| Sea Temperature | (TS) |
| Relative Humidity | (RH) |
| Precipitation (guage height) | (PRECIP) |
| Atmospheric Radiation | (RAD) |
In addition, values for the following variables were also gathered after 8/16/95.
| Platform Course | (PL_CRS) |
| Platform Speed Over Land | (PL_SPD) |
| Earth Relative Wind Direction | (DIR) |
| Earth Relative Wind Speed | (SPD) |
Details of each cruise including start and end dates, number of records, number of values, number of flags,
and percentage flagged are listed in Table 1. A total of 3,122,540 values were evaluated with 138,180 flags added by the preprocesser and Data Quality Evaluator for a total of 4.43 percent of the values being flagged.
Table 1: Statistical Cruise Information
| CTC | Dates | Number of Records | Number of values | Number of Flags | Percentage Flagged |
I__09S/00
I__08S/00 | 01/01/95 -01/19/95 | 25,914 | 285,054 | 5,725 | 2.01 |
| I__09N/00 | 01/24/95 - 03/05/95 | 57,690 | 634,590 | 29,460 | 4.64 |
I__05E/00
I__08N/00 | 03/11/95 - 04/15/95 | 49,783 | 547,613 | 20,331 | 3.71 |
| I__03_/00 | 04/23/95 - 06/05/95 | 55,266 | 607,926 | 38,200 | 6.28 |
I__04_/00
I__05W/00
I__07C/00 | 06/11/95 - 07/11/95 | 41,672 | 458,392 | 24,105 | 5.26 |
| I__07N/00 | 07/15/95 - 08/24/95 | 49,687 | 588,965 | 20,363 | 3.46 |
Summary:
The overall quality of the IMET data from the research vessel Knorr good except for the variables PL_SPD2, DIR, SPD and RAD. Many of the variables, however,
did demonstrate both major and minor problems. Some problems occurred in multiple variables and these will be discussed first. Many variables also had unique errors.
Table 2 details the distribution of flags among the different variables. Following Table 2 is a detailed discussion of the problems that were flagged for each of the variables.
Common Problems:
The meteorological data on cruises I__09S/00 and I__08S/00 were missing more than half of the time.
The data present were quality controlled, but the user must filter the many missing values. Another common
problem was when two or more variables had zero values simultaneously, either occurring singly or for many points in a row.
These zero values were given the "J" flag. A third problem was characterized by variables which "flatlined",
(i.e., reported the exact same values for a period of time). These values were also given the "J" flag. Finally, many isolated spikes occurred independently among the variables.
These spikes received the "S" flag.
Table 2: Number of Flags and Percentage Flagged for Each Variable
| Variable | B | F | G | I | H | J | K | S |
Total Number of Flags | Percentage of Variable Flagged |
| LAT | | 3 | | | 2 | 53 | | | 58 | 0.02 |
| LON | | 3 | | | 2 | 53 | | | 58 | 0.02 |
| PL_HD | | | | | | 2 | | | 2 | 0.00 |
| PL_CRS | | | | | | | | | | 0.00 |
| PL_SPD | | | | | | | | | | 0.00 |
| PL_SPD2 | 30,872 | | | | | | 20,687 | 1,478 | 53,037 | 18.94 |
| DIR | | | | | | | 1,467 | 66 | 1,533 | 14.16 |
| SPD | | | | | | 1 | 1,499 | 8 | 1,508 | 14.22 |
| P | | | 15 | | | 124 | | 2 | 141 | 0.05 |
| T | | | | 8 | | 193 | 2,083 | 189 | 2,473 | 0.88 |
| TS | | | 1 | 1 | | 132 | 104 | 1,254 | 1,492 | 0.53 |
| RH | 1,107 | | | 5 | | 127 | 684 | 1 | 1,924 | 0.69 |
| PRECIP | | | | | | 3,000 | 392 | 42 | 3,434 | 1.36 |
| RAD | 72,404 | | | | | 119 | | 1 | 72,524 | 25.9 |
| Total Number of Flags |
104,383 |
6 |
16 |
14 |
4 |
3,804 |
26,916 |
3,041 |
143,264 |
4.43 |
| Percentage of All Values Flagged |
3.34 |
0.00 |
0.00 |
0.00 |
0.00 |
0.12 |
0.86 |
0.10 |
4.43 | |
LAT and LON:
The position variables LAT and LON had no major problems. On 08/16/95 both of these had a discontinuity where
the values differed more than one degree from surrounding values. The erroneous values were given 53 "J" flags
while the discontinuous points were given an "H" flag. LAT and LON also received 3 "F" flags each from the prescreener
for unrealistic ship movement, caused by uncertainties or truncation error in navigation data.
PL_HD:
The platform heading data is very reliable. The resolution is excellent and no major problems were found.
On two occasions PL_HD spiked to zero when all surrounding values, platform course, and platform relative wind data
showed no evidence of a change in heading. These two points received the "J" flag.
PL_CRS:
This variable did not receive any flags. However, PL_CRS is computed using data from the GPS and fluctuates
greatly when the platform is close to stationary. Caution should be used when dealing with data from these periods.
PL_SPD:
Platform speed over the ground data was very reliable and did not receive any flags.
PL_SPD2:
This variable experienced a variety of major problems. First of all, negative values were recorded when the ship's
speed was less than 2 m/s and received "B" flags from the prescreener. The EDO speedlog which measures these values uses
doppler technology to measure the speed of the ship relative to water. Waves, wind, and ocean currents may cause realistic
negative values when the ship's speed is low . The user may wish to disregard these flags. Secondly, there were many spikes
in data. The analyst tried to flag all spikes which deviated from the local average by more than 3 m/s, but the data were
extremely noisy and some spikes may have been missed. Finally, there many periods when the data fluctuated more than 4 m/s
from the mean. While some degree of fluctuation may be cause by wave motion, these periods far exceed a reasonable amount.
All values during these periods were given the "K" flag. Figure 1 shows a typical plot involving all of the problems mentioned
above. In general, all PL_SPD2 data should be used cautiously and the Data Quality Evaluator recommends that these data be smoothed
or filtered.
Figure 1: Typical PL_SPD2 Plot with Flags
* Visual Assessment Tool plots only an "S" and does not include flagged spikes
in the time series.
DIR:
This variable was calculated at the DAC and has two major problems. Spikes occurred when there was a sudden change of speed and or direction in the ship's motion.
These accelerations influenced the measurement of the platform relative wind direction, which in turn biases the true wind direction. These spikes were flagged with an "S".
At times the ship's movement was reflected to a small degree (plus or minus 20 degrees) in the DIR data. When this occurred the "K" flag was assigned while the ship's speed
was less than 2 m/s. Tests have shown that errors in calculated earth relative winds are much smaller when the ship has a steady forward motion, so DIR is likely more accurate
when the ship is underway (Smith et al., 1997).
SPD:
Earth relative wind speed had only one tenth as many spikes as DIR. However, the ship's motion shows up in the data more often than in DIR. SPD was flagged as described above
with the "K" for the same reasons.
P:
Pressure received 15 "G" flags from the prescreener on 01/08/95 for values less than 4 standard deviations below climatology. The research vessel was located near the coast of
Antarctica, an area characterized by questionable climatology and highly variable weather. The analyst believes that the flagged values are accurate, but the flags were left in
place to highlight the relatively extreme event. Otherwise, P had no unique problems.
T:
On cruises I__09S/00 and I__08S/00 the temperature data were mostly missing and had values of exactly zero when present. All available temperature data was given the "J" flag
on these cruises. On other cruises, the air temperature occasionally experienced jumps of over 2 degrees C at times of low platform relative wind speed and high atmospheric radiation.
Radiational heating of the ship by direct sunlight may cause a buildup of heat around the instrument. When the wind was over 2 m/s this buildup did not occur. All values within these
jumps were assigned the "K" flag and should be used with caution. Eight events were identified where the temperature dropped suddenly several degrees from a cold front or precipitation event.
The "I" flag was assigned to these events.
TS:
The sea temperature data had numerous spikes, especially on cruise I__09N/00. Because they were so frequent, the analyst only flagged those with a magnitude greater than 0.3 degrees C with an "S".
Smoothing of the TS data is recommended for this cruise. The spikes were far less frequent on other cruises and should be flagged completely.
RH:
Relative humidity received 1,107 "B" flags from the prescreener for values over 100 percent. These values are probably due to a calibration problem of the instrument rather
than actual supersaturation. RH data were also affected by the ventilation problems mentioned above and were flagged appropriately with the "K" flag.
PRECIP:
The precipitation data had major problems caused by the self-siphoning rain gauge. Before 04/26/95 the gauge would leak any accumulating rainfall out in less than 24 hours.
Because the gauge leaked so quickly, precipitation totals and rates computed from the data would be inaccurate. Values indicating accumulating rainfall were flagged with a "K"
while values reflecting the drainage after the precipitation stopped were flagged with a "J". After 04/26/95 the gauge leaked much more slowly, but would empty completely at random.
Values recorded while the gauge was emptying were given the "J" flag while precipitation events were not flagged.
RAD:
Negative values for RAD were present during nighttime hours. Based on the radiation profiles, RAD is known to be a measure of incoming solar radiation and negative values would
be physically meaningless. Therefore, the negative values were assigned the "B" flag by the prescreener.
Final Comments:
The quality of the Knorr IMET data ranges from very good to fair, depending on the variable. The user should be familiar with the problems associated with each variable before
attempting to use the data. The PL_SPD2 data was particularly poor and the analyst would advise against using them if at all possible. The earth relative wind data is also questionable
and should be used with caution.
References:
Smith, S. R., C. Harvey, and D. M. Legler, 1996: Handbook of Quality Control Procedures and Methods for Surface Meteorology Data. WOCE Report No. 141/96, Report
WOCEMET 96-1, Center for Ocean Atmospheric Prediction Studies, Florida State University, Tallahassee FL 32301
Smith, S. R., M. A. Bourassa, and R. J. Sharp, 1997: Establishing More Truth in True Winds.
Submitted to J. Atmos. Ocean. Technol.