Knorr IMET Data Quality Control Report (1995)



David F. Zierden and Shawn R. Smith





World Ocean Circulation Experiment (WOCE)





Surface Meteorological Data Assembly Center

Center for Ocean Atmospheric Prediction Studies

Florida State University


June 6, 1997


Report WOCEMET 97-14




Version 1.0


Introduction:

This report summarizes the quality of surface meteorological data collected by the Knorr (identifier: KCEJ) IMET system during 9 WOCE cruises made in 1995. The data was provided to the Florida State University Data Assembly Center (DAC) in electronic format by B. Walden at the Woods Hole Oceanographic Institute. They were converted to standard DAC netCDF format and then processed using an automated data screening program which adds quality control flags to the data, highlighting potential problems. Finally, the Data Quality Evaluator reviews all the data and the preprocessor flags. Flags are then added, modified, and deleted according to the judgement of the Data Quality Evaluator and other DAC personnel. An in depth description of the WOCE quality control procedures can be found in Smith et al. (1996). The data quality control report summarizes all flags for the Knorr IMET data and explains the reasons why these flags were assigned.



Statistical Information:

The Knorr data were expected to include observations taken every minute on each of the WOCE cruises. Values for the following variables were collected on all cruises:

Time(TIME)
Latitude(LAT)
Longitude(LON)
Platform Heading(PL_HD)
Platform Speed Over Water(PL_SPD2)
Atmospheric Pressure(P)
Air Temperature(T)
Sea Temperature(TS)
Relative Humidity(RH)
Precipitation (guage height)(PRECIP)
Atmospheric Radiation(RAD)

In addition, values for the following variables were also gathered after 8/16/95.

Platform Course(PL_CRS)
Platform Speed Over Land(PL_SPD)
Earth Relative Wind Direction(DIR)
Earth Relative Wind Speed(SPD)

Details of each cruise including start and end dates, number of records, number of values, number of flags, and percentage flagged are listed in Table 1. A total of 3,122,540 values were evaluated with 138,180 flags added by the preprocesser and Data Quality Evaluator for a total of 4.43 percent of the values being flagged.

Table 1: Statistical Cruise Information

CTCDatesNumber of RecordsNumber of valuesNumber of FlagsPercentage Flagged
I__09S/00
I__08S/00
01/01/95 -01/19/9525,914285,0545,7252.01
I__09N/0001/24/95 - 03/05/9557,690634,59029,4604.64
I__05E/00
I__08N/00
03/11/95 - 04/15/9549,783547,61320,3313.71
I__03_/0004/23/95 - 06/05/9555,266607,92638,2006.28
I__04_/00
I__05W/00
I__07C/00
06/11/95 - 07/11/9541,672458,39224,1055.26
I__07N/0007/15/95 - 08/24/9549,687588,96520,3633.46



Summary:

The overall quality of the IMET data from the research vessel Knorr good except for the variables PL_SPD2, DIR, SPD and RAD. Many of the variables, however, did demonstrate both major and minor problems. Some problems occurred in multiple variables and these will be discussed first. Many variables also had unique errors. Table 2 details the distribution of flags among the different variables. Following Table 2 is a detailed discussion of the problems that were flagged for each of the variables.

Common Problems:

The meteorological data on cruises I__09S/00 and I__08S/00 were missing more than half of the time. The data present were quality controlled, but the user must filter the many missing values. Another common problem was when two or more variables had zero values simultaneously, either occurring singly or for many points in a row. These zero values were given the "J" flag. A third problem was characterized by variables which "flatlined", (i.e., reported the exact same values for a period of time). These values were also given the "J" flag. Finally, many isolated spikes occurred independently among the variables. These spikes received the "S" flag.

Table 2:
Number of Flags and Percentage Flagged for Each Variable

  
VariableBFGIHJKS Total Number of FlagsPercentage of Variable Flagged
LAT 3  253  580.02
LON 3  253  580.02
PL_HD     2  20.00
PL_CRS         0.00
PL_SPD         0.00
PL_SPD230,872     20,6871,47853,03718.94
DIR      1,467661,53314.16
SPD     11,49981,50814.22
P  15  124 21410.05
T   8 1932,0831892,4730.88
TS  11 1321041,2541,4920.53
RH1,107  5 12768411,9240.69
PRECIP     3,000392423,4341.36
RAD72,404    119 1 72,52425.9
Total Number of Flags
104,383

6

16

14

4

3,804

26,916

3,041

143,264

4.43
Percentage of All Values Flagged
3.34

0.00

0.00

0.00

0.00

0.12

0.86

0.10

4.43
 

LAT and LON:
The position variables LAT and LON had no major problems. On 08/16/95 both of these had a discontinuity where the values differed more than one degree from surrounding values. The erroneous values were given 53 "J" flags while the discontinuous points were given an "H" flag. LAT and LON also received 3 "F" flags each from the prescreener for unrealistic ship movement, caused by uncertainties or truncation error in navigation data.

PL_HD:
The platform heading data is very reliable. The resolution is excellent and no major problems were found. On two occasions PL_HD spiked to zero when all surrounding values, platform course, and platform relative wind data showed no evidence of a change in heading. These two points received the "J" flag.

PL_CRS:
This variable did not receive any flags. However, PL_CRS is computed using data from the GPS and fluctuates greatly when the platform is close to stationary. Caution should be used when dealing with data from these periods.

PL_SPD:
Platform speed over the ground data was very reliable and did not receive any flags.

PL_SPD2:
This variable experienced a variety of major problems. First of all, negative values were recorded when the ship's speed was less than 2 m/s and received "B" flags from the prescreener. The EDO speedlog which measures these values uses doppler technology to measure the speed of the ship relative to water. Waves, wind, and ocean currents may cause realistic negative values when the ship's speed is low . The user may wish to disregard these flags. Secondly, there were many spikes in data. The analyst tried to flag all spikes which deviated from the local average by more than 3 m/s, but the data were extremely noisy and some spikes may have been missed. Finally, there many periods when the data fluctuated more than 4 m/s from the mean. While some degree of fluctuation may be cause by wave motion, these periods far exceed a reasonable amount. All values during these periods were given the "K" flag. Figure 1 shows a typical plot involving all of the problems mentioned above. In general, all PL_SPD2 data should be used cautiously and the Data Quality Evaluator recommends that these data be smoothed or filtered.


Figure 1:
Typical PL_SPD2 Plot with Flags




* Visual Assessment Tool plots only an "S" and does not include flagged spikes
in the time series.

DIR:
This variable was calculated at the DAC and has two major problems. Spikes occurred when there was a sudden change of speed and or direction in the ship's motion. These accelerations influenced the measurement of the platform relative wind direction, which in turn biases the true wind direction. These spikes were flagged with an "S". At times the ship's movement was reflected to a small degree (plus or minus 20 degrees) in the DIR data. When this occurred the "K" flag was assigned while the ship's speed was less than 2 m/s. Tests have shown that errors in calculated earth relative winds are much smaller when the ship has a steady forward motion, so DIR is likely more accurate when the ship is underway (Smith et al., 1997).

SPD:
Earth relative wind speed had only one tenth as many spikes as DIR. However, the ship's motion shows up in the data more often than in DIR. SPD was flagged as described above with the "K" for the same reasons.

P:
Pressure received 15 "G" flags from the prescreener on 01/08/95 for values less than 4 standard deviations below climatology. The research vessel was located near the coast of Antarctica, an area characterized by questionable climatology and highly variable weather. The analyst believes that the flagged values are accurate, but the flags were left in place to highlight the relatively extreme event. Otherwise, P had no unique problems.

T:
On cruises I__09S/00 and I__08S/00 the temperature data were mostly missing and had values of exactly zero when present. All available temperature data was given the "J" flag on these cruises. On other cruises, the air temperature occasionally experienced jumps of over 2 degrees C at times of low platform relative wind speed and high atmospheric radiation. Radiational heating of the ship by direct sunlight may cause a buildup of heat around the instrument. When the wind was over 2 m/s this buildup did not occur. All values within these jumps were assigned the "K" flag and should be used with caution. Eight events were identified where the temperature dropped suddenly several degrees from a cold front or precipitation event. The "I" flag was assigned to these events.

TS:
The sea temperature data had numerous spikes, especially on cruise I__09N/00. Because they were so frequent, the analyst only flagged those with a magnitude greater than 0.3 degrees C with an "S". Smoothing of the TS data is recommended for this cruise. The spikes were far less frequent on other cruises and should be flagged completely.

RH:
Relative humidity received 1,107 "B" flags from the prescreener for values over 100 percent. These values are probably due to a calibration problem of the instrument rather than actual supersaturation. RH data were also affected by the ventilation problems mentioned above and were flagged appropriately with the "K" flag.

PRECIP:
The precipitation data had major problems caused by the self-siphoning rain gauge. Before 04/26/95 the gauge would leak any accumulating rainfall out in less than 24 hours. Because the gauge leaked so quickly, precipitation totals and rates computed from the data would be inaccurate. Values indicating accumulating rainfall were flagged with a "K" while values reflecting the drainage after the precipitation stopped were flagged with a "J". After 04/26/95 the gauge leaked much more slowly, but would empty completely at random. Values recorded while the gauge was emptying were given the "J" flag while precipitation events were not flagged.

RAD:
Negative values for RAD were present during nighttime hours. Based on the radiation profiles, RAD is known to be a measure of incoming solar radiation and negative values would be physically meaningless. Therefore, the negative values were assigned the "B" flag by the prescreener.


Final Comments:

The quality of the Knorr IMET data ranges from very good to fair, depending on the variable. The user should be familiar with the problems associated with each variable before attempting to use the data. The PL_SPD2 data was particularly poor and the analyst would advise against using them if at all possible. The earth relative wind data is also questionable and should be used with caution.


References:

Smith, S. R., C. Harvey, and D. M. Legler, 1996: Handbook of Quality Control Procedures and Methods for Surface Meteorology Data. WOCE Report No. 141/96, Report WOCEMET 96-1, Center for Ocean Atmospheric Prediction Studies, Florida State University, Tallahassee FL 32301

Smith, S. R., M. A. Bourassa, and R. J. Sharp, 1997: Establishing More Truth in True Winds. Submitted to J. Atmos. Ocean. Technol.