Design for CR 5815

A change in sensor 3 should not affect channels 4 and 5 calibration coefficients. With additional modifications to: align thermal coefficient determination techniques more closely to that of NOAA (CR 6706 investigations); correct the PRT usage on sensor change (CR 6321).

AVHRR Thermal Calibration

Calibration of AVHRR thermal channels requires information that is transmitted in the normal AVHRR data stream. The data stream is broken into major and minor frames. A description of the AVHRR data stream can be found in the references listed below. All input needed for coefficient determination can be obtained from one major frame. However, in practice 11 major frames are used to compute the internal blackbody temperature. A major frame consistes of five minor frames and a minor frame is equivalent to one AVHRR scan line.

The calibration coefficients for channels 3b, 4 and 5 are calculated based on three inputs: temperature of an internal blackbody target; the sensor responses when viewing the internal blackbody target; and the sensor responses when viewing cold space. In general the coefficients are found by extracting the needed information from the data stream; converting the information to input quantities; plugging in the inputs; and evaluating the formulae. Detailed descriptions and formaulae can be found in the references cited below.

Four Platinum Resistance Thermometers (PRTs) measure the temperature of the internal blackbody target. The PRT output is a 10-bit count that must be converted to temperature by a polynomial. Each of the four PRTs has its own set of constants for the conversion. Every minor frame contains three output values from one of the PRTs or a reference value. Therefore one major frame has values from all four PRTs. The reference value, some small value always less than 10, is used to determine the beginning of a sequence of PRT values. The PRTs are always sent in PRT1, PRT2, PRT3, PRT4 sequence in the four minor frames following a minor frame containing the reference value. A filtered average of the PRT temperatures taken over several major frames, typically 11, is the internal blackbody temperature used in the computations for all three channels.

As the scan mirror presents the internal blackbody target to the channel sensors, 10 samples are taken and placed in the minor frame. Next ten space samples are placed in the minor frame when the cold space target is scanned. In one major frame there are 50 internal and 50 space count values for each of the three channels. The filtered average of the blackbody and space counts for each channel is used for calculating its coefficients.

References

  1. KLM USER's Guide TOC
  2. Description of AVHRR data frames
  3. Chapter 7.1 AVHRR Calibration
  4. Appendix D Miscellaneous Parameters for the Polar Orbiter Satellites
  5. adapssup/DOCS/KLM_thermal_cal.html

General Design

The function c_calcoef, and its support functions ccupdt and ccprim, will be modified to eliminate channel 4 and 5 coefficient changes due to a 3a/3b sensor change. This will entail a major redesign of the code. Because of the scope of this redesign, the errors in PRT usage addressed by CR 6321 will be corrected. And, at the same time, the NOAA techniques for PRT usage and filtering of PRT, space and blackbody counts will be implemented as closely as possible.

General Design Considerations

  1. Findings from CR 6706. The stated NOAA algorithm, per communication with Jerry Sullivan, is:

    The above statement mentions a 2-sigma filter. However, when applied to test data sets, a 2-sigma filter was not sufficient to remove all significant differences between NOAA and EDC coefficients. A 4-sigma filter did remove all significant differences. We are awaiting confirmation of the exact filter being used. Also there still remains a slight, unaccounted for bias in the coefficients. The EDC coefficient 0 is slightly higher than NOAA's and coefficient 1 is slightly lower. (See following table. Difference is calculated as NOAA_coefficient_value - EDC_coefficient_value)

    		    Coefficient 0                  Coefficient 1
    	Chan     Range       Difference         Range        Difference
    	  4  180.3 to 180.7  -0.001541   -0.2066 to -0.2072   1.99E-06
    	  5  197.2 to 197.6  -0.001545   -0.2107 to -0.2112   1.38E-06
    
    There is also a question about the number of PRT values that NOAA uses. The underlined passage above may mean they use all 132 PRT values from the 44 lines instead of a single value from each line.

  2. The input image may be either an AVHRR archive image or a LAS image. Acceptable LAS images are ones created by AVHRRIN that have the minor frame (.mnr suffix) and AVHRR model information (.addr suffix) stored in a Label Services ancillary file . A distinction between the two must be made and the appropriate I/O libraries used. The calibration structure includes the member file_type which identifies the input image type: either ARCHIMAGE or LASIMAGE.

  3. All file pointers and buffers must be returned to their original state prior to return. For an archive image this entails a seek to the original line number and a reread of that line. For LAS images a simple rewind to beginning is all that's necessary -- lsread can only read forward and there is no other positioning mechanism.

  4. LAS images may have been windowed and the AVHRR model will not reflect that windowing. When calculating line numbers using the model, the master line must be taken into account to insure proper alignment between the image lines and model lines. Model information is obtained from an ancillary file created by AVHRRIN (.addr suffix).

  5. For the first 30 lines of the image PRT values from the first 55 lines will be used as input. For the last 30 lines of the image PRT values from the last 55 lines will be used.

  6. If new coefficients cannot be determined: noisy data, dropped lines, etc, the previous set of coefficients will be returned. It is the responsibility of the calling routine to determine whether or not to use the coefficients. Note: The c_calcoef routine is probably a better place to determine if the coefficients are garbage and to return zero-coefficients. However, to keep from having to modify the existing routines that call c_calcoef, I think I will maintain the status quo. (unless there is a major uproar demanding the change).

  7. Illumination correction must be applied throughout the image since the solar zenith angle changes. Ideally the correction should be calculated for each line. Current code only recalculates at the beginning of a sub-block so that will be maintained here as well. I also question whether the correction should be maintained at all. Reason: Since ATMOCOR has been modified to use water vapor correction it can no longer correct an image that has illumination correction applied.

  8. The calibration routines will be called for every line of the input image even though the coefficients only change once per calibration period. The reason this practice will continue is to detect changes in the channel 3 sensor and make appropriate modifications to the calibration and coefficient structures.

Detailed Design

  1. c_calcoef
  2. The interface function for all calibration is c_calcoef. It is designed to be called for each image line to be calibrated. On success it returns updated calibration and coefficient structures. These structures are actually only updated as necessary -- on 3A/3B sensor changes and once each CALPER (the CALibration PERiod is 5 lines). For the first and last 30 lines PRT temperatures from the first and last 55 lines are used. For the rest of the image each sub-block uses the block of PRT temperatures centered on the sub-block.

    Notes:

    1. the function ccload_struct will not change
    2. the function ccillum_corr will not change
    3. the function ccsconfig will not change

    Algorithm:

  3. cc_update
  4. The cc_update function calculates the thermal coefficients. The current coefficients are saved and the proper temperature to radiance function is called to actually calculate the coefficients. A zero-coefficient check is made and the original coefficients are returned if zeros are found otherwise the updated coefficients are returned.

    Notes:

    1. the two functions: cctmp2rad and cctmp2rad3 will remain unchanged except they will use mean space and blackbody counts instead of calculating them. The means will be passed instead of the sums and count.

    Algorithm:

  5. get_prt_temps
  6. The get_prt_temps function reads five lines of data from the input file and extracts the PRT counts. Once the lines are read the file is positioned exactly as it was on entry to the function. The counts are subjected to a median of three filter to eliminate two values from each line. Image lines contain: reference value, PRT 1 count, PRT 2 count, PRT 3 count, PRT 4 count. They are always in this order but the reference won't necessarily be the first one read, eg:

        line 1	194 194 194
        line 2	203 202 202
        line 3	128   0   0
        line 4	198 199  47
        line 5	202 202 202
    

    The reference line is line 3 in the example, PRT count 1 is line 4, PRT count 3 is line 1, and so on. Also note the two 1-bit errors in line 3, value 128, and line 4, value 47. The median of three filter is used to reduce the effect of these errors. If, after the median filter is applied, one, and only one, reference line is found, set that line as the index. If more than one reference line is found, the lines are drop lines or very noisy and temperatures cannot be determined. Set the values to zero and return. Otherwise calculate the temperature for each PRT and return.

    Notes:

    1. all archio and label services functions will remain unchanged
    2. It is possible to recover some temperatures even if the index cannot be determined for the current five lines. The previous index can be saved and used as long as you can also guarantee that a line has not been inserted or dropped. The scan line time stamp may be extracted and used to determine if the scan line sequence is correct. I've included some reference to this in the algorithm. Current code does not do this and I may decide not to implement it again. Why? Because in noisy regions the possibility of the time stamp being corrupted is just as likely as the PRT counts. Also, in drop line regions time stamps are either zero or invented and the PRT values are generally all zero. Pseudocode pertaining to this has a '!' in the first column.

    Algorithm:

  7. get_mean_counts
  8. The function get_mean_counts extracts count values from five lines of data and returns the filtered mean of those counts. Read the five lines and extract the space and internal blackbody counts for each of the three thermal channels. For simplicity channel 3 data are always extracted even if the sensor is 3A. See NOAA KLM User's Guide, Sec 4.1.3 for a description of the minor frame multiplexing of space and blackbody data. Reset the file and input buffers exactly as they were on entry to the function. Call the filter function for each set of counts and return the values.

    Notes:

      a. all archio and label services functions will remain unchanged

    Algorithm:

  9. filter
  10. The filter function is yet to be determined. NOAA claims to use a gross limits filter and a 2-sigma filter on the 44 PRT temperatures, 50 space counts and 50 blackbody counts. No details were given on either of these filters. The gross limits should be relatively easy to implement to throw out extreme values. There is code currently implemented in ccupdt that does this fairly well. The 2-sigma filter is also straightforward. However, during testing it was a 4-sigma filter that produced coefficients that closely modeled NOAA coefficients.

    The 2- and 4-sigma filters were implemented in testing as:

        compute x-bar, the simple mean of the data set:
    
            x-bar = 1/n * summation i=1 to n (xsubi)
        
        compute sigma:
    
            sigma = sqrt(1/(n-1) * summation i=1 to n( xsubi - x-bar )^2)
        
        and filter:
    
            if ( abs( x-bar - value ) < SIGMA_MULT*sigma )
    	    use the value
    	else
    	    throw value away
    

    The gross limits code in ccupdt keeps a running average of the space and blackbody counts and throws out any value that is not within 25 of this average. It tends to throw away too many values towards the end of the image so it needs to be changed to keep a windowed average, perhaps the last 500 values which equates to 50 lines worth. Other than that the filter works well. A similar technique could be applied to the PRT temperatures, although the median of three filter seems to work quite well for this without the additional overhead.