User's Guide

LNDETECT

Detect and zero bad data within an AVHRR archive image

Function:

Detect and zero bad data within an AVHRR archive image. Several statistical comparisons are made between adjacent lines of the input image to identify bad data. This data is then set to zero in the output image.

Parameters:

IN
Input image. The AVHRR archive image to be checked for bad data. If TAPEID is specified, IN must be the 16 character scene ID of the image to be read from tape.

OUT(--)
Output image. Name of the AVHRR archive image to be created. If NULL or a directory is specified without an image name, a unique name will be created using the scene ID of the input image and a time stamp code. When OUT is NULL, the output image will be placed in the current directory.

BANDS(2,4)
Bands. Bands to be used for detecting bad data. If a line is determined to be bad within any one of these bands, the entire line is considered bad in every band of the image. See User Notes 1 and 6.

TAPEID(--)
Tape ID. Identification number of the AVHRR archive tape on which the input image is located. If NULL, the input image must be located on disk.

ADJPERC(1.0)
Adjacency percentage. The number of bad pixels that must be adjacent to one another to consider the entire line as bad data. This value is expressed as a percentage of the total number of pixels in a line of data.

LINEPERC(11.0)
Line percentage. If more than LINEPERC percent of the pixels in a line of data are bad, the entire line is considered bad. This value is expressed as a percentage of the total number of pixels in a line of data.

MEANPERC(11.0,11.0)
Mean percentage. Percentage used to verify that the pixels lying above and below the current pixel are reasonably close to one another. If the difference of these two pixels is more than MEANPERC percent of their mean value, one or both of the pixels are considered bad and their values will not be used to determine the validity of the current pixel. One value must be supplied for each band specified in BANDS.

DIFPERC(22.0,22.0)
Difference percentage. Percentage used to verify the value of the current pixel with respect to the pixels lying above and below. If the absolute difference of the current pixel and the pixel lying above is greater than DIFPERC percent of the mean value of the pixels lying above and below, the current pixel is considered bad. One value must be supplied for each band specified in BANDS.

THRSMIN(0.20,0.20)
Threshold minimum. A threshold value of 3.27 * sqrt((variance of current line + variance of previous line) / number of samples) is computed for each line. If this value is less than THRSMIN, the variance within the line of data is too low to determine the status of the line using statistical methods. The status of these lines will be determined by a statistical comparison using another band from BANDS or using localized pixel tests. One value must be supplied for each band specified in BANDS.

SPOTBANDS(1,2)
Spot Noise Detection Bands. Bands that are to be examined for spot noise. All pixels determined to be spot noise will be set to zero in the output image. See User Notes 2 and 6.

SPOTPERC(60.0,60.0)
Spot percentage. If the value of the current pixel is more than SPOTPERC percent above or below the value of the pixel above or the mean value of the pixels lying on either side of it, the pixel is considered spot noise and is zeroed. If a pixel is determined to be spot noise in one band it is not automatically considered spot noise in all bands. This test will be run for each pixel in SPOTBANDS bands of the AVHRR archive image. One value must be supplied for each of the bands in SPOTBANDS. See User Note 2.

KERNWDTH(25,25)
Kernel width. Number of pixels contained in a one dimensional kernel window. If too many bad pixels are found within the kernel, the entire line is zeroed. One value must be supplied for each band specified in BANDS.

KERNPERC(60.0,60.0)
Kernel percentage. If the number of bad pixels within the kernel exceeds KERNPERC percent of the number of pixels in the kernel, the entire line is considered bad. One value must be supplied for each band specified in BANDS.

Examples:

  1. LAS> lndetect in=ah11081993130419 out=clean

    Bands two and four of the AVHRR archive image AH11081993130419 are examined and all data that does not statistically appear to belong in the image is zeroed. Spot detection is performed on bands one and two. The new archive image will be named CLEAN.

  2. LAS> lndetect in=ah12090193074053 out=-- bands=2 tapeid=047893 meanperc=11.0 difperc=22.0 thrsmin=0.20 spotbands=(1,2,3,4,5) spotperc=(60.0,60.0,60.0,60.0,60.0) kernwdth=25 kernperc=60.0

    The AVHRR archive tape 047893 is mounted and positioned at the beginning of image AH12090193074053. As the image is read from tape, all bands are examined for spot noise and band two is examined statistically for any data that does not appear to belong in the image. All bad data is zeroed.

Description/Algorithm:

The specified input image is processed in two passes. During the first pass, the bands specified by SPOTBANDS are checked for spot noise. After the spot noise has been zeroed, the mean and variance are computed for the line. These values are used to determine if the line appears to statistically belong in the image in relation to the surrounding lines. If the line appears to contain good data, additional tests are performed on each pixel to determine if the entire line is bad.

If a pixel has a value of zero or it is inconsistent with the pixel lying above, the pixel is assumed to be bad. If too many bad pixels are found within a line, or adjacent to one another, or too many are found within a kernel window, the entire line is considered bad. When a line passes all of these tests it is assumed to be a valid line of data and written to the output image.

If the algorithm finds too many bad lines in a row, it will reset itself by keeping the first line that does not look like the previous bad lines. During pass two, the first good line of data that follows each set of bad lines is examined to determine if it should be considered bad. During this pass, a line of data is considered bad if it's mean value is too different from the mean values of the lines surrounding it.

Nonfatal Error Messages:

    None

Fatal Error Messages:

  1. [lndetect-alloc] Error allocating memory

    An error occurred allocating memory. Retry and if the error persists, contact the system administrator.

  2. [lndetect-fatal] Fatal error encountered

    A fatal error was encountered. The message displayed preceding this message was the error encountered. Processing terminates.

  3. [lndetect-ngaps] Exceeded the maximum of <xxx> dropped line gaps

    Too many lines were dropped by LNDETECT. Adjust the parameters so the function is less sensitive to varying data and rerun or manually inspect the image to determine if it is worth keeping. See User Note 3.

  4. [lndetect-nvalues] Number of values for <xxxxx> not equal to <yyyyy>

    The number of values supplied for the specified parameter must match the number of values supplied for <yyyyy>. Re-specify the values for this parameter.

  5. [lndetect-sceneid] Invalid scene ID

    When reading an AVHRR image from an archive tape, the specified scene ID must begin with an 'a', the second character must be a 'g', 'h', 'l', or 'o' and it must be 16 characters long. Re-specify IN.

User Notes:

  1. BANDS specifies the bands that are to be used to detect bad data. When processing daytime data, band one or two should be used due to their sensitivity to the visible spectrum. If nighttime data is being processed, band three, four, or five should be used. During testing of the algorithm, it was found that the default bands are adequate in identifying bad data. Specifying more bands will increase processing time.

  2. Spot noise is the intermittent light or dark speckling that may occur within an image. Care must be taken when specifying the bands that should have apparent spot noise zeroed. The initial implementation was developed to reduce spot noise in all five AVHRR channels from problems as stated in User Note 4. As the use of AVHRR data expanded, it became a concern that LNDETECT may be zeroing data in the thermal channels that is useful in detecting forest fires.

  3. To decrease the algorithm's sensitivity to bad data, the values of ADJPERC, LINEPERC, MEANPERC, DIFPERC, SPOTPERC, and KERNPERC should be increased. Likewise, their values should be decreased to increase the algorithm's sensitivity. A manual inspection of the image may be required to determine which parameter values should be changed.

    MEANPERC and DIFPERC can be used to control the sensitivity to pixels that do not appear to belong in the image.

    ADJPERC, LINEPERC, SPOTPERC, and KERNPERC can be used to control the sensitivity to bad pixels. By increasing these values, the algorithm will allow more bad pixels per line before the entire line is considered bad.

  4. Bad data may be caused by numerous problems such as an antenna that cannot properly track the satellite directly overhead, tracking the satellite through an electrical storm, attempting to acquire data from the satellite while it's too close to the horizon, and/or reception hardware problems, etc.

    This function zeros all data that it determines is bad. This is considered the best approach in dealing with bad data when it will be used to create a composite image. When creating a composite image, good data contained in another input image will be used in the output image if bad data has been zeroed. If the bad data were to be fixed by averaging surrounding pixel values, this computed data might be used in the output image even if there was valid data in another input image.

  5. Specifying fewer bands in BANDS and SPOTBANDS will decrease processing time while specifying more bands will increase processing time.

  6. The band numbers specified in BANDS and SPOTBANDS should be supplied in ascending order. When the values for MEANPERC, DIFPERC, THRSMIN, KERNWDTH, KERNPERC, and SPOTPERC are retrieved, they are assigned to the specified bands in numerical order.