User's Guide

LINEDROP

Find and fix corrupt data segments and optionally fix dropped lines in an image.

Function:

Finds and fixes corrupt data segments and fixes dropped lines in an image. A corrupt segment of data is determined by evaluating the average change of adjacent pixels within a kernel window. Each corrupt pixel is replaced by the weighted average of the first valid pixels above and below it.

Parameters:

IN
Input image. Combinations of input images ("image1 + image2") may be used only if an output image file is specified. However, this is not recommended if INFILE is specified, because the same dropped line file will be used for all of the images.

OUT
Output image. If an image name is specified, the input image is copied to OUT and then repaired. If set to NULL (--), the input image itself is modified.

INFILE(--)
Input file. The input file containing the dropped lines to be fixed. The default extension is ";DRPL". This file is generated by AVHRRIN for AVHRR images. See User Note 5 for an example of the format of a dropped line file.

THRPERC(50.0)
Threshold percentage. The percentage of corrupt data allowed. This threshold value is used when deciding if a segment of pixels lying between two corrupt data segments should be considered corrupt. If the combined lengths of the corrupt segments is more than THRPERC percent of the combined lengths of the corrupt segments and the enclosed segment, the enclosed segment will be considered corrupt. THRPERC is also used to determine if a line of data has enough bad pixels in it to consider the entire line corrupt. If the percentage of corrupt pixels in a line is equal to or greater than this value, the entire line will be considered corrupt. This parameter has been tested to be effective with a value of 50.

THRVAL(220000)
Threshold value. Determines how sensitive LINEDROP will be when determining what constitutes corrupt data. If the average squared difference is greater than or equal to THRVAL, the data will be considered corrupt. For images with a maximum pixel value of 1024, it has been determined that 220000 is an effective threshold value.

KERNDIM(9)
Kernel dimension. The kernel will be one line by KERNDIM samples in size. This kernel defines the pixels to be used when determining corrupt segments. If KERNDIM is set to 11, 10 pixel differences will be used in determining if a corrupt segment has been located. KERNDIM must be an odd number.

NPIXELS(11)
Number of pixels. Specifies the number of pixels that will be replaced when a segment of corrupt data is identified. NPIXELS pixels located symmetrically about the center of the kernel will be replaced. It is recommended that NPIXELS be at least KERNDIM (all pixels within the kernel should be considered invalid). NPIXELS must be an odd number.

DROPFLG(NO)
Dropped line flag. Option to fix the line after each dropped line. The images of certain geographic regions tend to have an invalid line directly below blocks of dropped lines. This line does not have the high pixel variations of usual corrupt lines, and is not detected by LINEDROP's kernel. When processing images in these regions, DROPFLG should be YES. DROPFLG is only used if INFILE is specified.

  = YES:  Fix line after dropped lines.  The first 
          line immediately following each dropped 
          line will be fixed.
  = NO:   Do not fix line after dropped lines.  The 
          line immediately following each dropped 
          line will be fixed only if it is identified
          as a corrupt line.

MAXLFIX(7)
Maximum number of dropped lines to fix. If the number of lines in a dropped line block exceeds this number, the block will be zero-filled. MAXLFIX is only used if INFILE is specified.

PRINT(TERM)
Output destination. The destination of the output. The output includes a summary in the form (SL,SS,NL,NS:BAND) of all data blocks repaired.

  = --:        No Report
  = TERM:      Terminal.  Output is sent to the 
               user's terminal.
  = LP:        Line printer.  Output is sent to the 
               printer defined by $PRINTER.
  = Filename:  User-supplied file name.  Output is 
               sent to a disk file named 
               "FILENAME;PRT".

Examples:

  1. LAS> linedrop in=image out=-- thrperc=40 kerndim=15 npixels=17 print=hist

    Bad pixels in the input image IMAGE will be found and fixed. Since OUT is NULL, the input image will be updated. Any line with 40 percent or more bad pixels will have the entire line fixed. A 1-line by 15-sample kernel is used to determine bad pixels. When a corrupt segment is found, 17 pixels centered symmetrically about the kernel will be fixed. Since INFILE is NULL, there will be no consideration of dropped lines. A summary of the data blocks is written to the file "HIST;PRT".

  2. LAS> linedrop in=image out=outimage infile=drimage kerndim=11 npixels=19 print=outfix dropflg=yes maxlfix=30

    A new image OUTIMAGE will be created which will consist of the input image with corrupt data segments and dropped lines corrected. The dropped lines are located in the file DRIMAGE;DRPL. The line after each block of dropped lines will also be fixed, and if any blocks of dropped lines consist of more than 30 lines, they will be zero filled. For each corrupt segment identified, 19 pixels centered symmetrically about the kernel are replaced. A window 11 samples wide will be used to perform the difference checking.

Description/Algorithm:

LINEDROP finds and fixes corrupt line segments and optionally fixes dropped lines in an image. The function processes an image one line at a time, searching for bad pixels and dropped lines. A bad pixel is identified by using all of the pixels in a one dimensional kernel specified by KERNDIM. The average squared difference of all of the adjacent pixels within the kernel is calculated:

                                   N-1
                              1                   2
average squared difference = --- * SUM ( X  - X   )
                             N-1          i    i+1
                                   i=1

where X = the value of pixel i in the kernel
      N = the number of pixels in the kernel
If this calculated average exceeds the specified threshold value THRVAL, corrupt data has been identified. Pixels about the center of the kernel within a specified range (NPIXELS) are deemed bad.

Data segments may also be considered corrupt if they are surrounded by a sufficiently large number of bad pixels. If the sum of the number of pixels contained in two segments of corrupt data divided by the total number of pixels in and between the two corrupt segments exceeds a specified percentage (THRPERC), all of the pixels lying between the two corrupt data segments are also considered bad.

An entire line is considered corrupt data if the percentage of bad pixels in the line exceeds THRPERC.

A pixel identified as bad or one located in a dropped line is replaced by the average of the pixels above and below it. If the pixel above and/or below the pixel to be replaced is bad also, it will not be used in the average. The nearest valid pixel directly below the pixel to be replaced will be located and averaged with the nearest valid pixel above the pixel to be replaced.

A report of windows representing blocks of bad pixels that were fixed is sent to the user-specified destination.

Nonfatal Error Messages:

  1. [linedrop-close] Unable to close output file

    An error occurred trying to close the file specified by the input parameter PRINT. Examine the contents of the file, and rerun if necessary.

  2. [linedrop-maxline] Input values set to zero

    A block in the associated dropped line file has more lines to fix than the value in MAXLFIX. All lines in this block are set to zero. Increasing the value of MAXLFIX may correct this problem.

  3. [linedrop-vspan] Repair window (x, x, x, x : x) spans entire image vertically. Repair window will remain unchanged

    A repair window which spans the entire image (or specified image window) vertically cannot be repaired.

  4. [linedrop-warn] Nonfatal error encountered

    A nonfatal error was encountered during processing. Processing continues. The error message displayed preceding this message is the specific error that was encountered.

Fatal Error Messages:

  1. [linedrop-alloc] Error allocating dynamic memory

    An error occurred allocating memory. Rerun the function, and if the error reoccurs, contact the system manager.

  2. [linedrop-del] Error deleting null structure

    An error occurred deleting a structure within a linked list. Rerun the function, and if the error reoccurs, notify the system manager.

  3. [linedrop-fatal] Fatal error encountered

    A fatal error was encountered during processing. Processing is terminated. The error message displayed preceding this message is the specific error that was encountered.

  4. [linedrop-large] KERNDIM or NPIXELS larger than image or window

    KERNDIM or NPIXELS must not exceed the number of samples in the image or the specified image window. Respecify the appropriate parameter(s) and rerun.

  5. [linedrop-line] Too many dropped line correction values

    The number of blocks to correct specified in the dropped line file exceeds the maximum number allowed. The current limit is 1000. Decrease the number of blocks specified in INFILE.

  6. [linedrop-names] Input and output image names cannot be the same

    Respecify the parameters IN and/or OUT. To update an existing image, specify OUT to be NULL (--) .

  7. [linedrop-nbrln] Mismatched values in dropped line file

    The format of the associated dropped line file requires the line number for the first corrupt line in the block and the number of lines in the block. See User Note 5 for the correct format of the dropped line file. Correct the problem in INFILE.

  8. [linedrop-nmrimg] Invalid number of input images specified

    If an output image is specified in OUT, multiple input image files may be specified in IN. If OUT is NULL, only one input image file may be specified. Check the values of parameters IN and OUT and rerun.

  9. [linedrop-odd] KERNDIM and NPIXELS values must both be odd

    Both KERNDIM and NPIXELS must be odd numbers. Respecify the appropriate parameter(s) and rerun.

  10. [linedrop-open] Cannot open file to record blocks being fixed

    An error occurred opening the file PRINT. Check for write permission in the current directory.

  11. [linedrop-user] User termination -- linedrop is aborting

    General message informing user that processing has been terminated by the user entering <CTRL-C>.

  12. [linedrop-window] Improper use of windowing

    Windows may not be specified if INFILE is used. The windowed image may be copied to make a new image the size of the window specified. Caution should be used in this case to be sure the dropped line file corresponds to the image.

User Notes:

  1. If the user specifies input images with various data types as well as an output image, the output image will have the data type of the first input image specified.

  2. The output written to PRINT will contain repair windows which are relative to the input image window. Consequently, if IN=imgdat(5,5,10,10:2,3) and a repair window of (1,1,5,5:1) is written out:

           - line 1 in the repair window is actually referring to
                line 5 in the original image.
           - band 1 in the repair window is actually referring to
    	    band 2 in the original image.
    

  3. If the user specifies multiple input images along with a dropped line file, the dropped line file will be applied to each band of each image. No warning message will occur in this case. It is recommended that the user run each image separately if a dropped line file is to be used.

  4. The user should specify INFILE in order to effectively fix corrupt data segments that are adjacent to dropped lines. To fix corrupt data segments but not dropped lines, the user may set MAXLFIX equal to zero.

  5. The format of a dropped line file is:

        record 1   = number of blocks to fix (n), total number of 
    		 lines to fix
        record 2   = starting line of block 1, number of consecutive
    		 lines in block 1
           .
           .
           .
        record n+1 = starting line of block n, number of consecutive
    		 lines in block n
    
        Example:
                4  8
                3  1
                5  2
                8  1
               25  4
    

    In this example there are five blocks of dropped lines requiring corrections, totaling nine lines. The first dropped line to correct is line three. It will be corrected using the first valid line segment above and below this line. Lines five and six will both be corrected as well as line eight. A block of four lines starting at line twenty-five will be corrected using the first valid line segments above and below this block. If DROPFLG is YES, the line immediately following each block will be fixed. If INFILE is not specified, only corrupt lines will be located and fixed.

  6. A dropped line is an entire line consisting of zero-valued pixels. A corrupt line segment is a section of a line that is made up of invalid pixels. Corrupt segments consist of a wide variation of pixel values.

  7. LINEDROP differs from SEGMREPR and FIXLIN in that the corrupt data is located by the function; therefore, the user does not have to identify corrupt data before running LINEDROP. In comparison to LNFILL, LINEDROP fixes both corrupt lines and dropped lines, where LNFILL fixes only dropped lines. If an image runs through LINEDROP, then it does not have to run through LNFILL.