Dave Lloyd
Development and Maintenance Directorate
Archive Software Department
May 6, 2003
Prepared by: | Dave Lloyd |
---|---|
Software Engineer, | |
SAIC | |
Concurred by: | Deb Labahn |
Software Project Lead, | |
SAIC | |
Approved by: | Mike Benson |
Project Manager, | |
USGS |
Number | Date and Sections | Notes |
1 | May 6, 2003 | Document created |
2 | May 14, 2003 | Corrections and additions from peer review |
3 | May 20, 2003 | Added satellite number input parameter requirement. Added reference to channel 3b to albedo code. |
4 | Jun 5, 2003 | Updated requirements traceability matrix to reflect design and code elements. |
5 | ||
6 |
The NOAA CLAVR algorithm uses a series of tests on 2x2 arrays of pixels. These tests use a variety of channel data including: albedo, temperature, viewing angle geometry and location. The process involves testing each pixel of the array and making a cloud/no cloud decision based on a vote. For example, the Reflectance Gross Cloud Test (RGCT) test compares a 4-pixel array of channel 1 values. If all four values are less than the RGCT threshold the test result is TRUE. However, if any value is greater than the threshold, the test result is FALSE. Some tests are TRUE if at least two pixels are above or below the threshold. There are also spatial tests that compare the maximum or minimum of the four values for the test. Because of the nature of our composites, CLAVR must be reduced to single pixel tests only. This is because adjoining pixels may have come from different scenes with differing viewing angles and illumination invalidating all spatial comparisons. During testing of the prototype, it was found that eliminating spatial tests that relied on minimum or maximum of four values and reducing the remaining tests to single pixel tests produced acceptable results.
The original CLAVR-1 algorithm was implemented for the first generation of NOAA AVHRR sensors, TIROS-N through NOAA-14. The second generation, or KLM series, satellites have slightly different spectral responses in channel 1 and 2, a new channel 3A and a channel 3B which is equivalent to channel 3 of pre-KLM satellites. Since five channels are still the limit, the data stream will either contain channel 3A or 3B so tests involving channel 3 need to be modified. An experimental algorithm, CLAVR-x, was used to determine channel 3 albedo thresholds for KLM series satellites.
Another problem that surfaced during the prototype investigation was the over classification of cloud in winter scenes. When cold bare ground was encountered the tests using channel 4 temperature were failing and classifying the cold ground as cloud. Empirical results showed that a channel 4 temperature of 273K was sufficient for scenes in December. This will be implemented as a optional change to the threshold and will be user supplied so that further study can be done.
NOAA KLM User's Guide. http://www2.ncdc.noaa.gov/docs/klm/index.htm
CLAVR-x source code obtained from Andrew Heidinger, NOAA. Source is maintained in CLAVR/CLAVRX directory
gamma = acos[ cos(solz)cos(satz) + sin(solz)sin(satz)cos(relaz)]
where:
solz is solar zenith angle
satz is satellite zenith angle
relaz is relative azimuth angle
albedo = reflectance * cos(solz) / ausq
where:
solz is solar zenith angle
ausq is the mean sun/earth distance in AU's squared
C...FMFT THRESHOLD LOOK UP TABLES (200-320K) FOR LAND AND OCEAN C DATA FMFLND/ 0.00, 201 1 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 211 2 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 221 3 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 231 4 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 241 5 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 251 6 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 261 7 0.01, 0.03, 0.05, 0.08, 0.11, 0.14, 0.18, 0.23, 0.28, 0.34, 271 8 0.41, 0.48, 0.57, 0.66, 0.76, 0.87, 1.00, 1.13, 1.27, 1.42, 281 9 1.59, 1.76, 1.94, 2.14, 2.34, 2.55, 2.77, 3.00, 3.24, 3.48, 291 A 3.73, 3.99, 4.26, 4.52, 4.80, 5.08, 5.35, 5.64, 5.92, 6.20, 301 B 6.48, 6.76, 7.03, 7.30, 7.80, 7.80, 7.80, 7.80, 7.80, 7.80, 311 C 7.80, 7.80, 7.80, 7.80, 7.80, 7.80, 7.80, 7.80, 7.80, 7.80/Example of usage: channel 4 temperature T4 = 291K, f(T4) = 3.73
ADAPS | AVHRR Data Acquisition and Production System |
ASCII | American Standard Coded Information Interchange |
AVHRR | Advanced Very High Resolution Radiometer |
CCRS | Canada Centre for Remote Sensing |
CECANT | Cloud Elimination from Composites using Albedo and NDVI Trend |
CLAVR | Clear/Cloud Classification for AVHRR |
KLM | Not an acronym |
LAS | Land Analysis System |
NDVI | Normalized Difference Vegetation Index |
NOAA | National and Oceanic Atmospheric Administration |
TAE | Transportable Applications Environment |
TIROS | Television InfraRed Observation Satellite |
TMS | Triple Minus Split |
Test Number | Requirement Numbers | Description |
1a | 1.1, 1.1.1, 1.4.1, 1.4.3, 1.5, 1.5.1, 1.5.3, 2.1, 2.1.1, 2.1.3, 9.1 | In a TAE session create a local string variable and assign a string value. Tutor process with IN=@string_variable. Process should start up in tutor mode with variable assigned to the IN parameter. Exit the tutor session and create a local integer value and assign a value. Tutor process with IN=@integer_variable. A TAE-BADTYPE error should be returned. Repeat the steps for parameters THRESHOLD, BARREN and OUT |
1b | 1.2, 1.2.1, 1.2.2 | In a TAE session create a local string and assign value "NONE". Tutor process with SCALE=@string_variable. Process should start in tutor mode with variable assigned to SCALE. Repeat test with variable values of "US" and "GLOBAL", variable assignment should be accepted. Assign some random value to string_variable and tutor process. A TAE-INVSTR should be returned. Create a local integer variable and assign a value. Tutor process with SCALE=@integer_variable. A TAE-BADTYPE error should be returned. |
1c | 1.3, 1.3.1, 1.3.2 | In a TAE session create a three-value integer variable and assign it a valid date as (month day 4-digit_year). Tutor process with ACQDATE=@integer_array_variable. Process should start in tutor mode with values assigned to ACQDATE. Exit and assign a 2-digit year. Tutor process again and a TAE-RANGE error should be returned. Repeat test using invalid month values of 0 and 13, invalid day values of 0 and 32 (Note: each test should only have one invalid value at a time. Also invalid dates such as (2 30 2004) or (2 29 2003) cannot be tested at this level.) Create a local string variable and assign a value. Tutor process with ACQDATE=@string_variable. A TAE-BADTYPE error should be returned. |
1d | 1.4, 1.4.2, 1.5.2, 2.1, 2.1.2 | In a TAE session tutor process with THRESHOLD=-- BARREN=-- and OUT=--. Process should start in tutor mode with all three parameters containing null values. |
1e | 1.6, 1.6.1, 1.6.2 | In a TAE session tutor process with SATNUM=9. A TAE-RANGE error should be returned. |
3a | 3.1, 3.2, 3.3, 3.4 | In a TAE session tutor process. Check that there is level 1 help for all input parameters. Type 'help IN' and check that there is level 2 help for the IN parameter. Repeat for all other parameters. Type 'help *' and insure that there are Description/Algorithm, Examples and User Notes sections. |
4a | 4.1, 4.1.1, 4.1.2, 4.1.3, 4.2.2 | Copy one of the single scene test cases to a test directory and remove the addr file. Run process. A fatal error message stating invalid projection parameters in ddr should be issued. (Since addr is missing input is assumed to be a composite). Replace the addr and remove the mnr file. Run process. A fatal error message stating the minor frame file is missing should be issued. Replace the ddr file and edit the ddr by changing number of bands to 3. Run process. A fatal error stating number of bands for single scene must be 5. |
4b | 1.1.2, 4.2, 4.2.1, 4.6 | Copy one of the composite test cases to a test directory and remove the ddr. Run process. A fatal error message stating input image must be a LAS image should be issued. Replace ddr and edit the ddr by changing the number of bands to 14. Run process. A fatal error message stating a composite must have 13 bands. |
4c | 4.3, 4.4 | Copy one of the single scenes arch, ahdr and cal.par files to a test directory. Run AVHRRCAL using the par file and specifying SCALOPT="NONE". Run process using AVHRRCAL output file as the input with SCALE="NONE", correct acquisition date and specify an output file. Compare output .img file with cloud code .img file in the test directory. Files should match exactly. Repeat with AVHRRCAL SCALOPT values of "US" and "GLOBAL". In each case the output should match the test directory file exactly. |
4d | 4.5 | Copy one of the single scenes arch, ahdr and cal.par files to a test directory. Run AVHRRCAL using the par file and specifying ODTYPE as byte and SCALOPT="NONE". Run process using AVHRRCAL output file as the input with SCALE="NONE", correct acquisition date and specify and output file. Compare output .img file with cloud code .img file in the test directory. Files should match exactly. Repeat with AVHRRCAL ODTYPEs of I*2, I*4 and R*4. In each case the output should match the test directory file exactly. |
5a | 5.1, 5.3, 5.7 | Create a threshold file containing the single line "Rgct 44.0". Copy one of the test images to a test directory and run process specifying the newly created threshold file. Run process and compare output with test case output. Files should be exact. |
5b | 5.5 | Create a threshold file containing two lines "RGCT 1.0" and "RGCT 44.0". Run process as in test 5a. Output files should be exact. |
5c | 5.2, 5.8, 5.10 | Create a threshold file containing the single line "RRRR 1.0". Run process as in test 5a. A fatal error message stating invalid threshold name in threshold file should be issued. Repeat with threshold file containing the single line "LAT_min-60.0". The same fatal error message should be issued. |
5d | 5.4, 5.6, 5.6.1, 5.6.2, 5.6.3, 5.6.4, 5.6.5, 5.6.6, 5.6.7, 5.6.8, 5.6.9, 5.6.10, 5.6.11, 5.6.12, | Create a threshold file containing the 12 test name value pairs listed in requirements 5.6.1 through 5.6.12. Run process as in test 5a. Output files should be exact. Sort the threshold file is some other order and repeat test. Output files should be exact. |
5e | 5.9 | Create a threshold file containing the single line "C3AT". Run process as in test 5a. A fatal error message stating value for C3AT test is missing. |
5f | 5.11, 5.11.1, 5.11.2, 5.12 | Check that the 12 files CLAVR_threshold_01.dat through CLAVR_threshold_12.dat exist in ADAPSTABLES. Edit the CLAVR_threshold_06.dat file and remove the TGCT line. Copy the June 1996 single scene to a test directory and run process without specifying a threshold file. A fatal error stating test TGCT missing from ADAPSTABLES/CLAVR_threshold_06.dat should be issued. Remove the CLAVR_threshold_06.dat file and rerun test. A fatal error message stating ADAPSTABLES/CLAVR_threshold_06.dat is missing should be issued. |
6a | 6.1, 6.5 | Copy a single scene to a test directory and the barren mask. Remove the barren mask ddr and run process. A fatal error message stating barren mask file is not a LAS image should be issued. Replace the ddr and run the test again. A warning message stating barren mask cannot be used with a single scene input should be issued and process should run to completion. |
6b | 6.2 | Copy a composite scene to a test directory and the barren mask. Edit the barren mask ddr by changing the number of lines or number of samples to some other value. Run process. A fatal error stating barren mask must be same size as input image should be issued. Restore the original barren mask ddr and edit it by changing one of the projection parameters. Run process. A fatal error stating the barren mask projection must be the same as input should be issued. |
6c | 6.3, 6.4 | Copy a composite scene to a test directory and the barren mask. Map the mask values (the ones) in the barren mask to some value other than zero. Run process without specifying any barren mask. Process should run and output cloud code file will not match the test barren output file. Rerun the test specifying the mapped barren mask. A fatal error stating values out of range encountered in barren mask should be issued. |
7a | 7.1, 7.1.1, 7.1.2, 7.1.3, 7.3 | Copy a single scene and a composite scene to a test directory. Run process with each input image specifying output images. Inspect each input and output ddr. Ouput ddr must be a LAS image of BYTE data type and projection information must be the same as the input image. Output ddr must have the same number of lines and samples as input image ddr. |
7b | 7.2, 7.2.1, 7.2.2, 7.4 | Copy a single scene and a composite scene to a test directory. Run process without specifying an output image. Inspect the input image ddr's. Both image will contain an additional band, six for single scene and 14 for composite. Perform a PIXCOUNT on band six of single scene and band 14 of composite. Values must be in the set of values contained in the Clear Pixel and Cloud Pixel hexagonal boxes in App. A - Modified CLAVR Algorithm |
8a | 8.1, 8.1.1, 8.2, 8.2.1, 8.2.2, 8.3, 8.4, 8.6 | Copy a pre-KLM single scene to a test directory. Run process. Compare output with test output. Files should be the same. |
8b | 8.1.2, 8.2.3, 8.5 8.7, 8.8, 8.8.1, 8.8.2, 8.9, 8.10, 8.10.1, 8.10.2, 8.10.3, 8.10.4, 8.10.5 |
Copy a KLM composite image to a test directory. Run process. Compare output with test output. Files should be the same. |
Requirement Number | Test Number | Design | Code |
1.1 | 1a | 1.1 | clavr.pdf |
1.1.1 | 1a | 1.1 | clavr.pdf |
1.1.2 | 4b | 5 | getpar.c |
1.2 | 1b | 1.3 | clavr.pdf |
1.2.1 | 1b | 1.3 | clavr.pdf |
1.2.2 | 1b | 1.3 | clavr.pdf |
1.3 | 1c | 1.4 | clavr.pdf |
1.3.1 | 1c | 1.4 | clavr.pdf |
1.3.2 | 1c | 5 | getpar.c |
1.4 | 1d | 1.5 | clavr.pdf |
1.4.1 | 1a | 1.5 | clavr.pdf |
1.4.2 | 1d | 1.5, 5 | clavr.pdf, getpar.c |
1.4.3 | 1a | 1.5 | clavr.pdf |
1.5 | 1a | 1.6 | clavr.pdf |
1.5.1 | 1a | 1.6 | clavr.pdf |
1.5.2 | 1d | 1.6, 5 | clavr.pdf, getpar.c |
1.5.3 | 1a | 1.6 | clavr.pdf |
1.6 | 1e | 1.2 | clavr.pdf |
1.6.1 | 1e | 1.2 | clavr.pdf |
1.6.2 | 1e | 1.2, 5, 13 | clavr.pdf, getpar.c, getconstants.c |
1.7 | 1a | 2.1, 4 | clavr.pdf, combine_images.c |
1.7.1 | 1a | 2.1 | clavr.pdf |
1.7.2 | 1a | 2.1, 5 | clavr.pdf, getpar.c |
1.7.3 | 1a | 2.1 | clavr.pdf |
2.1 | 1a | 2.1, 4 | clavr.pdf, combine_images.c |
2.1.1 | 1a | 2.1 | clavr.pdf |
2.1.2 | 1d | 2.1, 5 | clavr.pdf, getpar.c |
2.1.3 | 1a | 2.1 | clavr.pdf |
3.1 | 3a | 3.1, 3.2 | clavr.pdf |
3.2 | 3a | 3.3 | clavr.pdf |
3.3 | 3a | 3.4 | clavr.pdf |
3.4 | 3a | 3.5 | clavr.pdf |
4.1 | 4a | 5 | getpar.c |
4.1.1 | 4a | 5 | getpar.c |
4.1.2 | 4a | 5 | getpar.c |
4.1.3 | 4a | 5 | getpar.c |
4.2 | 4b | 5 | getpar.c |
4.2.1 | 4b | 5 | getpar.c |
4.2.2 | 4a | 5 | getpar.c |
4.3 | 4c | 5, 9 | getpar.c, process.c |
4.4 | 4c | 5 | getpar.c |
4.5 | 4d | 5 | getpar.c |
5.1 | 5a | 6 | getthresh.c |
5.2 | 5c | 6 | getthresh.c |
5.3 | 5a | 6 | getthresh.c |
5.4 | 5d | 6 | getthresh.c |
5.5 | 5b | 6 | getthresh.c |
5.6 | 5d | 6 | getthresh.c |
5.6.1 | 5d | 6 | getthresh.c |
5.6.2 | 5d | 6 | getthresh.c |
5.6.3 | 5d | 6 | getthresh.c |
5.6.4 | 5d | 6 | getthresh.c |
5.6.5 | 5d | 6 | getthresh.c |
5.6.6 | 5d | 6 | getthresh.c |
5.6.7 | 5d | 6 | getthresh.c |
5.6.8 | 5d | 6 | getthresh.c |
5.6.9 | 5d | 6 | getthresh.c |
5.6.10 | 5d | 6 | getthresh.c |
5.6.11 | 5d | 6 | getthresh.c |
5.6.12 | 5d | 6 | getthresh.c |
5.7 | 5a | 6 | getthresh.c |
5.8 | 5c | 6 | getthresh.c |
5.9 | 5e | 6 | getthresh.c |
5.10 | 5c | 6 | getthresh.c |
5.11 | 5f | 6 | getthresh.c |
5.11.1 | 5f | 6 | getthresh.c |
5.11.2 | 5f | 6 | getthresh.c |
5.12 | 5f | 6 | getthresh.c |
6.1 | 6a | 5 | getpar.c |
6.2 | 6b | 5 | getpar.c |
6.3 | 6c | 1.6 | clavr.pdf |
6.4 | 6c | 5 | getpar.c |
6.5 | 6a | 5 | getpar.c |
7.1 | 7a | 5, 7 | getpar.c, open_images.c |
7.1.1 | 7a | 7 | open_images.c |
7.1.2 | 7a | 7 | open_images.c |
7.1.3 | 7a | 7 | open_images.c |
7.2 | 7b | 5, 4 | getpar.c, clavr.c, combine_images.c |
7.2.1 | 7b | 4 | combine_images.c |
7.2.2 | 7b | 7 | open_images.c |
7.3 | 7a | 7 | open_images.c |
7.4 | 7b | 10 | clavr_code.c |
8.1 | 8a | 9 | process.c |
8.1.1 | 8a | 9 | process.c |
8.1.2 | 8b | 9 | process.c |
8.2 | 8a | 9 | process.c |
8.2.1 | 8a | 9 | process.c |
8.2.2 | 8a | process.c | |
8.2.3 | 8b | 9 | process.c |
8.3 | 8a | 9 | process.c |
8.4 | 8a | 9, 11 | process.c, chan3temp2alb.c |
8.5 | 8b | 9, 12 | process.c, klm_chan3temp2alb.c |
8.6 | 8a | 9 | process.c |
8.7 | 8b | 9, 10 | process.c, clavr_code.c |
8.8 | 8b | 10 | clavr_code.c |
8.8.1 | 8b | 10 | clavr_code.c |
8.8.2 | 8b | 10 | clavr_code.c |
8.9 | 8b | 10 | clavr_code.c |
8.10 | 8b | 10 | clavr_code.c |
8.10.1 | 8b | 10 | clavr_code.c |
8.10.2 | 8b | 10 | clavr_code.c |
8.10.3 | 8b | 10 | clavr_code.c |
8.10.4 | 8b | 10 | clavr_code.c |
8.10.5 | 8b | 10 | clavr_code.c |
9.1 | 1a | 1, 2, 3, 4 | clavr.pdf, clavr.c |