The AVHRR Data Acquisition and Processing System (ADAPS) is designed to receive, archive, and process Advanced Very High Resolution Radiometer (AVHRR) data from National Oceanic Atmospheric Administration (NOAA) Tiros-N polar orbiting satellites.
ADAPS was developed as part of the Land Analysis System (LAS) as implemented at the U.S. Geological Survey, National Mapping Division's EROS Data Center (EDC). Other ADAPS or LAS environments may differ slightly in hardware configuration, system limitations, and operational procedures. ADAPS and LAS also use the Transportable Application Executive (TAE) as the user interface.
The development of ADAPS began in May of 1986 at EDC as a cooperative agreement with NOAA to establish and operate a reception and processing system for AVHRR data. The original goal was to ensure that limited quantities of geographically referenced AVHRR data for the conterminous United States were routinely available within 24 hours of acquisition from the NOAA satellites. This data was used in support of the federal earth science research and land management programs.
ADAPS originally used portions of a software package developed by the Remote Sensing Group of the Rosentheil School of Marine and Atmospheric Science (RSMAS) at the University of Miami. EDC development efforts for ADAPS involved development of acquisition software, modifying the RSMAS software, and augmenting it with selected LAS modules. The system became operational in May of 1987.
In 1988 and 1989, the USGS began experimental production of weekly and biweekly maximum normalized difference vegetation index (NDVI) composites from AVHRR data. These composites were used to monitor vegetation conditions over portions of North America. In early 1990, the project area was extended to the conterminous United States. The expanded study area and corresponding increase in data volume created the need to increase the efficiency of the processing flow and improve the geometric accuracy of the generated products.
An AVHRR image was precision corrected by displaying World Data Bank II (WDBII) line-work over an AVHRR image and moving the line-work, in at least three locations, to match coast lines or other features within the image. Using this method a one to two pixel accuracy was obtained. To increase the accuracy, a base image was created for an area of interest and each processed image was correlated to this base image. Using this method, an accuracy of 0.5 to 1 pixel was obtained.
By late 1990, the usefulness and demand for AVHRR data for land management and research prompted the purchase of a DOMSAT receiving antenna. The antenna allowed the acquisition of recorded Local Area Coverage (LAC) that is relayed over the DOMSAT link from Wallops Island, Virginia or Gilmore Creek, Alaska. The acquisition software for DOMSAT was added to ADAPS and the processing flow from acquisition to archiving was automated to keep up with the increased data flow. The project areas also increased to include all of North America and the Newly Independent States (NIS, former Soviet Union). Again, efficiencies in the generation of the composite products were also added to ADAPS.
In Early 1991 the only remaining dependency on the RSMAS software was the satellite model. The satellite model allows a user to geo-reference (find a latitude and longitude) any point in an image that is in the satellite perspective. To obtain a better understanding of the processing and accuracies of EDC products, a satellite model was researched and developed at EDC. In January of 1992, all processing was moved from a VMS/VAX DEC platform to a Silicon Graphics UNIX platform. The new model, a fully automated acquisition and archiving process, and a stream-lined geo-registered product process were also implemented as part of the move.
By April of 1992 an agreement to acquire all daytime NOAA-11 AVHRR data was in place between the USGS, NOAA, European Space Agency (ESA), Canadian Center for Remote Sensing (CCRS), and 21 receiving stations located around the world. All data acquired by the receiving stations is sent to and archived at EDC and ESA. Each acquired image is stitched into half-orbits to eliminate overlap between acquisitions (14 half-orbits per day). Refer to the Orbital Stitching Overview for more detail on orbital stitching. These stitched images are processed into global 10-day NDVI composites.
The process of creating a global base image is a very large undertaking to obtain a precise geometrically corrected product. A new, fully automated, method of precision correction called navigation was implemented in ADAPS. Each AVHRR image is correlated to a source of vector data or an AVHRR base image. The corrections needed to correlate the AVHRR image to the vector data or base image are used to update the satellite model. Using the navigation process, a 0.5 to 1.3 pixel accuracy was obtained in the geo-registered product.
By early 1994 global processing and navigation were in place as part of ADAPS. Atmospheric correction capabilities (Rayleigh and ozone) were also added to improve the radiometry of the geo-registered products.
There are three main parts to the ADAPS system:
The acquisition of AVHRR data is currently operational on a DEC VAX 3800. This system contains four gigabytes of disk space for acquisition and transfer, 16 megabytes of memory, acquisition hardware (antenna, frame sync, ...), a High Density Tape (HDT) recorder for off line acquisitions, National Bureau of Standards (NBS) time clock for satellite time corrections, and an Ethernet connection for image transfers to a Silicon Graphics system for processing.
The acquisition software looks at a "master schedule" to determine which images are to be acquired. When an image is acquired it is placed in a temporary network directory. A process in the background monitors the network directory for images and transfers them to the processing system. For more information on acquisition processing refer to Appendix A: Acquisition Dependancies of the System Managers Guide, XSCHED, and SCHEDIT User's Guides.
The EDC AVHRR processing system is a Silicon Graphics Challenge Series computer with an 8 processor CPU (100 MHZ), 256 megabytes of memory, five gigabytes of disk space for ingesting, reference aid processing, and archiving. On average, the system processes (acquire to archive) 80 images per day with an average image size of 55 megabytes. 50 gigabytes of disk space for daily and global geo-registered product generation.
Images received from the acquisition system, other receiving stations, or in Level-1B format are ingested and reformatted to the EDC AVHRR archive format. Metadata is calculated for each image and a band from the image is used to generate a quick-look print, browse image, and a microfiche chip. The quick-look print, browse image, and microfiche chip are used for image quality assessment and to place orders for the image data. The EDC AVHRR archive images are stacked on ANSI labeled 3480 tapes. Refer to ingesting and archiving in the ADAPS Programmer's Guide for more information. Also refer to the reference aid generation User's Guide.
An additional 50 gigabytes of disk space is reserved on the AVHRR processing system for the generation of products. These products include: geo-registered images, and 10 day NDVI composites of North America, NIS, and the globe.
ADAPS applications related to the AVHRR model and calibration, and LAS image processing applications are used in TAE procedure definition files (PDFs) to generate geo-registered products. These procedure PDFs are parameterized and easily customized to generate specific products. See the US94 PDF and User's Guide for an example of a geo-registered product procedure. For more information on product generation refer to the Geo-registered Products section of the ADAPS Overview document.
Other peripherals local to EDC are:
The AVHRR is a five-band scanner carried on board the NOAA Tiros-N meteorological satellites. The satellites operate in a near-polar, sun-synchronous orbit with a period of 102 minutes. This results in 14.1 orbits per day. The local solar time of an overpass for a given satellite is essentially unchanged from day-to-day for a given latitude. The sub-orbital tracks do not repeat daily however, due to the swath width and the satellite altitude, daily global coverage can still be obtained.
The AVHRR data have ten bit precision and are transmitted directly from the satellite to a ground receiving station. When the data are transmitted in real-time, they are referred to as High Resolution Picture Transmission (HRPT) data. However, if the data are recorded onboard the satellite and then transmitted, they are referred to as Local Area Coverage (LAC) data. Only ten minutes of LAC data can be recorded per orbit. The third type of data is recorded at a reduced resolution and then transmitted, and referred to as Global Area Coverage (GAC) data. GAC data includes a full 102 minute orbit plus 6 minutes of overlap with the next orbit. The LAC and GAC data are received at Wallops Island, Virginia or Gilmore Creek, Alaska and relayed to the World Weather Building in Suitland, Maryland, via a DOMSAT link, for permanent archiving. The local acquisition of HRPT data at Wallops Island and Gilmore Creek are also relayed to Suitland via the DOMSAT link. The relayed data is also received via a DOMSAT link established at EDC.
The AVHRR sensor scans six lines of data per second. Each line of data contains: satellite information (time code for each scan, ...), telemetry, internal target data, and space data (for calibration of the thermal bands), TIP data (ignored in ADAPS), and image data. The AVHRR data has 2048 ten bit pixels for each of the five bands of spectral data. The pixels for each band are arranged in a band interleaved by pixel (BIP) format. Bands 1 and 2 are in the visible spectrum and bands 3, 4, and 5 are in the near infrared (thermal) spectrum.
For a detailed description of the AVHRR data and TIROS-N series satellite instrumentation, refer to the NOAA Technical Memorandum NESS 95, NOAA Technical Memorandum NESS 107, and NOAA Polar Orbiter Data User's Guide.
The Navy Space Surveillance and North American Air Defense Command centers (NAADC) monitor the NOAA satellites and record their position and attribute information daily. The ADAPS model requires ephemeris elements within five days of the image acquisition to determine the satellite's position and attributes at a given time. The model propagates the satellite's position, defined in the ephemeris, to the location or time of interest within an AVHRR image.
Ephemeris from the Navy is called NAVSPASUR and ephemeris from NAADC is called TBUS. NAVSPASUR and TBUS contain slightly different information, but both can be used as input into the ADAPS system. The daily ephemeris elements are saved in a file (name based on the satellite ID) in the $ADAPSTABLES directory.
To geo-reference AVHRR data to the earth's surface, ADAPS models the position and attitude of the satellite and the scanning geometry of the AVHRR sensor. There are three main components of a satellite model:
The orbital model requires ephemeris (TBUS, NAVSPASUR, NASA 2-line) elements to determine the satellite's position and attributes at a given time. By modeling the satellite's orbit, the position of the satellite can be propagated to a time of interest within an AVHRR image.
The platform/sensor model calculates where the satellite is looking (bore sight) while scanning the earth and characterizes the non-linear scanning of the sensor mirror. Variations in the attitude of the spacecraft change the location that is actually being scanned. The actual location scanned is affected by the altitude, roll, pitch, and yaw of the platform. Although these variations change slowly, the magnitude over time can cause the true location (latitude/longitude) of a pixel in an AVHRR image to be from 4 to 11 pixels off.
Using the platform model without refinement to the attitude parameters is called systematic correction. Refinement to the attitude parameters using a "navigation" process is called precision correction. Refer to the Navigation User's Guide for more detail.
The earth model takes into account the shape of the earth (flat, spherical, elliptical, geoid), and the rotational velocity of the earth.
Two factors that affect the precision of a pixel location are:
The onboard satellite clock can drift up to one half of a second before it is corrected. This can cause a maximum of a three line error. A delta time adjustment can optionally be supplied to the model.
A second adjustment can be applied to correct for errors due to local terrain elevation. Given a source of elevation data, the elevation for a given latitude and longitude is used to adjust the pixel location for the variation from the earth model. For AVHRR data this correction is only significant at higher elevations.
The three components of the satellite model collectively comprise two types of satellite geometric models:
The forward model converts a line, sample, and an elevation to a latitude and longitude location within an image. The inverse model converts a latitude, longitude, and an elevation to a line and sample location within an image. Calculated line and sample locations that fall outside of the image are negative or larger than the number of lines and samples within the image.
Both the forward and inverse geometric models require ephemeris elements within five days of the image acquisition. Roll, pitch, yaw, altitude, time, and elevation parameters can be supplied to the model for precision correction. These correction parameters can be determined by "navigation" and a source of elevation data. Precision correction can obtain sub-pixel accuracy.
All AVHRR images received are ingested and reformatted into a standard EDC AVHRR archive format. More information on converting to EDC AVHRR archive format refer to the Data Ingest of the ADAPS Programmer's Guide. The reformatting to the EDC AVHRR archive format is done with the INGEST and INGEST1B functions.
During the reformatting process the TIP data is stripped from each line and optionally written to a file associated with the archive image. This data is not archived as part of the EDC AVHRR archive. The time stamp from each line is also checked to determine if there are any data gaps.
Data gaps are lines that are missing due to a transmission, frame synchronization, disk, or other problems during acquisition. Data gaps are detected by calculating the difference between the current and previous time-stamps and dividing the difference by a sixth of a second (AVHRR data is scanned at six lines per second). When the gap is greater than one, it is necessary to fill in the missing line(s) with valid time stamp(s) and zeroed lines to maintain the proper along-track perspective. These data gaps are also called dropped lines.
The output from the ingest process is an EDC archive header, image, and optional TIP file. These files are named and associated by a scene ID that is comprised of the type of data, satellite number, date, and start time of the image. The scene ID is used throughout the ADAPS system to track an image during processing. The following is a sample scene ID:
AH11072294214346 Where: A - AVHRR H - HRPT acquired data 11 - NOAA-11 satellite 07 - month of acquisition 22 - day of acquisition 94 - year of acquisition 21 - start hour of acquisition 43 - start minute of acquisition 46 - start second of acquisition
The EDC archive image file has an extension of ".arch" and the header and TIP files have the same root file name (scene ID) with extensions of ".ahdr" and ".tip", respectively.
Every AVHRR archive image consists of two files: A CEOS Inventory Exchange Format (IEF) header file and an AVHRR image file. The AVHRR archive images are archived on ANSI labeled tapes. See AVHRRSTACK User's Guide for details on the AVHRR archive tape format.
The AVHRR archive header file used by EDC is a CEOS_IEF data exchange file. The CEOS_IEF file has the following format:
The variable number of header records contains the following EDC specific information about the image:
For more information on the format and content of the Archive Header refer to the Archive Format section of the ADAPS Programmer's Guide. The archive header can be displayed in LAS with DSPAHDR.
The archive image file is similar to the "raw" data stream (HRPT minor frame) acquired from the satellite. Refer to the NOAA Technical Memorandum NESS 107 for a description of the HRPT minor frame. An archive record contains the first 103 words from the HRPT minor frame and the earth video data packed into 13,796 bytes. The first 103 (10 bit) words from the HRPT minor frame are packed into 140 bytes and the 10,240 (10 bit) words of earth video data are packed into 13,656 bytes. Every three words (30 bits) are packed into four bytes (32 bits), right justified. All unused bits are set to zero. The earth video data consists of five bands for each of the 2048 samples in a scan line. The 2048 samples are ordered in BIP format.
The TIP data from the HRPT minor frame is only extracted from newly acquired images on request and is not saved as part of the EDC AVHRR archive.
An archive image can be copied from tape with READARCH or reformatted into a LAS image with AVHRRIN.
Reference aids are used to assist in data quality assessment of an image, ordering image products, data tracking, and providing image location information. A quick-look print, browse image, microfiche print, and metadata information are reference aids generated by ADAPS as part of the ingest and archive process. All reference aids are generated from band two of daytime images or band four of nighttime images.
The quick-look product is a positive film product generated on a Harris facsimile film recorder. Quick-look products are used to assess cloud cover, line drops, and data quality on images needed for products required within 24 to 48 hours after acquisition. One band of the AVHRR image is enhanced and subsampled to fit onto a 3375 line by 2048 sample piece of film, then line-work from the WDBII is embedded into the image. Scene information such as the date, start time, direction of the image, upper left latitude/longitude, lower right latitude/longitude, etc. are rasterized into an annotation block and added to the top of the quick-look. The following is an example of a quick-look product:
To produce priority facsimiles a table located in $ADAPSTABLES called "priority.fax" contains a priority flag for each type of facsimile. The type of facsimile is based on the satellite, type of data (HRPT, LAC, or GAC), and the receiving station's CEOS ID. The priority is set in the table as a letter code from a to z, with a being highest. A priority of 0 indicates no facsimile is to be made.
A browse image is a band from the AVHRR image that is scaled from ten bits to eight bits, enhanced, and subsampled by four in the line direction and five in the sample direction. An average browse image is 1350 lines by 409 samples. Scene information similar to the quick-look product is written to the header record and the browse images are transferred to the Global Land Information System (GLIS). The browse image is matched with metadata on GLIS to assist customers in making image selections/orders.
Customers use GLIS to select images based on satellite, date, time, and area of interest. The browse images for the selected images are viewed on GLIS to determine which AVHRR images are to be ordered. The following is an example of a browse product viewed on GLIS:
A microfiche print is an alternate film product available to customers to aid in ordering/selecting AVHRR images. A microfiche contains 12 concatenated AVHRR images that have been subsampled to a 2185 line by 1024 sample image. Each AVHRR image is enhanced, subsampled, and embedded with line-work from the WDBII. Each image on the microfiche is assigned a unique microfiche number which is transferred as part of the metadata. Two 12-image microfiches are printed on a single film on the ColorFire film recorder. The generated film is duplicated and sets of microfiche are made available to customers. Three sets of microfiche are currently generated: North American HRPT, recorded LAC, and all other data received. The following is an example of a microfiche:
As the AVHRR data is received and archived, a mechanism is needed to track and locate images that have been processed. Metadata for each image is generated and transferred to a database which can be used to select and locate images. The metadata contains the type of data (HRPT, LAC, or GAC), satellite number (11 for NOAA-11), date, start time, end time, day/night flag, direction of the pass, equatorial crossing, starting orbit number, pass duration, number of lines, number of bands, bands present, archive tape location, receiving station code, microfiche number, and nine geo-location points. A portion of the metadata is also transferred to GLIS and matched with the browse image to assist customers in making selections/orders. For more information on the format and content of the metadata refer to the Metadata section of the ADAPS Programmer's Guide.
There are two types of products that are created directly from the EDC AVHRR archive. The first is a copy of selected images from the archive. See the Image Archive section for a description of the EDC AVHRR archive.
The second and more standard product is a Level-1b format. Level-1b is AVHRR data that have been quality controlled, assembled into discrete data sets, and to which earth location and calibration information have been appended (but not applied). Refer to the NOAA Polar Orbiter Data Users Guide for a detailed description of the Level-1b format.
There are two general formats for AVHRR Level-1b data that can be ordered, packed data format (most common) and unpacked data format. Packed data is three 10 bit pixels stored in four bytes. Two bits from each four bytes are unused. Unpacked data is one 10 bit pixel stored in two bytes (16 bits). Six bits from each two bytes are unused.
Both the archive copy and the Level-1b data can be written to 9-track, 3480, or 8mm tapes or to disk. Products created on disk are transferred via anonymous ftp. Refer to MKLEVEL1B and INGEST1B for more information on reading and writing the Level1b format.
AVHRR products generated with ADAPS and LAS applications can be radiometrically corrected, precision corrected, geo-registered to a specified projection, have associated solar zenith, satellite zenith and relative azimuth bands calculated, and atmospherically corrected for Rayleigh and ozone affects.
One of the standard products generated using ADAPS/LAS is a composited NDVI. An example flow of an NDVI composite generated from an EDC AVHRR arcive image is described in the Geo-registerd Products Flow section of the ADAPS Operations Overview.