Design Document for SPOTHDF2LAS
SPOTHDF2LAS
AUTHOR: Deb Hames
Functional Summary:
Reads SPOT Vegetation data (in HDF format) from disk and converts them to a
LAS image.
Comments:
HDF files are stored on CD's in zipped format. It is required that the user
copy and unzip the files to disk.
HDF files included but not used for this module are:
- The Copyright Information File
- The Physical Volume File
- The Quick Look Plane TIFF file
Background:
This application is specific to HDF format containing SPOT Vegetation data.
It does not support any other type of data in HDF format.
Some background information about HDF:
The Hierarchical Data Format, or HDF, is a multi-object file format used for
sharing scientific data in a distributed environment.
HDF can be viewed at several interactive levels. At its lowest level, HDF is a
physical file format for storing scientific data. At its highest level, HDF is
a collection of utilities and applications for manipulating, viewing, and
analyzing data stored in HDF files. (See HDF Command Line Utilities). Between
these levels, HDF is a software library that provides high-level and low-level
programming interfaces. It also includes supporting software that make it easy
to store, retrieve, visualize, analyze, and manage data in HDF files.
HDF Application Programming Interfaces, or API's, include several independent
sets of routines, with each set specifically designed to simplify the process
of storing and accessing one type of data. The following are the new multifile
HDF interfaces which serve as the tools used to read and write HDF files.
SD API Scientific Data Sets. Stores, manages and retrieves
multidimensional arrays of character or numeric data, along
with their dimensions and attributes, in more that one file.
VS API Vdatas. Stores, manages and retrieves multivariate data stored
as records in a table.
V API Vgroups. Creates groups of any primary HDF data structures
GR API General Raster Images. Stores, manages and retrieves raster
images, their dimensions and palettes in more than one file. It
can also manipulate unattached palettes in more than one file.
AN API Annotations. Stores, manages and retrieves text used to
describe a file or any of the data structures contained in the
file. This interface can operate on several files at once.
Note:
There are old interfaces that are still supported but not recommended to be
used. They are DFR8 API, DF24 API, DFP API, DFAN API and DFSD API. These old
interfaces are currently supported for backward compatibility only. The new
multifile interfaces provide simultaneous access to several HDF files from
within the application, which is not supported in the old single-file
interfaces.
HDF Command Line Utilities:
--------------------------
The HDF command line utilities are application programs that can be executed by
entering them at the command level. These utilities can:
- analyze and view HDF files
- convert from one format to another
- manipulate HDF files
The 'hdp' utility is probably the most useful utility. It provides quick
information about the contents and data objects in the HDF file.
See Chapter 15 of HDF Users Guide for 'hdp' as well as additional command line
utilities and their descriptions.
The above information was taken from the HDF (v4.lr3) Users Guide located at:
http://hdf.ncsa.uiuc.edu/doc.html
Some background on SPOT Vegetation HDF data and products:
The data is included in each SPOT HDF file as a 'Scientific Data Set (SDS)'
Therefore the SD API will be used.
SPOT Vegetation product format is composed of a mix of ASCII files, HDF files
and one TIFF (Tagged Image File Format) file. The TIFF file is only used as a
browse image. See Comments for files not used for this module and see
Requirements for files used in this module.
There is only one band per HDF image file.
There are two levels of enhanced products:
I. Primary Products or P Products
- zone is extracted from a single data strip
- The resolution for this product is 1-km.
II. Synthesis or S Products
- S1: Zone extracted from daily global synthesis of all data strips
acquired in a 24-hour period.
- The resolution for this product is 1-km
- S10: Zone extracted from ten-day global synthesis compiled from daily
synthesis over the previous ten days.
- S10.4 and S10.8: Derived from S10.
- The resolutions for these products are 4-km (S10.4) and
8-km for (S10.8)
The following set of projections are available for both P and S products:
- World cover:
Plate Carree
Goode Homolosine
Equatorial Mercator
Equivalent Sinusoidal
- Continental projections:
North American: Albers
Africa: Mercator or Plate Carree
Western Europe: Conic, Lambert
Asia, South America, Australia (TBD)
- Arctic, Antarctic:
Polar Stereo
The HDF files will contain the following data types for each of these files:
- 16 bits per pixel
- Spectral bands: B0, B2, B3 and MIR
- 8 bits per pixel
- NDVI, Status Map, Viewing Zenith Angle, Solar Zenith Angle,
Viewing Azimuth Angle, Synthesis Time Grid, Water Vapor Grid,
Ozone Grid, Aerosol Grid
- 32 bits per pixel
- 1B HRVIR Longitude Plane and 1B HRVIR Latitude Plane.
The following information is stored as Scientific Data Sets and will be
retrieved from the HDF image files using the SD API:
- dimensions (number of lines and number of samples)
- the number of bytes per pixel (data type)
- the image data
The following information is stored as Vdata and will be retrieved from the HDF
image files using the VS API:
- The band numbers and image descriptions. (ie b0, ndvi, status map...)
Each volume of the HDF files contain a LOG file or Logical volume description
file. This file contains the map projection information, geodetic system
parameters, projection parameters and geographic location. This is used to
update the DDR.
The above information was taken from The SPOT Image Home Page
The Vegetation User Guide:
http://www.spotimage.fr/data/images/vege/vegetat/book_1/e_frame.htm
Additional information on P and S Products:
http://www.spotimage.fr/home/proser/listvege/welcome.htm
Requirements:
Main Requirement:
- Convert files in SPOT Vegetation HDF format to LAS format.
Input Formats:
- Files are to be read from disk only and they must be unzipped.
- Only lower case file names can be supported due to LAS limitations
- Support multiple input files (one filename per HDF image)
- The maximum files on one CD is 16. Use 20 for the maximum
number of input files.
- Each HDF image file contains only one band.
- Support input images containing BYTE, I*2, I*4, and R*4 data.
- Support single resolution BSQ products
- Support the following image files:
- SPECTRAL BAND B# (band numbers are 0, 2 and 3)
- SPECTRAL BAND MIR
- NDVI
- STATUS MAP
- VIEWING ZENITH ANGLE
- SOLAR ZENITH ANGLE
- VIEWING AZIMUTH ANGLE
- SOLAR AZIMUTH ANGLE
- SYNTHESIS TIME GRID
- WATER VAPOR GRID
- OZONE GRID
- AEROSOL GRID
- 1B HRVIR LATITUDE PLANE
- 1B HRVIR LONGITUDE PLANE
(These file names are stored in the image file as Vdata instead of
Scientific Data Sets. They will be retrieved using VS API instead
of SD API. This information will then be used in the LAS DDR file.)
- The map projection, geodetic, projection parameters and geographic
information will be extracted from the ASCII logical volume
descriptor or LOG file.
Output Files:
- LAS image files with the associated DDR file.
- DDR file contains all required information (map projection, geodetic,
projection parameters and geographic)
- Output file names should be all lower case.
- Support multiple output LAS images. (One output LAS image for one
input HDF file.)
- The maximum number of outfiles is 20.
- Support single output LAS images. (All images in one output file.)
- A user cannot specify an 8 bit HDF file and a 16 bit HDF file
and expect a single output LAS file. If this happens
SPOTHDF2LAS will terminate with a fatal error.
DDR Information:
- If either the number of lines, number of samples or resolution do not
exist, then SPOTHDF2LAS will terminate with a fatal error.
- Update the DDR with the information provided by the log file. Use the
Projection Transformation Package Projection Parameters as a guide
for updating the DDR according to the Projection. If a field is
missing from the log file that is required for that projection in
LAS, then SPOTHDF2LAS will continue processing but will issue a
a non fatal error, warning to the user of the missing data.
PARAMETERS:
- INFILE
- Input HDF file(s). The host file and pathname of the HDF single band
Spot HDF filename(s). Windowing is not supported. If multiple HDF
files are requested in INFILE, and if the user requests one output
LAS image, the HDF files must be the same data type.
- OUT
- Output image(s). The output LAS image. The number of images
specified in OUT must either be one or equal to the number of input
file names specified in INFILE.
- LOGFILE
- Input LOG file. The file that contains the information needed to
update the DDR.
Scope/Limitations:
- Only SPOT Vegetation data is supported.
- Only lower case file names are supported.
- Windowing is not supported.
Overall design:
Flow diagram:
SPOTHDF2LAS
|
|----------getpar
|
|----------gethdfinfo
| |
| |-----------SDstart
| |-----------SDselect
| |-----------SDgetinfo
| |-----------SDendaccess
| |-----------SDend
| |-----------Hopen
| |-----------Vstart
| |-----------VSgetid
| |-----------VSattach
| |-----------VSinquire
| |-----------VSdetach
| |-----------Vend
| |-----------Hclose
|
|----------process_hdf
| |
| |-----------c_eopenr
| |-----------c_inittm
| |-----------c_startm
| |-----------getimage
| | |
| | |----------SDstart
| | |----------SDselect
| | |----------SDreaddata
| | |----------SDendaccess
| | |----------SDend
| |
| |----------c_ewrite
| |----------c_eclose
| |----------c_stoptm
|
|----------c_hstout
|
|----------c_complt
Algorithm:
Initialize TAE interface
Get the TAE global msgtim
Retrieve input Spot HDF file name(s)
Retrieve output image name(s)
Retrieve logfile
Begin Loop (loop on each input file)
Open input file for reading for SD interface
Initialize the SD interface
Get SD selected information (rank, dimensions and data type)
Copy dimensions to nl and ns
Copy data type to idtype
Terminate access to SD interface
Close HDF files
End Loop
If the number of LAS files equals 1
Then check all HDF input data types
If they are not the same
Then fatal error
Copy the HDF data type(s) to a LAS data type
Begin Loop (loop on each input file)
Open input file for reading - using VS interface
Initialize the VS interface
Get VS selected information (description of input file)
Copy the vdata name into an array
Terminate access to VS interface
Close the HDF files
End Loop
If number of LAS files equal one
(Process all the HDF files at once)
Allocate input hdf buffer
Copy the HDF input data type to the LAS output data type
Open the output image file for write access
Initialize the processing message
Initialize the SD interface
Open the input file(s) for reading
Read HDF image data into hdfbuf
Terminate access to SD interface
Close the HDF file
Free hdfbuf
Update the DDR using information from logfile
Close the output LAS file
Else
Begin Loop (Process the HDF files - one at a time)
Allocate input hdf buffer
Copy the HDF input data type to the LAS output data type
Open the output image file for write access
Initialize the processing message
Initialize the SD interface
Open the input file for reading
Read HDF image data into hdfbuf
Terminate access to SD interface
Close the HDF file
Free hdfbuf
Update the DDR using information from logfile
Close the output LAS file
End Loop
Create history record(s)
Output completion message
Test Cases:
- Test using one HDF image file and one output LAS image.
- The output of the LAS image is successful.
- Test using one 8 bit HDF file and one 16 bit HDF file.
- Test with two LAS outfiles. The output of both images is succesful.
- Test with one LAS outfile. SPOTHDF2LAS will fail. The data types
must be the same if one LAS output image is requested.
- Test using one HDF image file, one output LAS image, and using a logfile that
is missing a datum.
- The output of the LAS image is successful. A warning message will be
displayed, noting the missing datum. The DDR will be updated with all
information, except for the datum which defaults to CLARK 1866. The
datum in the DDR will be set to invalid.
- Test using one HDF image file, one output LAS image, and using a logfile that
is missing a projection.
- The output of the LAS image is successful. A warning message will be
displayed, noting the missing projection. The DDR will be updated
with all information, except for the projection and the projection
parameters. The projection in the DDR will be set to invalid.