MODAPS PCR 12-016
| PCR Number | 12-016 |
|---|---|
| Date | 2012-09-25 |
| Initiator | James Kuyper |
| Abstract | A comprehensive PCR [PGE01 v6.0.15] for all C6 changes to PGE01, but only for Terra. It is therefore essentially identical to PCR 12-003, which was for both Terra and Aqua, and was approved on 2012-03-20, but includes a few more recent changes not yet covered by any approved PCR. |
| Problem Statement | This is a comprehensive PCR for all C6 changes to PGE01, but only for Terra. It is therefore essentially identical to PCR 12-003, which was for both Terra and Aqua, and was approved on 2012-03-20. More recent changes not yet covered by any approved PCR, will be marked as “New” below.Bug numbers listed below are from the SDST Bugzilla system.1. Defect FixesBug 1675: Processing of the packets for a given granule halted if the last packet of a scan was defective.Bug 2199:The cumulated_gflags count for the INVALID_SENSOR_ANGLE flag was always 0, even if there were pixels where that flag was set.Bug 2426: Latitude and Longitude were set to -999.9 when the Geolocation algorithm failed.Bug 2937: The Height SDS usually contained the height of the surface above the ellipsoid, except when that height was between -1 and 1 meter (occurs mostly along shorelines), in which case it was set to the height of the surface above the geoid, a difference that could be dozens of meters.Bug 3559: The number of frames in a given view was determined by the frame count of last packet for that view.2. Feature Requests:2.1 Higher ResolutionBug 1954: The position calculated for a 1 km pixel represented only the exact center of a pixel as seen from the satellite, at the center of the time that the image data was collected.Bug 1968: MODIS geolocation files contained only 1 km geolocation.Bug 3446: The Land/SeaMask SDS has a resolution of only 1 km.DEM files: The Altitude and Land/Water mask values stored in our DEM files were old, and had only 30 arc-second resolution.2.2 Improved MetadataBug 1969: The MODIS geolocation algorithm includes an adjustment to the orientation of the mirror assembly relative to the spacecraft, for what are believed to be thermal effects on the mirror assembly. This adjustment is calculated from the orientation of the spacecraft relative to the Sun. The Geolocation product contained no information about those adjustments.Bug 2249: There was nothing in the Geolocation product indicating whether or not terrain correction was turned on.Bug 2471: The names of the input files used to create the ephemeris and attitude files (which are, themselves, inputs to the MODIS Geolocation) are written to the Geolocation products, to allow tracking of data sources. The names were extracted from the corresponding *.met files, and are frequently missing from those files because the original *.met files have been discarded and recreated.Bug 2472: MODIS Geolocation uses all ephemeris or attitude packets that are not marked as having fatally bad quality.Land/SeaMask: The Land/SeaMask SDS has a limited range of valid values (0-7).New: PGE01 products lacked Digital Object Identifier (DOI) metadata.2.3 MiscellaneousProduction Rules: When there are missing MODIS L0 data sets, or missing attitude files, it’s sometimes possible to run MODIS Geolocation to produce a product file containing only a small fraction of the normal amount of data, but C5 PGE01 did not do so.Packet Length: If a packet’s length was valid, but not consistent with the packet’s type, it was either truncated, or padded with 0’s, to the length appropriate for it’s type.New packet header validity checks: Packet header corruption can render the sequence_flag and source_ID_type_flag inconsistent with other packet header fields.Geoid Height: The geoid height has a resolution of one degree, and was extracted by nearest-neighbor “interpolation”. When we added altitude averaging, a 1 km pixel straddling a 1-degree line of latitude or longitude used different values for the geoid height in some of the pixels that get averaged. This effect was normally too small to be noticed. However, with Bug 2937 resolved, the open ocean now has Height values of exactly 0 meters above the geoid almost every where, except for many isolated pixels with a Height between -3 meters and +3 meters, clustered along the 1 degree lines of latitude and longitude.Geolocation Parameters LUT updates: When work started on the C6 code, time-dependent parameters in the LUT files covered only the time period ending 2009-04-15, and several other parameters had values that had not been adjusted since before launch. Current C5 LUT files extend up to 2011-07-26. |
| Description of Change | 1. Defect FixesBug 1675: Correct logic to continue processing even if the last packet of a given scan is defective. Improve packet consistency checks so that this issue comes up less often.Bug 2199: Correct code to match change made to high level design for collection 4 - delay accumulation of gflags until after the INVALID_SENSOR_ANGLE bit has been set.Bug 2426: Set Latitude and Longitude to -999.0 when the Geolocation algorithm fails.Bug 2937: Always set the Height SDS to the height above the geoid.Bug 3559: Set the number of frames equal to the maximum valid value read in for the frame count field. Improve packet consistency checks to reduce the frequency with which this gives a different number than the value in the last packet.2. Feature Requests2.1 Higher ResolutionBug 1954: Allow the band_number parameter to identify a 250 m or 500 m band. If it does, calculate the geolocation for the centers of the corresponding high resolution pixels, and then aggregate them up to 1 km using the same weighting factors used by the L1B program for aggregating high resolution image data. Set band_number to 3.Bug 1968: Add a new int8 SDS containing, for each high resolution pixel, the vertical offset between the actual height of that pixel and a bi-linear interpolation/extrapolation of the 1 km positions to the center of that pixel. This will cause a significant increase in the size of an uncompressed file, but since the offsets are usually zero, and almost always small, it compresses very efficiently. The files delivered by MODAPS are always compressed with hrepack.Bug 3446: Add an SDS that indicates whether, at a higher resolution, there would be any water in the area covered by a 1 km pixel.DEM Files: Get new DEM files containing high-resolution altitude information at 15 arc-second resolution from the USGS, and a new Land/Water mask at 15 arc-second resolution from the University of Maryland.2.2 Improved metadataBug 1969: Add an SDS containing the values of the thermal adjustments to the instrument orientation for every scan, and another containing the values of various temperatures extracted from the telemetry stream.Bug 2249: Add a file attribute that indicates whether or not terrain correction has been turned on.Bug 2471: Get the input file names for the ephemeris/attitude files directly from their headers.Bug 2472: Add an SDS containing the quality flags associated with the ephemeris and attitude data used for each frame of data.Land/SeaMask: Add a valid_range attribute to the Land/SeaMask SDS.New: Added DOI metadata, both as file attributes and as PSAs in the ECS metadata.2.3 MiscellaneousProduction rules: Allow PGE01 to run even if the L0 data set covering the current run time is not available, so long as the L0 data set covering the following time period is. Allow PGE01 to run even if no Attitude file is available to cover the current time period, so long as there is one available that covers an adjacent time period.Packet lengths: Discard packets whose packet length is inconsistent with the packet type, even if the length is a valid one.New packet header validity checks: Discard packets where the sequence_flag or source_ID_type_flag have values that are inconsistent with other packet header fields.Geoid Height: Interpolate the geoid height bilinearly.Geolocation Parameters LUT updates: Extend the range of the time-dependent parameters to 2013-01-01, and update the values of several other parameters. |
| Products Affected | L1A: MOD01. Geolocation: MOD03. |
| Software Affected | PGE01 v6.0.15 |
| Processing String to Receive the Change | Terra Forward, Terra Reprocessing |
| Downstream Product Effects | 1. Defect FixesBug 1675: There will be slightly fewer granules with missing scans at the end.Bug 2199: The cumulated_gflags file attribute will have an accurate count of the number of INVALID_SENSOR_ANGLE flags set.Bug 2426: Latitude and Longitudes SDSs will have -999.0 rather than -999.9 when the Geolocation algorithms fails.Bug 2937: The Height SDS will always contain the height above the geoid.Bug 3559: The number of frames for a given scan will always be large enough to include all frames of data actually collected during that scan.2. Feature Requests2.1 Higher ResolutionBug 1954: All geolocation values will be shifted by small amounts.Bug 1968: Only PGEs written to use the new offset SDSs will be affected.Bug 3446: Only PGES written to make use of new WaterPresent SDS will be affected by its presence. The Land/SeaMask will be shifted somewhat.DEM Files: All geolocation data will be shifted somewhat, especially in those locations where the updated DEM and Land/Water mask data has been changed the most.2.2 Improved metadataBug 1969: Only PGEs written to make use of the new thermal adjustment SDS will be affected.Bug 2249: Only PGEs written to read the new terrain correction file attribute will be affected.Bug 2471: The input file names for the ephemeris/attitude files will more frequently be meaningful.Bug 2472: Only PGEs written to read the new quality flags SDS will be affected.Land/SeaMask: Only PGEs written to read the new valid_range attribute on the Land/SeaMask SDS will be affected.New: Only PGEs written to read the new DOI metadata will be affected.2.3 MiscellaneousProduction Rules: Downstream PGEs capable of processing incomplete L1 products will have slightly more usable data to process.Packet Lengths: More packets with corrupt headers will be discarded.New packet header validity checks: More packets with corrupt headers will be discarded.Geoid Height: The Height SDS will contain fewer spurious little hills and pits clustered along 1 degree lines.Geolocation Parameters LUT updates: All geolocation data will be shifted slightly |
| Data Granules to be Used for Science Testing | 2006-209/224 |
| Justification | 1.Defect FixesBug 1675: Recovery of packets from scans after the one that ended with a defective packet.Bug 2199: Improves accuracy of cumulated_gflags.Bug 2426: The File Specification and the value of the _FillValue SDS attribute both say -999.0, and have done so for many years. When fill values are the result of missing data rather than a failure of the geolocation algorithm, they’re already set to -999.0.Bug 2937: The file specification has specified that the Height SDS contains the height above the geoid since 1998-02-10.Bug 3559: The last packet of a view may contain a corrupted frame count value, which we cannot detect. Also, if a packet contains an undetected corrupt time stamp, it will often be processed as if it were the last packet of a view for the wrong scan. In either case, it will almost always have a frame count smaller than the correct number of frames for that view.2. Feature Requests2.1 Higher resolutionBug 1954: If surface of the earth is significantly curved in a given pixel, the weighted average position will more accurately describe the entire pixel than the position of the exact center.Bug 1968: The new offset SDSs allow exact reconstruction of the high-resolution positions that were aggregated to give the 1 km positions.Bug 3446: The Cloud Mask algorithm needs this information to produce accurate results along shorelines.DEM Files: C6 MODIS geolocation will performs its calculations at 500 m resolution, which for best results requires DEM files with a resolution of 15 arc-seconds or better.2.2 Improved MetadataBug 1969: C6 PGE60 will be changed to copy the thermal adjustment information to the control point matching files, for use in calibration of the Geolocation algorithm.Bug 2249: Official MODIS Geolocation products have never been created with terrain correction turned off, and probably never will be. However, the MODIS geolocation software is publicly available, and may be run by an outsider with terrain correction turned off. This change will make it easier to identify such products.Bug 2471: Improves traceability of the sources of ephemeris and attitude information used by MODIS Geolocation.Bug 2472: Allows geolocation data that has poor quality, but not so poor as to be fatally bad, to be distinguished from data with higher quality.Land/SeaMask: Improves self-documentation of MODIS geolocation files.New: NASA has informed us that all C6 MODIS products must have DOI metadata.2.3 MiscellaneousProduction Rules: The last scan of a given time period starts in one L0 data set, but many of the packets for that scan are often contained in the following data set. The SDP Toolkit routines allow ephemeris and attitude data to be extrapolated up to 60 seconds beyond the actual coverage of those files. For both of these reasons, geolocation data can sometimes be generated for a given time period, even if a corresponding input file is unavailable. Unfortunately, orbital metadata cannot be extrapolated before the time stamp of the first packet of ephemeris data, so ephemeris files covering the current period and the previous period (which might be the same file) are both still mandatory.Packet Lengths: If a packet’s original length was corrupted to be too small, padding it with zeros to the correct length produces meaningless data. If the packet type had been corrupted, rather than the packet length, changing the length to match is a useless “fix”.New packet header validity checks: This is an easy thing to check, and improves data quality.Geoid Height: This change will greatly reduce (but will not eliminate) spurious little “hills” and “pits” clustered along the 1 degree lines, and limits their heights to ± 1 meter.Geolocation Parameters LUT updates: Analysis suggests that these changes should improve geolocation accuracy. |
| Effective Date for Implementation of Change | 2012-06-28 |
| Status | Jack Xiong approved for Level 1B, Robert Wolfe for Geolocation, Steve Platnick approved for Atmospheres, Steve Running approved for Land, Bryan Franz approved for Oceans and Michael King approved on 11/25/2012. |
