ARA - Atmospheric Radiation Analysis

Improved Initialization Inversion system (3I)

Introduction : The Improved Initialization Inversion system (3I)

Spaceborne radiometers observe spectral radiances that are emitted or backscattered by the atmosphere and the surface into the direction of the satellite. For the derivation of atmospheric and surface geophysical variables from the radiance spectrum, retrieval algorithms are required that comprise all steps needed to translate instrument data into the final products. These steps include the simulation of observed radiances and brightness temperatures with forward radiative transfer models, the ingestion of auxiliary databases, the inversion process to obtain geophysical products, and the generation of gridded products.
The 3I method was developed at LMD for this purpose and has been extensively discussed in the literature. For its complete description, the reader is referred to Chédin and Scott (1984, 1985), Chédin et al. (1985, 1989, 1994), and Chédin (1988). An updated overview of the method used for reanalyzing of the TOVS observations for the TOVS-Pathfinder Path-B dataset is given here ( Scott et al. (1999)).
The 3I inversion algorithm is a direct, non-iterative, physical statistical method. It uses data from the HIRS (infrared) and MSU (microwave) radiometers.

Description of 3I

• Calibration and collocation
  Starting from the level 1B data, HIRS and MSU observations are calibrated using coefficients provided by NOAA following the procedures set forth in the NOAA Polar Orbiter User s Guides.
  The spatial resolution of 3I retrievals is a compromise between the spatial resolutions of the HIRS and MSU sounders. A 3 x 2 (at the edges of an orbit), or 3 x 3, or 3 x 4 (at nadir) array of HIRS spots is grouped together and collocated with the nearest MSU spot(s). Such boxes represent a surface of nearly 100 x 100 km2, and retrievals are performed for each array. The same kind of mapping is applied to the topography dataset, which describes the terrain elevation and the percentage of water covering the surface.
• Cloud detection
  The 3I inversion scheme requires information about the presence of clouds within the field-of-view. Owing to their relative insensitivity to clouds, microwave channels play a major role in their detection. The box is identified as cloud-free, partially cloudy, or overcast depending on the outcome of a series of seven (night)/eight (day) threshold and coherence tests based on the work by Wahiche et al. (1986), and improved by Stubenrauch et al. (1999a) and Stubenrauch et al. (2006), applied to each HIRS spot in the box. Additional tests aim at unambiguously distinguishing between clouds and special surface conditions (e.g., warm surfaces, snow, sea ice, etc.). Cloud identification in the polar regions, where clouds are sometimes warmer than the surface, has been improved with respect to the original version (Francis 1994). Clouds over snow and sea are identified by a difference between the 3.7- and 11- micron brightness temperatures. The cloud detection is performed differently over sea ice, water, and land.

Fig. 1. The 3I processing flow chart.

• Inversion
  Provided the observed brightness temperatures correspond to clear areas or have been properly cleared, the 3I procedure follows two principal steps (see Fig. 1).
  1. Retrieval of the initial guess solution: the observed clear column radiances are first used to retrieve the best initial guess solution. The procedure makes uses of the Thermodynamic Initial Guess Retrieval ( TIGR) dataset. The selected set of observed radiances (or equivalent brightness temperatures) and corresponding a priori information on the situation observed are compared with each equivalent set archived in TIGR and the closest is retained.
  2. The basis for the retrieval of the exact solution is a maximum probability estimation procedure aimed at minimizing the differences between the brightness temperatures associated with the initial guess and the observed ones. Use is made of the Jacobian associated with the retrieved initial guess in the ( TIGR) dataset.
• The Thermodynamic Initial Guess Retrieval ( TIGR) dataset
  This (frozen) library of atmospheres, the TIGR dataset, consists of about 1800 situations (recently extended to about 2300 situations for a better representation of the tropical regions; see below) selected by statistical methods out of 80 000 radiosonde reports. Clear sky transmittances, radiances, and weighting functions for all TOVS sounding channels are precomputed for each situation in TIGR by the Automatized Atmospheric Absorption Atlas ( Automatized Atmospheric Absorption Atlas (4A)) fast line-byline model of Scott and Chédin (1981). Calculations are performed for 10 viewing angles between 0° (nadir) and 60° (the maximum value for angular scanning), for 19 values of surface pressure (up to about 500 hPa for elevated terrain), and for two surface types: land and sea. These results are also stored within the TIGR dataset. It is worth pointing out that TIGR is not sensitive to the relative quality of the radio soundings sampled in it but only to their representativeness and plausibility. In fact, it is sensitive to the quality of the relationship between thermodynamic quantities and radiative quantities. For that reason, great attention has been paid to the validation of the 4A model.
  The situations in TIGR have been stratified by a hierarchical ascending classification into five airmass classes, depending on their virtual temperature profiles (Achard (1991); Chédin et al. (1994)). Given an observation, that is, the HIRS-2 and MSU brightness temperatures, a distance between the observation and the gravity center of each class in the TIGR dataset, using a set of four TOVS channels, is calculated in order to assign one of these airmass types (hereafter called tropical, midlatitude 1 and 2, and polar 1 and 2) to this observation (see also TIGR...).
• Selection of the first guess solution of the inversion process
  Identification of the closest situation is performed through a pattern recognition approach using the observed brightness temperatures to describe the state of the underlying atmosphere. In case of cloudy situations, the initial guess is obtained after a two-step procedure, one of which is the cloud clearing process ( Chédin and Scott (1984); Chédin et al. (1985)). Clear cases involve only step 2.
  
  1. CLOUD CLEARING OF TEMPERATURE-SENSITIVE CHANNELS
     The earth is approximately 60% cloud covered on average, thus a method is needed to remove the effects of clouds on the radiances. Because the quality of the retrieved products is fundamentally determined by the quality of the cloud clearing, this step is of paramount importance. The cloud clearing algorithm is an integral part of the whole retrieval system as it takes advantage of coincident knowledge of other parameters. To derive the brightness temperatures that would have been observed under clear sky conditions, 3I relies upon the coupling between MSU and HIRS channels through the so-called Psi method (Chédin et al. (1985)). Interchannel regression techniques are applied based upon the TIGR dataset where the infrared channels are the predictands and the microwave and noncontaminated infrared channels are the predictors.
     See (Chaboureau (1998)) for more details.
  2. SELECTION OF THE FIRST GUESS
     The proximity recognition for the first guess search is then performed by comparing the observed or cloud cleared radiances with the calculated radiances corresponding to each archived atmospheric situation belonging to the previously determined airmass type. The mean of the closest situations is taken as the initial solution.
• Temperature profile retrieval: A Bayesian approach
  The principal difficulty in inverting the radiative transfer equation for retrieving geophysical parameters (i.e., minimizing the differences between the observation and the initial solution) is related to the fact that standard mathematical approaches do not yield unique and/or stable solutions. Colinearities among the variables, inherent to the physical aspect of this problem, render traditional estimates (e.g., ordinary least squares) less accurate and less useful than usually expected. Adding prior information to the data directly leads to Bayesian statistics, indicating that estimates should be modified according to knowledge that may be available prior to the gathering of the data. In our case, this a priori knowledge is extracted from the TIGR dataset through differential temperature covariance matrices, one for each condition of observation: surface pressure, viewing angle, land/sea flag, clear/ cloudy flag, airmass type, etc... (Chédin et al. (1985)). These matrices are computed by applying the proximity recognition algorithm to all the TIGR subsets, assuming they are of either clear or cloudy situations
• Cloud parameters (top pressure, effective amount, and type)
  After the estimation of the temperature profile, cloud properties are determined from the radiances averaged over all pixels declared cloudy within each 100 km x 100 km box, assuming that they are covered by a single, homogeneous cloud layer. The average cloud-top pressure and the effective cloud amount are obtained by a weighted-chi2 method using four 15- micron CO2-band radiances (HIRS channels 4 to 7) and the 11- micron atmospheric window radiance. The weights are channel- and cloud-level dependent (Stubenrauch et al. (1999b)). The empirical weights reflect the usefulness of a spectral channel at a cloud level for the determination of the effective cloud amount. This new method is much less sensitive to errors in the temperature profile than the original 3I method (see Wahiche et al. (1986)), which, like other currently used methods, involved a denominator getting near zero under certain conditions (low clouds). The cloud-top pressure is transformed into cloud-top temperature using the 3Iretrieved atmospheric temperature profiles. A cloud cover fraction is also determined as the fraction of cloudy HIRS pixels in each grid box.
  The 3I cloud parameters have been carefully evaluated on a global scale (Stubenrauch et al. (1999a, 1999c)) by comparison with time space collocated, recently reprocessed ISCCP cloud parameters (Rossow et al. (1996)), which are being intensively checked by many ongoing studies. These comparisons reveal considerable improvement in the 3I cloud parameters. The remaining disagreements with ISCCP can be explained by grid heterogeneities (vertical and horizontal) or by differences in cloud detection (Stubenrauch et al. (1999a, 1999c)).
• Cloud clearing of moisture temperature sensitive channels
  The cloud characteristics are used to cloud clear infrared channels (i.e., HIRS channels 7, 8, 10, 11, 12, 13, 18, 19) that are sensitive to both moisture and temperature. Use is made of the well-known formula (Smith 1967) that expresses the observed radiance as a function of the equivalent clear radiance (same situation in the absence of cloud) and the cloudy radiance (full coverage of a black cloud at the correct top pressure). This is done only when the effective cloud amount is less than 60%.
• Surface temperature
  Radiances in spectral windows carry information about the temperature of the earth s surface. For the derivation of surface temperature, contribution to the observed radiances from water vapor, surface emissivity, clouds, etc., have to be accounted for. In the medium infrared, the surface emissivity has a value close to 1, slightly smaller over land than over sea. It is somewhat smaller in the near infrared (3.7- micron windows), particularly over land, and it is affected by the reststrahlen (decrease of the emissivity due to SO2 radiative properties) effect over bare soils and deserts near 8 micron (channel 10).
  The surface temperature retrieval algorithm depends on the clear/cloudy flag. For clear or partially clear boxes, the surface temperature is obtained through regressions whose coefficients are obtained from TIGR (one set of coefficients per observing condition). Shortwave window channels are not used during the day. Channel 10 is not used over desert areas [detected by a threshold test on the difference between the brightness temperatures of channels 8 (near 11 micron), and 10]. For cloudy boxes, a ridge-type (regularized least squares) estimator is used based on the same cloud cleared channels. No attempt is made when the effective cloud amount is larger than 60%. Improvements made to the algorithm for icecovered, very cold surfaces (Francis (1994)) were recently incorporated. Although not the best instrument for retrieving surface temperature, the advantage of TOVS is that fields are coherent with the other variables describing the atmospheric state (temperature and moisture profiles, clouds, etc.). Moreover, the accuracy of 3I sea surface temperature is satisfactory (see below).
• Microwave emissivity and sea ice detection
  The microwave emissivity is derived from the MSU channel 1. Emissivity is particularly useful for identifying sea ice in a field of view. Sea ice is detected by a high microwave surface emissivity (> 76%) associated with an 11- micron brightness temperature less than 268 K. Open sea is defined by a microwave surface emissivity less than 65%.
• Moisture profile retrieval: A nonlinear neural network estimate
  It is well known that the main difficulties in retrieving water vapor from TOVS arise from the coarse vertical resolution of the water vapor sensitive channels (mainly channels 8, 10, 11, 12), limited information near the surface, contamination by surface conditions, and the removal of cloud effects. Moreover, the inversion process is highly nonlinear.
  Over the last decade, neural networks have proven their ability to handle nonlinear problems and have increasingly been used in forward or inverse radiative transfer problems related to satellite-borne observations (Escobar-Munoz et al. (1993); Cheruy et al. (1996); Rieu et al. (1996)).
  The original 3I algorithm used a ridge-type linearized estimation method (Chédin et al. (1985)). Recently, a new algorithm has been derived that consists of a multilayered perceptron (Rumelhart et al. (1986)). To retrieve the vertical distribution of humidity, brightness temperatures of the four channels most sensitive to water vapor absorption (HIRS 8, 10, 11, and 12), are combined with six channels at 15 micron (from HIRS 2 to 7) to introduce information about the vertical structure of temperature. Outputs from the network are the precipitable water contents for five standard layers (100 300, 300 500, 500-700, 700 850, and 850-hPa surfaces). Outputs are then transformed into water vapor contents above the surface, 850, 700, 500, and 300 hPa. The surface pressure can take on three different values corresponding to the lower levels of the vertical discretization of the radiative transfer model 4A (1013, 955, and 900 hPa). After various tests, we adopted an architecture with one hidden layer (with 15 neurons). No retrieval is attempted in cases where surface pressure is lower than 850 hPa. The training of the neural network (Chaboureau et al. (1998a)) is performed using the TIGR dataset whose tropical class has been extended to about 900 situations, emphasizing more humid atmospheres and a better Gaussian behavior as shown in Fig. 2 (Chevallier et al. (1998)). Moreover, to take into account existing discontinuities between the surface skin temperature and the surface air temperature, the skin temperature associated with each extended TIGR (TIGR-3) situation is obtained by adding to the surface air temperature a Gaussian noise (random drawing) of standard deviation varying with the situation (airmass type; land/sea flag) from 1.7 to 3 K. Discontinuities are limited to three times the standard deviation.
  
  Fig. 2. Histograms (40 classes) of the water vapor content of the extended tropical class of TIGR (TIGR-3) (top) and in the preceding TIGR-2 tropical class (bottom).

  The entire 3I algorithm is illustrated by a flow diagram in Fig. 1. As seen in this figure, the TIGR dataset plays a key role at many steps of the algorithm. For a weakly nonlinear problem, such as temperature inversion, it is the source of the initial guess solution (through pattern recognition) and of the a priori information (covariances matrices) involved in the Bayesian temperature estimate. For stronger nonlinearities, like water vapor inversion, it is used for training the neural networks from which proceeds the nonlinear water vapor estimate. Transformations from the thermodynamic world (pressure, temperature, water vapor, etc.) to the radiative world may be treated the same way. TIGR appears as a sort of professor dataset or an atmospheric radiation interpreter.

• Vertical atmospheric longwave radiative budget
  The various geophysical variables retrieved from 3I contain most of the information needed to derive the vertical distribution of the atmospheric longwave radiative fluxes. A new generation of radiative transfer models, based on the neural network technique, has been designed with the purpose of computing, as rapidly and accurately as possible, the vertical (upward, downward, net) longwave radiative budget from the top of the atmosphere to the surface, for clear as well as for cloudy situations. Flux profiles may be computed starting either from the thermodynamic description of the situation, or from the observed TOVS radiances. The neural networks are trained using large ensembles of flux profiles computed either with a classic wide band model (Morcrette (1991); Zhong and Haigh (1995)), or from a line-by-line model 4A (Scott and Chédin (1981)). The dramatic saving of computing time offered by these models (more than 10 times faster than a classic wide band model; 106 times faster than a line-by-line model) allows for an improved estimation of the longwave radiative properties of the atmosphere in general circulation model (GCM) simulations. Details are given in Chevallier et al. (1998), where the validation of these new models is discussed in detail. Flux profiles computed from the TOVS Path- B approach are available as experimental products upon request from the authors.

Monitoring of observational and computational biases

Like most physical retrieval methods, the 3I method estimates geophysical variables by minimizing the differences between a set of observed and computed brightness temperatures. As a consequence, systematic biases between simulated and observed brightness temperatures can be problematic, not only for the retrieval accuracy, but also for further analyses of the climate variability and evolution. As these biases may differ from satellite to satellite, spurious trends may result. Removal of biases, due to either inter satellite changes in the spectral intervals of the channels or to instrumental drifts and individual channel evolution over the lifetime of a given satellite, requires developing an automatic correction scheme to infer and regularly update these adjustments. At LMD, we use collocated satellite-radiosonde datasets from NOAA/NESDIS: the so-called DSD5 files (Uddstrom and McMillin (1993)). Latitude, longitude, time, and measurement of the radiosonde are first extracted from the DSD5 archive. Brightness temperature observations are then extracted from the level 1B Pathfinder archive and collocated with the radiosondes (window: 100 km x 3 h). Radiosonde reports are screened for quality and the number of reported levels below 30 hPa. They are then used as inputs to the forward model, which simulates brightness temperatures for all the infrared and microwave HIRS and MSU channels. Simulated brightness temperatures are compared with observed values. Monthly averaged empirical adjustments are computed and stored separately for clear sky, over land, over sea, and for three airmass classes (tropical, midlatitude, polar). This correction procedure allows biases due to the radiative transfer model, to the instrument, or to unexpected events (e.g., the eruption of the Pinatubo volcano) to be taken into account and eliminated quite accurately.
For the new, recently launched, instruments as the Advanced Infrared Radiation Sounder (AIRS) on board the Aqua/NASA platform, a modified procedure has been developed which substitutes the European Centre for Medium-Range Weather Forecasts (ECMWF) ERA-40 reanalysis files to the NESDIS DSD5 radiosonde files.

Collocation statistics availability :

• Statistics computed with the DSD5
  • NOAA10 : July 1987 to september 1991
  • NOAA11 : July 1989 to december 1994
  • NOAA12 : July 1991 to september 1995
• Statistics computed with the ERA40
  • Not available at this time

Ask for a 3I algorithm

The 3I Algorithm is available as a freeware product but only for academic use and to those wishing to use the code for scientific research.
For a purpose other than a research or academic use please, contact-us :

Dr. Noelle Scott
Laboratoire de Météorologie Dynamique
Analyse du Rayonnement Atmosphérique
Ecole Polytechnique
Route départementale 36
91128 PALAISEAU CEDEX
FRANCE