EOS-Aura/OMI NO2 slant column retrieval: stability & uncertainties
TROPOMI : Introduction  |  SCD error estimate & statistical uncertainty OMI : Introduction  |  SCD error estimate & statistical uncertainty

SCD error estimate & statistical uncertainty

Note: The data presented are from a preliminary version of the collection 4 data. The final version of the data, extending to the end of the OMI mission, will be presented here in due time -- differences are, however, expected to be small.

The uncertainty of the retrieved SCDs is an important quantity: the lower the uncertainty, the higher the quality of the SCDs and hence of the subsequent NO₂ data product. On the one hand, the DOAS retrieval of the SCDs provides an estimate of the SCD error. On the other hand, the spatial variability of the SCDs over a remote Pacific Ocean sector can be used as an independent statistical estimate of the random component of the SCD uncertainty. For the analysis of both these indicators, all ground pixels of a given Pacific Ocean orbit with latitudes in the range [-60°:+60°] are considered, divided into three cloud regimes:

• all pixels with valid retrieval: qa_value > 0.50
• clear-sky pixels (i.e. pixels with a cloud radiance fracton < 0.5): qa_value > 0.75
• cloudy pixels: 0.50 < qa_value < 0.75

(For details see van Geffen et al., 2020, Sect. 4.6.)

Figure 1 shows the SCD statistical uncertainty over all valid pixels with gray lines. This quantity clearly varies considerably from day to day. To make the analysis visually easier, a 21-day running mean, shown in red in Fig. 1, is more convenient. Hence, the subsequent plots on this page show the running means, though the averages and trend lines shown are based on the original daily data.

Figure 1 SCD statistical uncertainty daily data (thin gray line) and 21-day running mean (red) using all pixels with successful retrieval, as function of time. Vertical lines indicate specific moments in time. See the text for further details.

The first thing to strikes out in Fig. 1 is a strong upward peak around 9 March 2016. Jacques Claas provided the following information:

According to the OMI weekly status reports from 14 and 21 March 2016, no irradiance and calibration measurements could be taken between 3 March and 16 March. While executing one of the so-called Stored Instruction Sequences (SIS) onboard the OMI-IAM on 3 March, an error occured which could only be solved on 16 March.
The missing calibration measurements included the orbital background measurements. As a consequence it was not possible anymore to generate a daily dark current map which must be subtracted from the radiance signal. Instead the most recent but less accurate dark current map was used in the data processing.

Judging from the numbers, the data cannot be trusted from 4 to 14 March 2016 (inclusive). In the following analyses the data of these 11 days has been marked as missing.

The vertical dashed lines in Fig. 1 are at the same dates as the vertical lines in the plots of the wavelength calibration offset analysis and the events mentioned there seem to also affect the SCD and DOAS uncertainties -- notably those of 30 Sep. 2008, 15 Dec. 2011 and 16 Nov. 2019 -- which is why the vertical lines are also plotted in the graphs shown below.

 
The DOAS (blue) and statistical (red) uncertainties are shown in Figure 2 for all ground pixels with a valid retrieval, in Figure 3 for clear-sky pixels only, and in Figure 4 for cloudy pixels only.

Lines are plotted to indicate the averages, both over the full period and over the "Zara period" period 2005-2015 (i.e. the data period used by Zara et al., 2018), and a linear fit over the full period; Table 1 below gives numbers for these.

From the plots it is very clear that the full-period averages have little meaning: the uncertainties increase quite substantially over time. The increases in the uncertainties is different in the main periods marked by the vertical lines:

• 1 Oct. 2004 to 30 Sep. 2008 shows an increase
• 30 Sep. 2008 to 15 Dec. 2011 there is perhaps no increase
• 15 Dec. 2011 to 16 Nov. 2019 again shows an increase
• 16 Nov. 2019 to present the increase is perhaps a little stronger

but the individual time periods are too short to be sure of differences in the increases.

Figure 2 SCD DOAS uncertainty estimate (blue) and statistical uncertainty (red) 21-day running means using all pixels with successful retrieval, as function of time. Dotted horizontal lines indicate averages over the full data period, thin-dashed horizontal lines indicate averages over the "Zara period" 2005-2015 (Zara et al., 2018), and thick-dashed lines indicate a linear fit over calendar years of the full data period; see Table 1 below for numbers.

Figure 3 As Fig. 2, but for clear-sky pixels only.

Figure 4 As Fig. 2, but for cloudy pixels only.

Another way to look at changes over time is presented without seasonal variation obscuring those changes is shown in Figure 5: yearly average uncertainties, over calendar years.

Figure 5 Yearly averaged DOAS and statistical uncertainties of calendar years 2005 onwards, in absolute numbers (left) and relative to 2018/19 (right).

Table 1 shows an overview of the SCD statistical uncertainty and DOAS uncertainty estimates shown in Figs. 2-4 over the listed periods for:

• column 2:
  OMI/QA4ECV collection 3 data over the "Zara period" 2005-2015 (taken from Zara et al., 2018), for which only averages are available
• column 3:
  OMI collection 4 data over the same "Zara period" for direct comparison with column 2
• columns 4 & 5:
  OMI collection 4 data over the full data period, both averages and slopes over calendar years -- given the large slopes, the averages themselves have limited meaning, as can be seen from Figs. 2-4
• columns 6 & 7:
  TROPOMI collection 3 data over the period after the ground pixel size reduction on 6 Aug. 2019, both averages and slopes (see this page for details)

for the three different cloud regimes, all given in two units. Some remarks:

• The numerical values of the statistical uncertainties can be compared directly, since these are based on the fysical quantity SCD.
• Since the DOAS implementations of OMI/QA4ECV and the other retrievals is different, their formulation of the DOAS uncertainty estimate differs slightly. Comparison of the averages over the same period in columns 2 and 3 shows, however, that differences are marginal, hence also the numerical values of the DOAS uncertainties can be compared directly.
• Comparison of columns 2 and 3 shows that the two algorithms essentially give the same uncertainties: differences are negligible.
• Column 5 shows that the uncertainties in the OMI data increase considerably, as a result of which simple averages (columns 2, 3 and 4) have little meaning.
• TROPOMI uncertainties (column 6) are on average much lower than the OMI uncertainties and the increase over time of the TROPOMI uncertainties (column 7) is about half of the increase over time of OMI (column 5) -- although obviously the TROPOMI data clearly spans a much shorter time period.
• The difference between the DOAS and statistical uncertainties is much larger for OMI than TROPOMI: for OMI the DOAS uncertainty is relatively larger than the statistical uncertainty because of the lower signal-to-noise ratio (SNR) of OMI compared to TROPOMI.

Table 1 SCD statistical uncertainty and DOAS uncertainty estimates of OMI and TROPOMI over the listed periods for for the three different cloud regimes, all given in two units; see the main text for details. See the text above the table for a short discussion

 
Figure 6 shows the running mean of the RMS of the average DOAS uncertainty shown in Figs. 2-4.

The row anomaly linked events mentioned earlier clearly have their impact on the RMS, including a clear change around the second line (1 Dec. 2009). Especially 30 Sep. 2008 marks a clear change in the RMS. Hence, both the overall averange and the linear fit are no good representatives of the change over time.

Figure 6 21-day running means of the RMS of the DOAS uncertainty. Dotted horizontal lines indicate averages over the full periods and the dashed line indicates a linear fit over calendar years of the full period.

Figure 7 shows the running mean of the average SCD values of all ground pixels that lie with a latitude within the Pacific Ocean range [-60°:+60°] for the full data period.

There is quite a bit of variation in the SCD over time, but one would expect that the SCD seen over a long time is more or less constant -- after all this is the Pacific Ocean area without NO₂ sources -- but there appear to be changes in the SCD value associated with (some of) the same dates as in the uncertainty and wavelength calibration graphs:

• 1 Oct. 2004 to 30 Sep. 2008 the SCD seems on average constant at 1.27 μmol/m2
• 30 Sep. 2008 to 15 Dec. 2011 there is a clear increase in the SCD
• 15 Dec. 2011 to 16 Nov. 2019 the SCD seems on average constant at 1.37 μmol/m2, 7.8% above the average over the first period; there may, however, be a small decrease over time during the period
• 16 Nov. 2019 to present the SCD seems on average constant at 1.33 μmol/m2

The overall average shown in Fig. 7 is 1.34 μmol/m2 (8.1e15 molec/cm2). Neither that average nor the lineaer fit shown are good representatives of the SCD over time.

Figure 7 21-day running mean of the SCD values of all ground pixels. The dotted horizontal line indicates the average over the full period and the dashed line is a linear fit over calendar years of the full period.

last modified: 26 June 2026
Contact: Jos van Geffen   < geffen [at] knmi [dot] nl >
Copyright © KNMI / ESA