Comprehensive evaluation of the Copernicus Atmosphere Monitoring Service (CAMS) reanalysis against independent observations: Reactive gases

The Copernicus Atmosphere Monitoring Service (CAMS; http://atmosphere.copernicus.eu) is a component of the European Union’s Earth Observation Programme Copernicus. This service is designed to meet the needs of policy makers and stakeholders for data and information concerning environmental issues such as climate change, air pollution, and other atmospheric challenges like volcanic eruptions. The CAMS core services include, among others, the daily production of near-real-time (NRT) analyses and forecasts of global atmospheric composition (AC), European air quality products with an ensemble system of regional models, and solar and ultraviolet (UV) radiation products. CAMS is also producing global reanalysis data sets of reactive trace gases, greenhouse gases, and aerosol concentrations. These retrospective analyses of AC are beneficial for air quality and climate studies (e.g., Bechtold et al., 2009; Benedetti et al., 2014), solar spectral irradiance studies (e.g., Mueller and Träger-Chatterjee, 2014; Polo et al., 2017), monitoring of stratospheric composition (e.g., Lefever et al., 2015; Errera et al., 2019), or as boundary condition for regional models (e.g., Schere et al., 2012; Giordano et al., 2015; Im et al., 2015). Within the CAMS preparatory project, Monitoring Atmospheric Composition and Climate (MACC), a 10-year reanalysis, was produced (Inness et al., 2013). About 3,000 users have downloaded the MACC reanalysis, which covers the years 2003–2012, since its release in 2013. The MACC reanalysis has not been extended because of major changes in the Integrated Forecasting System (IFS) model configuration (Flemming et al., 2015). After the release of a test reanalysis for reactive gases and aerosols (CAMS Interim Reanalysis; Flemming et al., 2017), the CAMS reanalysis was produced, which covers the years from 2003 onward (Inness et al., 2019). The MACC reanalysis suffered from known inconsistencies in the assimilated data, which led to drifts in carbon monoxide (CO) and ozone (O₃) model fields and limited its use for reliable trend studies.

An important part of the CAMS service is the provision of independent quality assurance information to the CAMS users. A dedicated validation team produces updated evaluations of the CAMS forecast products every 3 months, based on a multitude of independent observational data sets. A description of these validation activities and results are presented in Eskes et al. (2015), Cuevas et al. (2015), and Wagner et al. (2015). During the production of the CAMS reanalysis, a series of validation reports have been produced to monitor the stability of the data sets (Bennouna et al., 2020). All validation reports are publicly available and can be downloaded from the CAMS quality assurance webpages at https://atmosphere.copernicus.eu/quality-assurance. Inness et al. (2019) provide a comprehensive description of the CAMS modeling system used for the CAMS reanalysis and in this context also present selected initial comparisons to observations for the period 2003–2016 in order to demonstrate improvements compared to previous reanalysis runs, that is, the CAMS interim reanalysis (Flemming et al., 2017) and the MACC reanalysis (Inness et al., 2013). Wang et al. (2020) present additional validation results from comparisons with airborne field campaign data for the period 2003–2016. An intercomparison study of tropospheric O₃ reanalysis products based on the same period has been conducted by Huijnen et al. (2020).

This article presents the full evaluation results from the CAMS validation team with independent observational data, covering 16 years of reanalysis (2003–2018). Here, we evaluate the CAMS reanalysis of reactive gases, namely, O₃, CO, nitrogen dioxide (NO₂), and formaldehyde (HCHO). In the present study, the focus is set on the temporal consistency and stability of the model and on the quantification of seasonal and interannual biases. In order to thoroughly assess the impact of data assimilation and the influence of emissions, the results from a separate model run, for which data assimilation has been switched off (further referred to as “control run”), are included and analyzed. Improvements and shortcomings of the CAMS reanalysis compared to the previous MACC reanalysis are likewise quantified and discussed. The article is structured in the following way: Section 2 provides an overview of the reanalysis model system, the validation data, and methods. Section 3 discusses the validation results, and Section 4 handles the conclusions.

The CAMS reanalysis consists of 3D time-consistent AC fields, including aerosols and chemical species. The meteorological model is based on the IFS cycle 42R1, with interactive O₃ and aerosol feeding its radiation scheme, 60 hybrid sigma/pressure (model) levels in the vertical up to the top level at 0.1 hPa, and a horizontal resolution of approximately 80 km (Inness et al., 2019). For the CAMS reanalysis, IFS includes the modified Carbon Bond 2005 Chemical Mechanism (CB05) tropospheric chemistry scheme (Williams et al., 2013), which was originally developed for the TM5 chemistry transport model (CTM; Huijnen et al., 2010). The model computes stratospheric O₃ using the same Cariolle scheme (Cariolle and Teyssèdre, 2007), as in the meteorological production of IFS, while stratospheric NO_x is constrained through a climatological ratio of HNO₃/O₃ at 10 hPa. Inness et al. (2015, 2019) provide a detailed description of data assimilation for chemical trace gases and Benedetti et al. (2009) for aerosols. Table 1 lists the data sets used in the assimilation system, and Figure S1 displays a time series for data assimilation in the CAMS reanalysis. Anthropogenic reactive gas emissions are based on MACCity (Granier et al., 2011), where wintertime CO emissions have been scaled up over Europe and the United States (Stein et al., 2014). Monthly mean biogenic emissions are derived from hourly calculations by the Model of Emissions of Gases and Aerosols from Nature (MEGAN) using NASA’s Modern-Era Retrospective Analysis for Research and Applications (MERRA) reanalyzed meteorology (Sindelarova et al., 2014). NRT fire emissions are taken from the Global Fire Assimilation System (GFAS) v1.2 (Kaiser et al., 2012). Table S1 lists the major differences between the MACC and CAMS reanalysis data sets. In order to assess the impact of data assimilation, our evaluations also include the CAMS control run. The control run applies the same settings as the CAMS reanalysis, except that data assimilation is switched off. It consists of 24-h cycling forecasts and uses the meteorological fields from the CAMS reanalysis. A more detailed documentation of the CAMS reanalysis model setup can be found on the CAMS Confluence webpage. The CAMS 3D reanalysis products are stored as 3-hourly fields. Data are publicly available from the CAMS Atmosphere Data Store.

All data sets used in our validations are listed in Tables 2 and 3. As we use a wide range of different observations, more comprehensive descriptions of the individual observational data sets and validation algorithms are provided in the supplement (Section S1) and in Eskes et al. (2018). Validation metrics are listed in Table 4. More detailed information and a discussion concerning the use of the respective validation metrics can be found in Eskes et al. (2015) and Wagner et al. (2015). Table 5 lists all acronyms.

In the stratosphere, O₃ is validated with vertical profile observations from satellites and sondes as well as with partial column observations from the Network for the Detection of Atmospheric Composition Change (NDACC).

Figure 1 shows the results of the evaluation with Michelson Interferometer for Passive Atmospheric Sounding (MIPAS), Microwave Limb Sounder (MLS), and Atmospheric Chemistry Experiment-Fourier-Transform Spectrometer (ACE-FTS) satellite data averaged over all longitudes and over the three most interesting latitude bands for stratospheric O₃: Antarctic (90–60°S), Tropics (30°S–30°N), and Arctic (60–90°N). In the upper stratosphere (3–10 hPa, see top row), the absolute value of the bias is generally less than ±10% for all instruments, except in 2003–2004 in the north polar region, where larger biases appear. This is related to the degraded quality of the assimilated SCIAMACHY and MIPAS data in 2003 and also to the lack of MLS O₃ profile data for assimilation until the beginning of August 2004 (Inness et al., 2019). A negative bias against MLS (maximum 4%), ACE-FTS (maximum 10%), and MIPAS (maximum 12%) is systematically present in this layer since mid-2004. All limb data sets show that this O₃ deficit has a seasonal component: The negative biases are more pronounced in summer than in winter (see Figure S2). The seasonal biases with respect to MLS are much smaller than for MIPAS. This indicates that the CAMS reanalysis is much more constrained by MLS than by MIPAS and that the seasonal patterns mostly reflect differences between MIPAS and MLS. Seasonal biases between MIPAS and MLS have already been published for the midlatitudes (Errera et al., 2019) and are shown here in the tropical and polar regions. While such interinstrument biases deserve more investigation, they are beyond the scope of this article.

In the middle stratosphere (10–30 hPa, middle row), after 2004, the bias is generally within ±5% for all limb-scanning satellite data sets. This is in very good agreement with a recent comparison of ACE-FTS with MLS and MIPAS observations, which also reports biases within 5% in the middle stratosphere (Sheese et al., 2017). In the lower stratosphere (30–70 hPa, bottom row), the spread of the biases against the different instruments is larger. The bias against MLS and ACE-FTS is positive (3%–7%) in the Tropics. In the polar regions, the bias against MLS is low (<1.5%), except during the O₃ hole events (September–October in the 90°–60°S latitude band) when O₃ abundances are approximately 3% larger than in the MLS data set, that is, O₃ depletion is slightly underestimated. The biases against MIPAS and ACE-FTS are negative but remain always lower than 10% and 5%, respectively.

Figure 2 presents the deseasonalized time series of the normalized mean biases based on the monthly climatologies shown in Figure S2. The importance of high-quality assimilation data most obviously shows for the year 2003, where the limited availability of such data has a negative impact on the validation results. The bias against ACE-FTS is stable over time, except for a slight drift after 2013 in the lower tropical stratosphere (but <1% over 5 years). The deseasonalized biases with both MLS and ACE-FTS exhibit an increase of approximately 2% over the last year (2018) in the polar middle stratosphere and in all three regions of the lower stratosphere. This feature continues in 2019 and is likely related to the switch from reprocessed to NRT MLS V4 data.

Figure S3 gives a global overview of the agreement between the CAMS and MACC reanalysis and the observations by the limb-scanning instruments, averaged over the whole period. The correlation between the reanalysis and the observations is very good (at least >0.8 for the pressure range 2–200 hPa or altitude range 10–45 km for ACE-FTS).

The CAMS reanalysis setup does not include explicit modeling of stratospheric chemistry. The stratospheric O₃ profile is constrained using the Cariolle parametrization (e.g., Cariolle and Teyssèdre, 2007). In practice, this leads to considerable biases in the stratospheric profile observed in the control run when comparing with O₃sonde (Figure 3; Figures S4–S7, Table S2) and NDACC data (Figure S8, Table S3) in the stratosphere, reflecting uncertainties in the Cariolle O₃ parametrization. Large positive biases of up to 40% appear over Antarctica and the Arctic and modified normalized mean biases (MNMBs) up to 20% over the Northern midlatitudes. For NDACC stations in the Southern midlatitudes and over the Tropics, the control run shows negative biases up to –15% (Figure S8).

The assimilation of O₃ total columns and stratospheric profiles in the CAMS reanalysis successfully compensates for this lack in explicit stratospheric chemistry and thus proves to be very effective in constraining stratospheric O₃. The seasonal and interannual evaluation with O₃sonde data shows a very stable and consistent performance of the CAMS reanalysis during all years and seasons (Figures S4–S7) with MNMBs mostly smaller than 5%. Only over the Arctic, a very small change in bias from negative to positive is visible from 2017 onward, which likewise appears in the control run.

In the troposphere, our validations rely on sonde observations (Figure 4; Table S4) and In-service Aircraft for a Global Observing System (IAGOS) measurements (Figure 5; Table S5).

For the CAMS reanalysis, MNMBs in all regions are mostly within 10%, with respect to sonde observations. Larger positive biases appear over the Tropics and Antarctica.

The control run underestimates O₃ with MNMBs between 10% and 20% in all regions except the Tropics, where O₃ is overestimated with MNMBs up to 20% (Figure 4; Figures S9–S11). The continual overestimation of O₃ could potentially relate to an overestimation of precursor emissions (fire and biogenic emissions) in this region (see Section 3.4). In the high latitudes and Northern midlatitudes, the control run has a seasonal pattern in the bias, with larger negative biases during winter and spring.

Data assimilation effectively increases tropospheric O₃, which improves MNMBs in most regions for the CAMS reanalysis compared to the control run. Likewise, the seasonality in the biases is largely eliminated by the data assimilation. In the Tropics, data assimilation is less effective in improving the biases for the CAMS reanalysis compared to the control run.

Two periods with larger MNMBs in the time series are noticeable for the CAMS reanalysis, which affect the long-term consistency and relate to data assimilation issues: Firstly, from 2003 to 2004, where MNMBs show a drop until August 2004 (Figure 4), already explained in the stratospheric section. Secondly, from 2013 onward, where MNMBs of the CAMS reanalysis are suddenly increasing especially over Antarctica. The change in MNMBs is accompanied by a more pronounced seasonal variation of biases. The interannual seasonal time series in Figure 6 and Figures S9–S11 clearly shows shifts in MNMB, which are most distinct over the high latitudes (Arctic, Antarctica). This effect was still under investigation in the study of Inness et al. (2019). It can now be attributed to the switch from 13 L V8.6 SBUV/2 to 21 L NRT SBUV/2 data in July 2013 and might also be affected by the loss of Envisat data in April 2012 (SBUV: Solar Backscatter Ultraviolet Radiometer). These changes in the assimilation system cause a jump in the bias in December to February (DJF) over Antarctica and a drop in biases in June to August (JJA; Figure 6). Over the Arctic and Northern midlatitudes, biases likewise drop during DJF 2013/2014 (Figures S9 and S10). In the Tropics, we find larger differences between the control run and the CAMS reanalysis after these changes in 2013 (Figure S11).

The evaluation with IAGOS data shows MNMBs of –5% and 25% over North America, –10% to 20% over Europe, between –10% and 40% over East Asia, and ±35% over India (Figure 5). O₃ is mostly overestimated over the more polluted metropolitan sites, especially over India and East Asia (see Figures S12–S15). The control run has larger positive MNMBs over India and partly over East Asia, but lower MNMBs over Europe and North America (see also Figure S16). Over India and East Asia, biases increase during July–September, revealing problems of the model to correctly simulate the low O₃ values during the Asian monsoon season (Figure S16). The seasonal pattern in the biases is almost absent for Europe and North America and reduced for India and East Asia for the CAMS reanalysis, which is more constant over the years, however, with partly larger positive biases compared to the control run.

Modeled surface O₃ is compared to Global Atmosphere Watch (GAW) data (Figure 7), International Arctic Systems for Observing the Atmosphere (IASOA) network data (Figure S17), European Monitoring and Evaluation Programme (EMEP) observations (Figure 8 and Figure S18) and to IAGOS aircraft measurements (Figure S19).

Figure 7 shows the time series of MNMBs for surface O₃ calculated from model and GAW observational data, as regional average over stations in four different latitude zones. Biases for surface O₃ are generally larger than for O₃ in the free troposphere but remain within ±30%. Largest biases for both CAMS reanalysis and control run appear during Arctic spring (MNMBs up to 80%).

A closer look at Arctic stations from the IASOA network (Figure S17, Table S6) reveals that the High Arctic coastal stations (Alert, Barrow, and Villum Research Station) are influenced by O₃ depletion events during arctic spring (MAM: March to May). These halogen chemistry reactions are not represented in the simulations, and the model is thus unable to capture the low concentrations measured in spring at these sites. European Arctic IASOA sites (Esrange, Karasjok, Oulanka, Pallas, and Tustervatn), which are located inland, are not affected by O₃ depletion events and thus show smaller biases during springtime (see Figure S17).

Our evaluations show that the impact of data assimilation is rather small at the surface, which reflects in almost identical biases for the CAMS reanalysis and control run (Figure 7; Figures S17–S19). Differences between the control run and the CAMS reanalysis appear for high-latitude regions (Figure 9; Figure S20), where data assimilation increases surface O₃, which partly improves the negative bias in the control run but partly also leads to overestimations of modeled O₃, especially during Arctic/Antarctic spring (MAM/SON, respectively; SON = September to November).

For Antarctica (Figure 9), the changes in the assimilation system described for tropospheric O₃ are visible even at the surface in a distinct shift in bias from negative to positive during DJF, and MAM 2012/2013, whereas the control run remains stable.

In the Northern midlatitudes (Figure 7), the interannual time series of biases show a constant seasonal pattern in the biases, with negative biases during DJF and MAM and larger positive biases during JJA and SON. This seasonal pattern in surface O₃ biases is very common in global CTMs and has been discussed in various studies before (e.g., Ordóñez et al., 2010; Val Martin et al., 2014; Wagner et al., 2015).

For Europe, we have additionally investigated these seasonal variations of MNMBs for different latitude zones for EMEP data sets (Figure 8). The seasonal mean variability of biases for the CAMS reanalysis has been separated in nighttime (0–3 UTC) and in midday time (12–15 UTC) values (UTC: coordinated universal time). For Northern Europe, and to a smaller extent for Southern Europe, the overestimation in summer and underestimation during winter are stronger for nighttime O₃ than for daytime O₃. This likely means that nocturnal O₃ destruction processes in the boundary layer (like NO_x titration) are not reproduced correctly in the model. The global model has difficulties to resolve such regional subgrid processes (see e.g., Wagner et al., 2015). For stations above 1,000 m, only positive biases are present throughout the year without changes between nighttime and daytime. The overestimation reaches a maximum in October (MNMBs of 15%).

There is a drift in the interannual time series of seasonal MNMBs toward larger positive biases for the Northern midlatitudes (Figure 10). Similar drifts can be observed in the evaluation with EMEP data for Northern Europe (Figure S18) as well as in the evaluation with IAGOS surface data over Europe and East Asia (Figures S19 and S21). Park et al. (2020) accordingly note inadequateness of the CAMS reanalysis to capture O₃ trends for East Asia. In the Tropics, a drift is visible for JJA (Figure S22).

Although further investigations and sensitivity analysis will be needed to prove this, it is likely that unreasonable trends in the emissions are responsible for the drifts (see Section 3.4). In a recent study, Gaubert et al. (2020) show that running CTMs with biased CO and volatile organic compound (VOC) emissions can lead to poorly modeled O₃.

Compared to the previous MACC reanalysis, we see large improvements for the CAMS reanalysis for O₃ (see Figures 1–5 and Figure 7; Tables S3, S7, and S8): in the upper and middle stratosphere over the poles, the refinements compared to the MACC reanalysis are most striking. The overestimation of O₃ below around 15 hPa completely disappears in the CAMS reanalysis due to a better setup of the variational bias correction scheme, which is now applied only to total column O₃ retrievals but not to profiles from MLS or MIPAS (Inness et al., 2013, 2019). The absence of the drift also leads to improvements for the free troposphere and surface. Apart from the drift, the MACC reanalysis also shows large negative biases (MNMBs down to –150%) in the high latitudes (Figure 7). This is a known issue of the former coupled IFS-MOZART-3 (MOZART: Model for OZone And Related chemical Tracers) CTM used in MACC (see e.g., Wagner et al., 2015). The improvements from MACC to CAMS mostly relate to changes in the chemistry module, that is, the replacement of the coupled model system (IFS and MOZART-3 CTM) used for the MACC reanalysis by the online-coupled model C-IFS (with CB05 of the TM5 CTM) used for the CAMS reanalysis (Flemming et al., 2015). As a result, MNMBs, especially in high-latitude regions, are considerably smaller and more stable. Furthermore, the seasonality of O₃ is better captured.

Modeled CO is compared to Measurements of Pollution in the Troposphere (MOPITT) and Infrared Atmospheric Sounding Interferometer (IASI) total column satellite measurements (Figures 11 and 12; Figures S23 and S24), to NDACC partial column measurements (Figure 13; Figure S25 and Table S9), and to IAGOS aircraft data (Figure 14; Figures S26 and S27 and Tables S10 and S11) and GAW CO surface observations (Figure 15; Figures S28 and S29 and Table S12).

Figure 11 shows MOPITT total column values as a function of latitude and time and the biases in comparison with the CAMS reanalysis and control. Observed CO total columns are slightly underestimated by the model over all regions with MNMBs mostly within ±10%. Larger MNMBs (up to 20%) appear over tropical regions, especially during the years 2012–2015.

The control run has larger CO in all regions, especially during the winter season. Largest positive MNMBs (up to 50%) show up over the Southern Hemisphere (SH) during November to May. For later years (2012 onward), the overestimation also reaches over to the northern hemispheric low latitudes. This effect in the control run accordingly appears in the validation with NDACC data in the SH (Figure 13) and has likewise been described by Flemming et al. (2017) for the CAMS interim reanalysis. In their study, the authors assume that the overestimation of CO in the SH points to deficiencies in the simulation of the global chemical loss and production of CO as well as problems with large-scale transport. To a minor extent, an overestimation of the GFAS biomass burning emissions for Central Africa, Maritime South East Asia, and South America could also seasonally contribute to this according to Flemming et al. (2017). In the frame of our validations, we find that the biogenic VOC emissions (MEGAN-MACC; Sindelarova et al., 2014; Sindelarova, 2018) as chemical source term for CO may also play a considerable role concerning the overestimation of CO in the SH observed for the control run, as discussed below. In a comparison of available isoprene emission data, Sindelarova (2018) shows that the MEGAN-MERRA biogenic emissions (used in the CAMS reanalysis and control run) are about 1.5–2 times higher and show larger year-to-year differences than other available data sets. Additionally, the TM5 model seems to be more sensitive to changes in VOC (NMVOC) rates, as described by Zeng et al. (2015). In combination with larger biogenic emissions, this effect might contribute to the variation and magnitude of biases for CO shown for the control run. Data assimilation reduces total column CO for the CAMS reanalysis compared to the control run, and positive biases remain only for stations in the high latitudes of the SH (see Figure 13).

Figure S23 displays observed time series of total column CO concentrations of MOPITT and IASI in comparison to the CAMS reanalysis and control run over different regions. Figures 12 and S24 show the resulting time series of MNMBs. The differences of total column CO between MOPITT and IASI are discussed by, for example, Illingworth et al. (2011) and George et al. (2015) and will not be further addressed in the frame of this article. For Europe and the United States, biases of the CAMS reanalysis compared with total column data remain stable over the entire period with a seasonal variation showing larger underestimation during the summer season (up to –12%) and lower underestimation during winter (up to –5%). A similar seasonal pattern appears over East Asia and South Africa, whereas over North Africa, it is reversed. For IASI data, the seasonal pattern is stronger, and MNMBs are generally larger (up to –18% in summer and ±8% in winter). The control run, however, has larger variable CO concentrations over all regions. In comparison with MOPITT, this partly leads to an overestimation of CO during the winter season (Europe, United States, and East Asia).

The variability in the time series of biases in the control run closely resembles the annual variability of the CO burden of the control run, with low total column CO in 2008 (a La Niña year) and large total column CO in 2015 (an El Niño year with high fire activity). Main drivers for the spatial and temporal CO burden are wildfire emissions and anthropogenic emissions (Flemming and Inness, 2019).

An overestimation of fire emissions could explain larger CO in the SH and maxima during the El Niño year 2015. However, it does not explain the variability observed for other years like the increasing biases between 2011 and 2015 in the control run.

Various studies (Hassler et al., 2016; Elguindi et al., 2020) describe an inaccurate representation of CO emission trends in the MACCity inventory for Europe and the United States. Hassler et al. (2016) show that reduction trends of vehicle CO emissions in U.S. cities after 2007 are not captured correctly in the MACCity inventory. Elguindi et al. (2020) show large uncertainties for regions like China, where various bottom-up inventories likewise cannot capture reduction trends in emissions especially after 2010. Moreover, the MACCity emission inventory has not been updated after 2010, and recent years are only based on projections of past trends.

Next to anthropogenic and fire emissions, another source for CO lies in the oxidation of biogenic VOC emissions as mentioned before. The annual variability in CO biases in the control run closely resembles the annual variation of the MEGAN-MACC isoprene emissions (see Sindelarova et al., 2014; Sindelarova, 2018; Figure 3), with low values during the La Niña year 2008 and an increase during the years after up until the El Nino year 2015/2016. Further investigations, including sensitivity tests, are needed to thoroughly disentangle the specific influence of the different emission sources on the variability and drifts of the biases in the control run.

For most regions shown in Figure 12, the biases remain stable for the CAMS reanalysis, which clearly illustrates how well assimilated MOPITT data manages to constrain modeled CO and thus successfully prevents drifts caused by the emissions. This underlines the importance of a stable and consistent satellite CO product for assimilation. Only for East Asia, a drift in bias visibly remains in the CAMS reanalysis (Figure 12).

The big fire events in 2003, 2008, and 2012 over Siberia could be captured correctly by the model (Figure S23). In the control run, the magnitude of the events is mostly slightly overestimated, which points to an overestimation of the fire emissions during these events. Autumn and winter 2015/2016 is the period with the largest positive biases. Especially over Asia, autumn 2015 is an exceptional season with increased biases of up to +10%. September and October 2015 were marked by a strong El Niño event, which intensified during the dry season over large regions of Indonesia. During these months, the largest amount of fire emissions were recorded in Indonesia since 1997 based on GFED emissions time series (Huijnen et al., 2016b). The overestimation of emissions during December to March is strongly present in the control run especially for South Asia but could be greatly reduced by the data assimilation.

Figure 13 and Table S9 show relative differences of model and Fourier-transform infrared (FTIR) partial columns (NDACC) for different sites and latitudes. The results for the control run accordingly show the large positive error in the SH and the drifts toward positive biases from 2011 onward for the Northern midlatitudes as described in the validation with satellite data before. The relative differences for the stations in the SH show a significant positive trend in the relative biases: Maido: +0.7%, Lauder: +0.2%, and Wollongong: +1.1% (Figure S25). A negative trend at the Antarctic station Arrival Heights is likely related to the observations, which are dropping in 2009 (Figure S25). For FTIR stations located in the United States and Europe, the validation reveals an underestimation of modeled CO partial columns with values fluctuating around –5% and smaller (Toronto). For Jungfraujoch, Zugspitze, and Toronto, the largest underestimation of CO appears during the summer months (JJA) when CO is low, with values of up to 4% lower compared to the relative bias during the winter months. For three Arctic sites (Eureka, Ny Alesund, and Thule), CO partial columns are slightly underestimated by the model in the order of –5%, which is close to the reported measurement systematic uncertainty. For all three Arctic sites, the strongest underestimation (up to –10%) takes place during the end of winter (January to February, with only few measurements due to polar night) and early spring (March to April). For three sites located in the low latitudes of the NH (Izana, Mauna Loa, and Altzomoni) and five sites in the SH results show that the data assimilation mostly corrects the positive biases in the control run. For some sites (Alzomi and Wollongong), the assimilation overcorrects to negative biases (around –10%). The correlation between the model and observations is very high (between 0.79 and 0.96).

The evaluation with IAGOS aircraft measurements (Figure 14; Figures S26 and S27, Table S11) and GAW surface observations (Figure 15; Figures S28 and S29) shows that CO mixing ratios in the free troposphere and at the surface are mostly underestimated by the CAMS reanalysis and control run during 2003 until 2013 for both Europe and East Asia. For later years, the control run partly overestimates CO in the free troposphere. For North America, the validation at the surface shows partly positive MNMBs up to 20%–50% (Figure 15; Figure S26). Traffic emissions have been scaled during the winter season for North America and Europe (Stein et al., 2014), which improves the biases for the winter period in both regions (Figures S27–S29). However, during the summer season, surface CO is still too low in Europe, which points to too low emissions in the MACCity inventory (accordingly reported by Hassler et al., 2016) or too large CO sinks. The strong overestimation of modeled surface CO over India points to an overestimation of emissions in this region (Figure S26).

Compared to the MACC reanalysis, the CAMS reanalysis is more consistent over time and has reduced biases (Figures 13–15; Tables S9–S12). For NDACC stations in the high northern latitudes (Figure 13), the MACC reanalysis has a negative trend in bias from 2008 onward (biases of up to –20%). This is related to discontinuities in the data assimilation of CO satellite data in the MACC reanalysis, namely the assimilation of IASI satellite data in April 2008, which leads to a decrease of CO especially in the high northern latitudes (see also Flemming et al., 2017; Inness et al., 2019). The distinct drop in bias in April 2008 can be noticed accordingly for validations at the surface with GAW data for North America (Figure 15). The CAMS reanalysis that assimilates only MOPITT data is more consistent here and thus shows significant improvement. For tropical regions in the Northern Hemisphere (Izana, Mauna Loa, and Altzomoni), the CAMS reanalysis has an increased bias, which is not observed in the MACC run, however. The validation with GAW surface data shows that the MACC reanalysis has a negative offset for surface CO mixing ratios in Europe and North America (see Figure 15, GAW; Figure S26, IAGOS), which is likely related to the unrealistically low wintertime road traffic emissions that were still unscaled in the MACC reanalysis (Inness et al., 2013, 2019).

Tropospheric NO₂ columns have been validated with SCIAMACHY and GOME-2 satellite data (Figures 16–18; Figure S30). Global monthly mean map comparisons show that the spatial distribution of tropospheric NO₂ is overall reproduced well by the CAMS reanalysis (see Figure 16 as an example of seasonal averages for 2018). However, some differences are apparent between observations and simulations: Simulated shipping signals are generally larger than observed ones, and values over anthropogenic pollution hotspots broadly around the Benelux countries and the German Ruhr area tend to be underestimated (especially in winter), while values over other pollution hotspots are overestimated (e.g., Moscow and Teheran). Boreal forest fire emissions are regularly overestimated over Siberia, Canada, and Alaska mainly from June to August (JJA), and values over fire emission regions are also regularly overestimated in Eastern Russia mainly in April and over South Africa in JJA. The CAMS reanalysis overestimates values over the Persian Gulf and the Red Sea mainly in JJA and SON. Further analysis shows that overestimations of fire emissions do not always occur in both NO₂ and HCHO, especially for boreal forest fires. This difference points to uncertainties regarding fire emission factors, with the results indicating a different performance depending on the trace gas, region, and season.

Time series comparisons based on SCIAMACHY and GOME-2 satellite retrievals are shown in Figure 17. Inness et al. (2019) present initial results based on the CAMS reanalysis forecasts for Europe and East Asia. Here, time series of the CAMS analysis results, control run, and MACC reanalysis are shown for East Asia, Europe, North Africa, and South Africa. Corresponding time series of the MNMB (only positive daily averages are taken into account for the calculation of the MNMB based on the satellite data) are given in Figure 18. The CAMS reanalysis reasonably reproduces the magnitude and seasonality in general. Larger differences between the control run and the CAMS reanalysis are apparent only for East Asia. Inness et al. (2019) have shown that CAMS reanalysis forecasts reproduce the satellite-observed increase of wintertime values up to 2014 better than the former MACC reanalysis. This could be the result of assimilating NO₂ from more instruments (with different overpass times) in the CAMS reanalysis, namely, SCIAMACHY, Ozone Monitoring Instrument, and GOME-2. However, maxima are still underestimated by the model. Satellite observations show a strong decrease of wintertime NO₂ over Europe after 2011 and over China after 2014. This tendency is generally captured by the CAMS reanalysis, however, to a lower extent. It shows that control and CAMS reanalysis are closer to each other than in previous years. For summer NO₂, the model does not reproduce the decreasing trend visible for the satellite observations from 2011 onward. Especially for East Asia, values in the model increase instead (see Figure 18). In the time series of MNMBs, an increasing trend in MNMBs of CAMS reanalysis and control can thus be noticed for both, for maxima in winter and minima in summer, most clearly seen for East Asia. This trend likely originates from unrealistic emission trends in the model. Another issue appears in 2003: The CAMS reanalysis (in contrast to the control run) shows a strong variation of values from 1 month to another and fails to reproduce the observed seasonality due to assimilation of degraded quality NO₂ data described in Inness et al. (2019). The seasonal cycle (Figure 17) is partly underestimated over Europe, related to the underestimation of values over Central European pollution hotspots described above. The CAMS reanalysis overestimates the seasonal cycle over South Africa, with an underestimation of values around December to April (by about 0.2 × 10¹⁵ molecule cm⁻²) and an overestimation of values by about 0.5 × 10¹⁵ molecule cm⁻² around JJA (the latter was not present in the MACC reanalysis), which is probably related to an overestimation of fire emissions in this region. As for South Africa, the seasonal cycle is overestimated for North Africa as well, but mainly because of an overestimation of values around December, which, however, improves for the last years of the time series (see also Figure 18). The control run is closer to satellite-retrieved maxima over both regions in Africa (dominated by fire emissions) and vice versa over Europe and East Asia (dominated by anthropogenic emissions). Interannual variations are not always captured by the CAMS model. For Europe, reduced satellite-observed NO₂ columns are clearly noticeable over Europe during winter 2007 and 2008, while the simulations do not show a reduction.

Figure S30 shows climatological monthly averages of the MNMB for each region. The monthly MNMB shows little variation from year to year, reflected by low standard deviations. The absolute value of the MNMB of the CAMS reanalysis is in general largest over North Africa around April, over East Asia and South Africa in JJA, and over Europe from October to January. Offsets between the CAMS reanalysis, CAMS control, and MACC reanalysis are visible for East Asia with the CAMS model runs showing smaller absolute values of the MNMB throughout the year. Further offsets are found for North Africa and South Africa.

Tropospheric HCHO columns have been validated with SCIAMACHY and GOME-2 satellite data (Figures 19–21; Figure S31). Global monthly mean map comparisons (see Figure 19 as an example of seasonal averages for 2018) to satellite retrievals from SCIAMACHY and GOME-2 show that the magnitude of oceanic and continental background values and the overall spatial distribution are well represented by the CAMS reanalysis. Compared to SCIAMACHY and GOME-2 satellite retrievals, there is an overestimation of values over regions with fire activity in Central Africa mainly in DJF/MAM and over Eastern Russia around April, as well as over regions with fire and biogenic emissions in Northern Australia mainly during SON and DJF. As there is no HCHO data assimilated, there is only a little difference between CAMS reanalysis and control run for HCHO in general. Time series comparisons are shown in Figure 20 for selected regions, together with the MNMBs in Figure 21 (only positive daily averages are taken into account for the calculation of the MNMB based on the satellite data). East Asia and the Eastern United States are considered as regions dominated by biogenic emissions with some anthropogenic input. The CAMS reanalysis reproduces satellite observations for Eastern United States with respect to absolute values and seasonality but does not match the maxima of the satellite retrievals for individual years. Seasonality over East Asia is generally underestimated with differences of up to approximately 1 × 10¹⁵ molecule cm⁻² shown in the time series, the MACC reanalysis is closer to satellite-retrieved HCHO columns than the CAMS reanalysis here. For the regions North Africa and Indonesia, which are dominated by biogenic and pyrogenic sources, the reanalysis runs show a positive offset compared to satellite retrievals. The seasonality is in agreement with the retrievals for Indonesia and overestimated for North Africa. For September and October 2015 over Indonesia, satellite retrievals and simulations show a distinct maximum, which is, however, much more pronounced in the simulations. September, October, and November 2015 were strong El Nino months (e.g., NOAA El Niño webpage), which caused droughts and higher fire activity in Indonesia. Fire emissions used by the CAMS reanalysis seem to be largely overestimated for this El Nino year, resulting in an overestimation of up to a factor of 1.8 compared to the observations. A similar overestimation was also reported for the CAMS NRT product, for which it was shown that this is not due to cloud flagging applied to the satellite and model data (Huijnen et al., 2016a). Note that weaker El Nino conditions occurred for periods in 2006 and 2009, which resulted in maxima in the satellite-retrieved values that are reproduced for 2006 but again overestimated for 2009. Figure S31 shows climatological monthly averages of the MNMB for each region. A seasonal cycle of the MNMB is only found for Eastern United States, with values around –0.9 in DJF and about 0.2 from spring to autumn for the CAMS reanalyses. As for NO₂, the monthly MNMB shows little variation from year to year (low standard deviations).

Comparison with the MACC reanalysis shows that the CAMS reanalysis has lower MNMBs for Indonesia and East Asia during the summer period (Figure S31), likely related to the differences in the fire emissions used (GFED/GFAS v0 in MACC and GFAS v1.2 in CAMS). For other regions and periods, the CAMS reanalysis results in larger MNMBs.

CAMS provides its users with a variety of different products in the field of air quality and AC. Beside the NRT forecasts, there is also a large and growing interest in long-term retrospective analysis (reanalysis) data sets. After the release of the MACC reanalysis in 2013, and an interim test product in 2015, CAMS has now produced a new reanalysis data set (CAMS reanalysis), which is freely available to the public.

We have comprehensively validated the reactive gas species (O₃, CO, NO₂, and HCHO) of this new product in the period 2003–2018 with multiple independent observations. For reanalysis data sets, a temporal stability of the model results over time is crucial, for example, for trend studies on chemical species. Special focus was thus set on the long-term consistency shown in the time series of biases and on the assessment of seasonal and interseasonal changes in biases. In order to thoroughly evaluate the impact of data assimilation on the long-term quality of results, a comparison with a control run without assimilated data is conducted. Finally, improvements and shortcomings of the CAMS reanalysis compared to the previous MACC reanalysis are quantified and discussed.

Our evaluations show that the CAMS reanalysis reproduces O₃ with MNMBs mostly within ±10% in the stratosphere and troposphere of the Northern midlatitudes compared to sonde observations and satellite instruments. Larger biases (up to ±38%) appear over the high latitudes, the Tropics, and generally for surface O₃. Total column CO over Europe, the United States, East Asia, and North Africa is reproduced with MNMBs mostly within ±10%. Larger MNMBs appear over East Asia and for surface CO, reaching up to ±40%. The CAMS reanalysis performs reasonably well regarding the magnitude and seasonality of NO₂ in comparison with SCIAMACHY and GOME-2 NO₂ satellite retrievals. Stronger shipping signals show up compared to the satellite observations, and NO₂ in boreal fire regions is overestimated in summer, whereas NO₂ over the pollution hotspots of Central Europe is underestimated in winter. Modeled HCHO columns mostly show a good agreement with SCIAMACHY and GOME-2 satellite observations. For regions dominated by biogenic emissions with some anthropogenic input (East Asia and Eastern United States), the CAMS reanalysis reproduces absolute values and seasonality but fails to match the maxima of the satellite retrievals for individual years. The seasonality over East Asia is generally underestimated with differences of up to approximately 1 × 10¹⁵ molecule cm^–2. For regions where biogenic and pyrogenic sources dominantly influence HCHO columns (North Africa and Indonesia), the CAMS reanalysis shows a positive offset compared to satellite retrievals. Concerning the long-term consistency, our results show that the CAMS model system mostly provides a stable and accurate representation of the global distribution of reactive gases over time.

However, the comparison with the control without data assimilation reveals some shortcomings in the model and emissions: The lack of an explicit modeling of stratospheric chemistry leads to large biases for stratospheric O₃. For tropospheric and surface O₃, the model shows seasonal patterns in the biases in midlatitude and high-latitude regions with larger negative MNMBs during winter and spring. For the Arctic, large positive biases appear during O₃ depletion events in spring. Furthermore, the control run shows large overestimation for CO in the SH, likely related to the overestimation of fire and biogenic emissions, together with shortcomings in the simulation of the global loss, production, and large-scale transport of CO. Overestimations of HCHO concentrations likewise suggest that fire emissions are overestimated over boreal regions, Indonesia, Africa, and East Asia, especially during years with high fire activity like during the strong El Niño event 2015/2016.

Finally, we also discovered positive drifts in the interannual time series of biases for various species (CO, O₃, and NO₂) in the control run, likely triggered by unrealistic emission trends, especially after 2010. Data assimilation is able to successfully constrain stratospheric and tropospheric O₃ and CO and thus ensures the long-term consistency and stability of the CAMS reanalysis. However, this works less effectively near the surface and for short-lived species like NO₂.

Our evaluations concerning the long-term stability of the CAMS reanalysis show that the consistency in the quality of model results is also essentially affected by limitations in the availability of high-quality data for assimilation and by changes in the assimilated satellite data sets. Especially during the first years, degraded quality data and the lack of O₃ profile data deteriorated the validation results for O₃ and NO₂. Modifications in the assimilation system in 2012/2013 cause jumps in the interannual seasonal time series of biases for tropospheric and surface O₃, especially over high-latitude regions. For trend analysis, these effects related to changes in the data assimilation need to be considered and removed.

Compared to the MACC reanalysis, the CAMS reanalysis has systematically lower biases, better correlation, and a weaker seasonal pattern for O₃ especially in the free troposphere and at the surface. Aside from the improved data assimilation, especially the change in the chemistry module, that is, the online-coupled IFS model combined with the different chemistry treatment in the CB05 model leads to a reduction of biases for tropospheric and surface O₃. Largest improvement in the magnitude of biases is thus found over the Arctic and Antarctic regions. For CO, the comparison with the MACC reanalysis shows that the scaling of the winter road traffic emissions and a more consistent data assimilation improve the results. For NO₂, the CAMS reanalysis shows a better reproduction of wintertime NO₂ over East Asia. For South Africa, however, the CAMS reanalysis has larger underestimations of observed NO₂ during SH summer and autumn. For HCHO, improvements compared to the MACC reanalysis can be seen over Indonesia, but for North Africa and the Eastern United States, the MACC reanalysis shows smaller biases.

For next-generation CAMS reanalyses, challenges in the data assimilation will include the integration of more species (e.g., HCHO) and additional sensors (O₃ profile data turned out to have a crucial impact) while assuring good long-term stability of results. New sensors such as sentinel 5P will provide promising perspectives. Our results concerning the control run suggest that deficiencies in the model’s chemistry and transport scheme need to be investigated and improved further, especially in combination with the emission data sets, to remove large zonal errors such as the overestimation of CO in the SH as well as drifts and seasonal patterns in the biases of the control run. Given the large impact of anthropogenic, fire, and biogenic emissions on CO, more care should be taken to investigate and consolidate emission rates and trends. Simple scaling approaches as conducted for the wintertime traffic emissions could be replaced by more sophisticated approximations, available from recent bottom-up and top-down inventories. A more comprehensive stratospheric chemistry scheme could improve the model results in the stratosphere. Furthermore, the integration of small-scale local processes, such as halogen chemistry, could help to reduce more local sources of errors in the model system, such as the large biases for surface O₃ over the Arctic in springtime.

Bennouna et al. (2020): Validation report of the CAMS global Reanalysis of aerosols and reactive gases, years 2003–2019, Copernicus Atmosphere Monitoring Service (CAMS) report: Available at CAMS84_2018SC2_D5.1.1-2019.pdf, Accessed 4 May 2021, DOI: http://dx.doi.org/10.24380/2v3p-ab79.

The CAMS reanalysis data set that was not produced in the frame of our study but as a product of CAMS can be found at Atmosphere Data Store. Available at https://ads.atmosphere.copernicus.eu. Accessed 4 May 2021.

The supplemental files for this article can be found in a composite file (docx).

The Copernicus Atmosphere Monitoring Service is operated by the European Centre for Medium-Range Weather Forecasts (ECMWF) on behalf of the European Commission as part of the Copernicus program (http://copernicus.eu).

The authors would like to acknowledge the provision of Global Atmosphere Watch and European Monitoring and Evaluation Programme hourly station data (CO and O₃) and aerosol data from the World Data Centre of Greenhouse Gases (WDCGG), from the EBAS database, and from the Norwegian Institute for Air Research (NILU) database. The authors also thank the providers of Near-real-time data to the Copernicus Atmosphere Monitoring Service validation team, namely, Institute of Atmospheric Sciences and Climate (ISAC) of the Italian National Research Council (CNR), South African Weather Service, the University of York and National Centre for Atmospheric Science (NCAS, AMF, UK), the Instituto Nacional de Meteorologia e Geofisica (INMG, Cape Verde), National Air Pollution Monitoring Network (NABEL; Federal Office for the Environment FOEN and Swiss Federal Laboratories for Materials Testing and Research EMPA), Japan Meteorological Agency (JMA), Alfred Wegener Institute, Umweltbundesamt (Austria), National Meteorological Service (Argentina), and Umweltbundesamt (UBA, Germany).

The authors wish to acknowledge the provision of ozonesonde data by the World Ozone and Ultraviolet Radiation Data Centre established at EC in Toronto (http://woudc.org), by the Data Host Facility of the Network for the Detection of Atmospheric Composition Change established at NOAA (http://ndacc.org), by the Norwegian Institute for Air Research NILU (https://www.nilu.no/en/), and by the National Aeronautics and Space Administration NASA (https://tropo.gsfc.nasa.gov/shadoz/).

Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) lv2 ozone profiles were provided by ESA’s Climate Change Initiative Program (Ozone_cci). SCIAMACHY lv1 radiances were provided to Institut für Umweltphysik Universität Bremen (IUP-UB) by ESA through DLR/DFD. GOME-2 lv1 radiances were provided to IUP-UB by EUMETSAT. The authors thank Quentin Errera for providing validation tools originally built for the BASCOE Data Assimilation System.

ACE-FTS (http://www.ace.uwaterloo.ca/data.php) is an instrument of the Canadian-led Atmospheric Chemistry Experiment (also known as SCISAT) that is mainly supported by the CSA and NSERC. The authors acknowledge ESA for provision of the MIPAS and acknowledge the Microwave Limb Sounder (https://mls.jpl.nasa.gov/data/overview.php) mission scientists and associated NASA personnel for the production of the data used in this study. The data used in this publication were obtained as part of the Network for the Detection of Atmospheric Composition Change and are publicly available (see http://www.ndacc.org).

No specific funding for this article.

The authors have declared that no competing interests exist.

Contributed to conception and design: AW, YB, AMB, SC, YC, QE, KMH, JK, BL, NS.

Contributed to acquisition of data: JF, AI, AR, VT.

Contributed to analysis and interpretation of data: AW, YB, AMB, SC, YC, KMH, JK, BL, NS, CZ.

Drafted and/or revised the article: AW, SB, YB, AMB, GB, SC, YC, QE, HE, JE, AI, KMH, JK, ZK, BL, MS, NS.

Approved the submitted version for publication: AW, YB, AMB, GB, SC, YC, QE, HE, JF, KMH, AI, JK, BL, AR, NS, VT, CZ.

How to cite this article: Wagner, A, Bennouna, Y, Blechschmidt, A-M, Brasseur, G, Chabrillat, S, Christophe, Y, Errera, Q, Eskes, H, Flemming, J, Hansen, KM, Inness, A, Kapsomenakis, J, Langerock, B, Richter, A, Sudarchikova, N, Thouret, V, Zerefos, C. 2021. Comprehensive evaluation of the Copernicus Atmosphere Monitoring Service (CAMS) reanalysis against independent observations: Reactive gases. Elementa: Science of the Anthropocene 9(1). DOI: https://doi.org/10.1525/elementa.2020.00171.

Domain Editor-in-Chief: Detlev Helmig, Boulder AIR LLC, Boulder, CO, USA

Knowledge Domain: Atmospheric Science