Skip to content

Meteorological Data Assessment Process

The assessment of the accuracy of the MERRA-2 parameters is based upon comparisons with observations from the corresponding parameters reported to the National Center for Environmental Information (NCEI). Hourly assessments are compared against the Integrated Surface Database (ISD) while daily average assessments are compared against the Global Summary of the Day (GSOD).

In the following sections, scatter plots associated with each meteorological parameter provide a visual indication of the agreement between the MERRA-2 values and surface observations. A more quantitative assessment of this agreement is given by the statistical parameters enumerated on each scatter plot and summarized in tables. These statistics between the model estimates and surface observations include the slope, intercept and Pearson correlation coefficient associated with a linear least squares regression fit, the Mean Bias Error (MBE) and Root Mean Square Error (RMSE).

Hourly Assessment

The assessment of the accuracy of the hourly MERRA-2 parameters is based upon the surface observations reported in the NCEI Integrated Surface Database (ISD). After quality control codes available with the database were applied to the data, only the reports closest to the top of the hour are retained for potential validation. The next qualification is that there must be a minimum of 2,200 hourly reports in any given year (about 25%) for a site year to be considered. Surface sites are then selected within half-degree grid boxes for each available surface types (18 total including forest types, shrublands, grass and crop lands, permanent snow or ice, etc.) while also having as much data as possible. This selection process allows for a greater number of surface sites in Africa, South America, Asia, and Eastern Europe to be evaluated against MERRA-2. The years of interest for hourly validation are from 2001 through 2019.

NCEI ISD Surface Stations

ISD Station Map

Hourly Surface Validation Summary

Presented here are the results of comparing the original MERRA-2 hourly meteorological parameters to corresponding surface observations from 2,606 stations reported in the NOAA/NCEI Integrated Surface Data files. The Slope, Intercept, and R-squared (Rsqrd) listed in this table are associated with a linear least squares fit of hourly observations at each surface station to the corresponding MERRA-2 values for the overlapping grid cell for 2001 through 2019. The Mean Bias Error (MBE), Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) represent mean values over the comparison years.

Parameter MBE MAE RMSE Slope Intercept Rsqrd
Temperature (°C) -0.58 2.40 3.30 0.96 1.03 0.95
Dew Point Temperature (°C) 0.04 2.40 3.48 1.01 0.01 0.93
Precipitation (mm) 0.011 -0.58 0.95 0.79 0.04 0.10
Elevation Corrected Atmospheric Pressure (hPa) 0.17 1.07 1.94 1.01 -11.17 1.00
Wind Speed (m/s) 0.53 1.82 2.47 0.73 0.57 0.40
  • The parameters are evaluated at the hourly temporal level.
  • The height above the surface over the entire grid box area is 2m for each parameter, except for the wind which is estimated at 10m.

Daily Average Assessment

Comparison statistics for the daily assessment are based upon comparisons of the given parameter for every third year beginning in 1981 through 2014, (e.g. 1981, 1984, 1987, etc.).

The daily observational data reported in the NCEI GSOD files are hourly observations from globally distributed ground stations with observations typically beginning at 0Z. For the analysis reported herein, the daily values were derived from the hourly observations filtered by an "85%" selection criteria applied to the observations reported for each station. Namely, only data from NCEI stations reporting 85% or greater of the possible 1-hourly observations per day and 85% or greater of the possible days per month were used to determine the daily average value of the given parameter.

2004 NCEI Surface Stations Meeting 85% Selection Criteria

NCEI Stations A

2012 NCEI Surface Stations Meeting 85% Selection Criteria

NCEI Stations B

The maps above show the global distribution of the surface stations in the NCEI data files for 2004 and 2012 after applying our 85% selection criteria. There are 2704 surface stations in 2004 and 4731 in 2012 with the majority located in the northern hemisphere. As noted above, the MERRA-2 values represent the average value over their original source spatial resolution while the NCEI values are local observations.

Additionally, along the right side of each scatter plot is a color bar providing a measure of the distribution of the NCEI:MERRA-2 data pairs. The column on the left side of the vertical color bar gives the number of data pairs per parameter bin associated with each color. The parameter bin size for the temperature values is 1°C; for pressure 1mb; for precipitation accumulation 5mm; and for specific humidity 1g/kg. The column on the right side of color bar gives the percentage of total number of MERRA-2 ground site data pairs associated with each color.

Daily Average Surface Validation Summary

Presented here are the results of comparing the re-gridded MERRA-2 daily meteorological parameters to corresponding surface observation reported in the National Center for Environmental Information (NCEI) (formerly National Climatic Data Center) Global Summary of the Day (GSOD) files. The Slope, Intercept, and R-squared (Rsqrd) listed in this table are associated with a linear least squares fit of daily observations at each surface station to the corresponding MERRA-2 values for the overlapping grid cell for every 3rd year from 1981 – 2014 (e.g. 1981, 1984, 1987, etc.). The Mean Bias Error (MBE) and Root Mean Square Error (RMSE) represent mean values over the comparison years.

Parameter MBE RMSE Slope Intercept Rsqrd
Average Temperature (°C) -0.52 2.29 1.00 -0.48 0.96
Minimum Temperature (°C) 0.50 3.05 0.97 0.73 0.93
Maximum Temperature (°C) -1.38 3.16 0.98 -1.01 0.95
Relative Humidity (%) 4.44 12.06 0.78 19.88 0.61
Dew Point Temperature (°C) 0.37 2.53 0.95 0.69 0.95
Precipitation (mm/day) 0.46 6.81 0.36 1.71 0.23
Specific Humidity (g/kg) 0.01 1.05 0.93 0.49 0.95
Uncorrected Atmospheric Pressure (hPa) -8.59 24.40 0.95 40.97 0.81
Elevation Corrected Atmospheric Pressure (mb) 0.03 2.82 1.00 1.08 1.00
Wind Speed (m/s) 0.37 1.65 0.73 1.36 0.55
  • The parameters are evaluated at the daily temporal level.
  • The height above the surface over the entire grid box area is 2m for each parameter, except for the wind which is estimated at 10m.