Roger Andrews: IPCC AR5 Climate Models – Another Bust

Posted: February 1, 2013 by tallbloke in Analysis, data, Measurement, methodology, Natural Variation, Ocean dynamics
Tags: , , , ,

My thanks to Roger Andrews for this succinct examination of the model-data comparisons from the leaked draft of IPCC AR5. He is absolutely right to draw attention to how poorly the models do in terms of tracking changes in sea surface temperature (SST). This metric is indicative of ocean heat content (OHC), which is the more important indicator of global change (though poorly understood and measured), because the ocean is the climate object with nearly all the heat capacity.

Roger Andrews – 1-2-2013

In each of its Assessment Reports the IPCC presents a global model-versus-observations comparison to show how good its climate models are. Its latest offering, reproduced below, appears in Figure TS-7(a) of the AR5 Draft. The Figure compares four “global mean temperature” series (HadCRUT4, GISS, NCDC and Japan Meteorological Agency) against the CMIP3 and CMIP5 climate models. At a glance, which is about all the IPCC’s average Policymaker will give it, the fit looks pretty good.


But when we dig deeper – and we don’t have to dig very far – we find that the fit is actually pretty bad.

The IPCC graph conceals this by comparing area-weighted averages of “land” air temperature and “ocean” SST observations with climate model simulations that have been area-weighted in the same way. It doesn’t compare observed land air temperatures directly with modeled land air temperatures or observed SSTs directly with modeled SSTs, which is the way model performance should be checked. So to rectify this omission I’m showing these comparisons here.

First some brief notes on data and presentation. The IPCC’s four “mean global temperature” series are all more or less the same, so to simplify things I compare the models only with the CRUTEM4 land air temperature and HadSST3 ocean SST series that HadCRUT4 is based on. I downloaded the HadSST3 data from the Met Office website (here) and the CRUTEM4 data (version from the KNMI Climate Explorer website (here). For the model simulations I use the CMIP5 multi-model means, also downloaded from KNMI, which I was able to confirm are indeed what the IPCC used to construct the CMIP5 plot on its Figure. I don’t take the comparisons back before 1900 because model-observed mismatches before then could be a result of bad data, and to avoid visually-biasing the comparisons I plot temperatures as anomalies relative to 1961-1990 means, the same baseline the IPCC uses.

Here now is the CRUTEM4 land air temperature series plotted against the CMIP5 land air temperature (tas) multi-model means. The fit isn’t perfect, but it’s close enough for climate work:


And here is the HadSST3 ocean SST series plotted against the CMIP5 multi-model SST (tos) means: The fit after 1950 isn’t great and before 1950 it’s a bust:


And the bust could be a lot larger. There are reasons to believe that HadSST3 has been “corrected” to the point where any resemblance between it and reality before 1950 is coincidental. At the same time there are reasons to believe that the unadjusted ICOADS SST series from which HadSST3 is derived, although itself hardly a model of observational excellence, may be a lot closer to the truth. Here’s what we get when we replot the comparison using ICOADS instead of HadSST3:



To sum up, the IPCC’s anthropogenically-forced CMIP5 climate models can broadly explain observed air temperature trends on land but fall woefully short of explaining observed SST trends in the oceans, where over 99% of the heat in the atmosphere and oceans is stored. Others may disagree, but I think this invalidates the models. Again.

  1. oldbrew says:

    This may be relevant.

    ‘When warming is due to increased greenhouse gases, the gradient of sea surface temperature (SST) across the tropical Pacific weakens, but when it is due to increased solar radiation, the gradient increases.’

  2. Roger Andrews says:


    I didn’t say anything about the CMIP3 models because the CMIP3 models on Climate Explorer clearly aren’t what the IPCC used to plot its CMIP3 trace. But if we accept that these models are valid simulations CMIP3 becomes even more other-worldly than CMIP5. Even the land air fit is a bust.

  3. Bob Tisdale says:

    Roger: I only presented the CMIP3 models as a reference in the first post linked above.


  4. tchannon says:

    Good, you kept the message simple. That is the hardest part.

  5. Vincent Gray says:

    I would argue that chalk can be shown to be the same as cheese if you fiddle it enough. The climate is controlled by convection and latent heat transfer, but I agree that in this case the sun is the more important.
    I noted this when I commented on AR5 and I am glad Roger has noted it without my having to leak it twice, He will find there are other examples such as methane.