So you like reanalysis, the 3 var trick

Posted: April 10, 2015 by tchannon in climate, Forecasting, humour, Uncertainty

NCAR Research Data Archive Blog

1938 atmospheric river that caused the Los Angeles flood of 1938
At least one person asked why I used a reanalysis that does not assimilate satellite water vapor data to study an atmospheric river (AR) event.

That’s a good question because the NOAA/CIRES Twentieth Century Global Reanalysis Version 2c (20thCR V2c) only ingests three things: surface pressure, sea ice coverage and sea surface temperature. The rest of the analysis is generated by the physical models of NOAA’s Global Forecast System (GFS).

I find this disturbing.

Cited continues…

The short answer is that 20thCR V2c extends all the way back to 1851, which means you can compare storm intensity between storms, including events that occurred before the modern satellite era.

What was it, one land thermometer in the southern hemisphere during 1850? Seen the SST data source maps, even today these are very sparse.
Total counts of grid cells data points for decades can be accessed some way down a blog post from last year here.

Worrying. I’d like to see the results of validation checks done independently by unfriendly outsiders, auditors from hell.

Dig a bit deeper

What is the 20th Century Reanalysis Project?

The goal of this project is to use a Kalman filter-based technique to produce a global tropospheric circulation dataset at four-times-daily resolution back to 1851. The only dataset available for the early 20th century consists of error-ridden hand-drawn analyses of the mean sea level pressure field over the Northern Hemisphere. Modern data assimilation systems have the potential to improve upon these maps, but prior to 1948, few digitized upper-air sounding observations are available for such a reanalysis. The global tropospheric circulation dataset will provide an important validation check on the climate models used to make 21st century climate projections in the upcoming Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC).

Validation check on a model?

Umm… from top “The rest of the analysis is generated by the physical models of NOAA’s Global Forecast System (GFS).”

Oh I see, a physical model, wassat then, made of wood?

So what do the British establishment have to say?

Centre for Environmental Data Archival
Science and Technoloy Facilities Council
National Environment Research Council (NERC)


The 20th Century Reanalysis (Version 2) Dataset was produced by the National Oceanic and Atmospheric Administration (NOAA) and University of Colorado’s Cooperative Institute for Research in Environmental Sciences (CIRES), members of the international Twentieth Century Reanalysis (20CR) project. The data were produced through international cooperation under the auspices of the international Atmospheric Circulation Reconstructions over the Earth (ACRE) initiative, and working groups of the Global Climate Observing System (GCOS) and the World Climate Research Program (WCRP). The 20CR project produced comprehensive global atmospheric circulation data spanning from late 19th century to the early 21st century, having only assimilated surface synoptic pressure observations and using observed monthly sea-surface temperature and sea-ice distributions from the HadISST dataset, courtesy of the Met Office Hadley Centre, as boundary conditions. Analyses every 6 hours on a 2 degree grid were produced to give the most likely state of the atmosphere based on a 56 ensemble member runs. Means, spreads and all ensemble members for each time step are available in the dataset archived at the BADC. The current 20CR output produced spans from 1871 to present with the BADC copy being updated when possible.

And provides data it seems no-one can read if the “tearing my hair out” from programmer space is correct. I couldn’t find the file format documentation either. Life is too short.

New blog… and from there Dame Julia “The science is well ahead of our ability to implement it.”

Post by Tim

  1. TerryS says:

    So they are using the output of models to validate the output of models.

    Should give them plenty of scope to prove themselves right and ignore reality.

    Utility for extracting data from GRIB files:
    Documentation for GRIB:

  2. tchannon says:

    No no… physical models and unphysical models which everyone knows are different. 🙂

    I don’t mind being wrong when the outcome is good.
    C source eh? Drop into new directory, look for Unixisms, none, fire up the C compiler I use, worked, surely not. Even compiles ansi without any warnings. And test run, it seems to be live.

    Have to go and get some data files later.

    Thank you TerryS.

  3. tchannon says:

    Wrote too soon, grab data, no dice. Need to decode nul nul nul GFS SIG

    There is supposed to be FORTRAN 90 source somewhere. Can do F. but not that version.

    Actually doing anything with the data is improbable, GB of the stuff.

    Another dig yielded up this which can read the files for its own purposes.

  4. JKrob says:


    Geee, you write some serious gibberish there 😉

    As for processing/viewing GRIB data, I prefer WINGRIDDS myself for some reason…Hmmmm

    As for reanalysis of the historic atmosphere, I don’t have that much of a problem post-1940’s but to try & do it before there was any 3-d analysis (RAOBs) is laughable.


  5. tchannon says:

    Cross purposes, I didn’t say GRIB data, TerryS did.
    GRIB is widely known.

    Does it handle GFS SIG ?

  6. gymnosperm says:

    ” use a Kalman filter-based technique to produce a global tropospheric circulation dataset at four-times-daily resolution back to 1851″

    Very clear-cut case of having their head up their maths. ™

    Even by quantum standards, one of the few things mathematics cannot do is manufacture data. It can manufacture predictions, hypotheses to be tested, but to call these data is absolutely wrong.

  7. ren says:

    Stratospheric observations during the IPY
    The first part of winter 2008-2009 has been characterized
    by a stable and cold polar vortex which allowed
    the persistent formation of PSC particles. In
    mid-January of 2009, however, the most intense sudden
    stratospheric warming (SSW) ever observed occurred
    [Manney et al. 2009, Di Biagio et al. 2010]. SSWs strongly
    affect the dynamics and thermal structure of the Arctic
    stratosphere causing the breakdown of the eastward
    winter circulation, the build up of a westward circulation,
    and the reversal of the latitudinal temperature
    gradient. As the 2009 SSW developed, the stratopause
    lowered, the mean zonal circulation reversed, and ultimately
    the polar vortex in the lower stratosphere split
    in two (see Figure 12).
    Figure 13 shows the lidar backscatter ratio (both
    parallel and cross polarized components) and the depolarization
    ratio on January 17 and 18, when the lidar
    detected a PSC layer. The layer extended between 16
    and 23 km altitude on January 17 and then it moved to
    slightly lower altitudes (16-20 km) on January 18. The
    observed PSCs can be classified as type Ia due to their
    low backscatter ratio and moderate depolarization values
    [e.g., Browell et al. 1990, Toon et al. 1990]. This indicates
    that their particles are liquid and composed of
    nitric acid trihydrate (NAT), as also suggested by the
    temperature values at the altitudes of the observed PSC
    layer (not shown) which are below the NAT threshold.
    Lidar and GBMS measurements at Thule observed
    the occurrence of the major SSW, sampling air inside
    the polar vortex at first and following the propagation
    of the SSW down to the lower stratosphere afterwards.
    The contour plots in Figures 14, 15, and 16 show the
    changes of the atmospheric chemical composition over
    Thule and temperature associated with the SSW. Figure
    14 shows a sudden increase in N2O mixing ratio
    (mr) which occurred on January 24 at around 35 km altitude
    and over the whole stratosphere between days
    26 and 28. At higher levels, the vortex splitting and the
    vortex edge transit over Thule was marked by a rapid
    decrease in CO mr. CO data (not shown) indicate that
    in the upper stratosphere (45-50 km) the vortex broke
    up over Thule on January 19-20.

  8. ren says:

    Pressure spikes above the polar circle north and south in the winter, 2009.

  9. ren says:

    It can be seen that the effect of solar activity on the polar vortex is extremely complex. If solar activity will fall scientists finally wake up.

  10. Bob Weber says:

    Let’s hope so ren, but I wouldn’t count on it. You can lead a horse to water, but you can’t make them think…

  11. ren says:

    Bob these studies in Greenland show that if you want to, you can. But it needs a comprehensive studies.

  12. tchannon says:

    ren and others,

    Are you aware the pretty plots are based on “reanalysis”, are beyond actual data. Reality is messy.

  13. JKrob says:


    (here we go again…) like I said before, I agree the data presented of the 3D atmosphere before there was 3-D observations is, of course, impossible. However, the plots provided by ren are from 2009 & 2015…fairly recent stuff.

    The whole reanalysis project is an attempt to show a global observation of the 3-D atmosphere. Of course, it is not perfect. The observation system does not have the ability to sample the 3-D atmosphere for every square mile. But with observations from platforms like METAR, buoys, ACARS and satellite sounding obs being fairly close to RAOB obs, it is not too bad…but it is not perfect. It (data used in reanalysis) is the same data ‘snapshot’ used to initialize the forecast models at that time.

    You seem to think it is unusable. If you do not agree with the data provided, and have observation data from that time of better quality, please, feel free to show the ‘correct reality’ data plot with the data in error highlited.

  14. dscott says:

    You know tallbloke you might want to work on a productive angle in bringing bad papers to light to force retractions:

  15. ren says:

    Forecasts stratospheric (decomposition of ozone) are quite accurate. Pressure also above the the polar circle. This can be linked to the circulation in the troposphere.

  16. tchannon says:

    ren, don’t worry too much about what I wrote which not aimed at you.

    Reminding people to hold a degree of caution over computer created data is the point.