Arctic ice data conflict, example of software tool usage

Posted: December 10, 2015 by tchannon in methodology

Moderator Tim Channon writes: This article is unusual for the Talkshop. I may for personal reasons be leaving the Talkshop, a consequence of recent serious troubles, why I have been quiet. I hope this proves untrue.
I want to get some items out in the world for others. From my point of view leave a legacy. I am releasing the Synth software, done rapidly so none of this is neat or complete.

This is a simple example of usage with perhaps some current interest. No answers are intended for more the whole idea is about gaining insight by you playing around with data and ideas.

Image

Figure 1

According to published datasets the characteristics of sea ice changed around 2006/7. Dataset used here is NSIDC/NOAA D02135 monthly. ( if you try and decode the published data, read the notes, has offsets which need compensation, do not use as is)

Based on experience I think this change is in the data gathering, perhaps geometry, not actual ice change. In the past in various articles I’ve mentioned some of the dubious practices.

Only tools used here are text editor, Openoffice Calc for post processing and the Synth software. Done under W8.1 but could be just as well under Linux.You will be able to clone this, doesn’t take rocket science skills with a PC, hopefully discover more.Image

Figure 2

This is what happens if the tool is told to remove annual.
fs=12 annual=1 interp=nk origin=1978.875 temp.txt

temp is the casual filename I’ve used for the data, a single column of ascii in a text file. A couple of months of data are missing, marked as nk (software ignores these). There was satellite failure yet some published dataset have miraculously filled the gap, there is no data. (older post on my blog mentions the details)

Output files tem.met and temp.rem are plotted.

Notice the obvious problem 2007 onwards, why?

As a simple ploy I fed in subsets of the data, 1978.875 .. 2006.875 and 2006.875 to latest. Using the exact same start date and given the accuracy of the software computing the annual change is trivial. For annual the periods are literally locked.
fs=12 annual=1 interp=nk origin=1978.875 temp.txt (data excludes newest)
fs=12 annual=1 interp=nk origin=2006.875 temp1.txt

Putting this together, relatively complicated to do, easy once you know how.

Image

Figure 3, this may or may not be accurate, look for yourself

Figure 1 shows annual for early and late, using both produces figure 3. The software give the amplitudes and phases.

How? The output .asc files where imported, time series created, result combined and subtracted from original data, producing the remainder. One twist. the offset is different so the late simulation uses the offset from the early data, otherwise there is a step.

Valid? Who knows. Why the data change? Needs finding out. I’ve posted I give up on sea ice after very detail analysis of daily data, been working on this stuff for years. Concluded the data is very dubious. Linear trend is probably human error.
Now a lot of data is being hidden from public. Why?

Can more be done with the above?

Of course. Lets take the created data used for figure 3 as the input for more processing, making sure the invalid data is marked appropriately.(the use of an irregular timebase, extra data column could be used instead but isn’t)

fs=12 gens=3 interp=nk origin=1978.875 temp3.txt

This time in normal mode, not annual. Use 3 terms (generators).

Image

Figure 4, matched, output allowed to extend to 2020.

There is no party trick here, you will be able to clone these results, if you think there is cheating all the C sources are provided. Here the software is stepping outside of the capability of DFT, data is too short in time. This software will match shape, even a tiny part of a longer curve. This result is ambiguous, not to be trusted, why you have a brain but if this leads to insight, good.

Is there more periodic kind of detail in the data? Not in my opinion of any use, you could look. A rough rule is KISS, keep it simple stupid. (there is no definite limit on the term count, speed will limit things first, I’ve tried 100+, program uses little memory, mostly a few megabytes)

Now lets do something which might open some eyes.

Image

Figure 5. Output timebase has been altered. Very easy to do this.

Changed cell C17 to 1950 and C16 to 4, output start date and 4 samples a year, aliasing is not a problem in this case. Data from figure 3 is plotted to its own separate timescale.

Here I am illustrating the effect of inferring waves. The waves periods are irrationally related so the phase varies, hence might add or subtract, leading to the greater shape complexity.

The waves here are approximations.

There is a long period wave and a common one in climatic data, 21 years or so. In my opinion probably of solar magnetic origin.

Following is a guess.

Image

Figure 6, plot from a different simple example, 3 term again.

Solar heat enters mostly in the middle latitudes, flows north and south to polar regions.

If we flip this or the ice data upside down there is roughly speaking a fit after a delay of roughly a decade.

I expect a refined work could be done, this example is supposed to be leading to ideas, insight, not formal results.

Please ask questions whilst you can.

Putting this in the published examples.

Software: Talkshop top menu, Portal, Tim’s software

The texts, documents are poorly written, I wish this was not so. Contribute and help if you can.

Stupid questions are fine, in private if you need to. No question is stupid, nor the person asking. Maybe I am in yoda mode🙂

Post by Tim

Comments
  1. tchannon says:

    Tim comments, asking too much on why is inappropriate in public. Lets get the software so others can use it and compile it. Improve it.

    I will try and produce some more examples, on the less obvious details.

  2. frankpwhite says:

    How much data is missing and for which years and months? Any idea how it went missing? Instrument failure?

  3. Scute says:

    Tim

    “I hope this proves untrue.”

    I hope so too. I’ve always enjoyed getting an unexpected late night post from you, like this one. I might not comment much but I’m around, taking it in- all clever stuff, all food for thought.

  4. Paul Vaughan says:

    “2007 onwards, why?”

    same answer as before (recall that we’ve been over this before, but I know many did miss it):

    Lack of multi-year ice caused a nonlinear annual cycle change, but anomalies are still relative to the longer-term climatology (average annual cycle).

    This is well-known and not controversial.

    It’s easy enough to get around this with a nonlinear toolkit (why I always laugh so hard at WE’s decompositions at wuwt …and even harder at the obsequious clowns who fall for them).

    Fun Flashback: Does everyone remember Livina & Lenton? …whose sea ice stats suggested: swim inside sheets of ice (insert grandiose symphony music for effect HERE) and skate (or snowmobile) on open water …because that makes sense (/sarc).. Good times. (The fun of brain-dead (common-sense-check-free) anomaly-think. L&L took gold — no contest)

  5. Paul Vaughan says:

    I see Jupiter-aliasing of the terrestrial year in “gen[0] = 86.3531” here:

    What timing!

    I’ve just outlined that over here:
    https://tallbloke.wordpress.com/suggestions-16/comment-page-1/#comment-110840

    Dear suspicious observers:

    Welcome to your clue.

  6. tchannon says:

    frankpwhite,

    In the context of monthly, two months when there was sudden satellite instrument failure and no backup.

    1987.875 9.66 11.5 11.12
    1987.958 nk nk nk nk
    1988.042 nk nk nk nk
    1988.125 13.5 15.65 1.85 2.83

    I’ve probably written this up, too many articles to quickly find anything. (could be either blog, 1000 articles from me)

    For starters the following gives some idea of the satellite mishmash but the whole story is much more complicated. This too I’ve shown links somewhere.


    From https://daedalearth.wordpress.com/2013/07/12/dubious-arctic-ice-data-does-not-support-the-official-storyline/

    The above suggests the annual cycle change is also about satellite failure. Different orbit, probably not fully compensated.
    Story here over NASA Stereo (sea ice 2day / daily data), a sick story. Maybe I’ll tell on this later.

  7. tchannon says:

    Scute, hope so indeed.

  8. tchannon says:

    Paul, with such short data put a huge uncertainty on a long figure.

  9. Graeme No.3 says:

    Tim,
    “I hope this proves untrue”. So do I. My best wishes for you in the future.

  10. ren says:

    The temperature in the stratosphere is very low. Strong polar vortex.

  11. ren says:

    Frozen Bering Strait. Very low temperatures in Alaska.

  12. Paul Vaughan says:

    Tim (December 10, 2015 at 8:11 am ) wrote:
    “Paul, with such short data put a huge uncertainty on a long figure.”

    Agree on both points — that and TB’s wise to leverage every opportunity to pry back open minds swayed by dark agents closing the door on truth, so orthogonal (ERSST EOF 3 vs. 4) polar-tropical aliasing insights aside here’s what’s multidecadally coherent:

    _Sun_
    _Climate_
    Of course those asymmetric multidecadal aberrations are on this backbone: _101_

    Marcia Wyatt stormed onto the stage a few years ago with fractional differintegral awareness.
    …but then she lost the plot. (China will help us correct USA.)

    EOF3 polar-tropical orbital aliasing vs. EOF4 orthogonal sun-climate differintegral asymmetry

    agree on both points

  13. michael hart says:

    I hope your troubles can be resolved satisfactorily, Tim.
    I would certainly miss your input of competent technical sanity in the public arena.

  14. Nicola Scafetta has a couple of papers on topics discussed here.

    “Testing an astronomically based decadal-scale empirical harmonic climate model versus the general circulation climate models.” Journal of Atmospheric and Solar-Terrestrial Physics 80 (2012): 124-137. URL: http://arxiv.org/pdf/1201.1301.pdf

    Multi-scale harmonic model for solar and climate cyclical variation throughout the Holocene based on Jupiter–Saturn tidal frequencies plus the 11-year solar dynamo cycle. Journal of Atmospheric and Solar-Terrestrial Physics, 80, 296-311 (2012) URL: http://arxiv.org/pdf/1203.4143.pdf

  15. tchannon says:

    Author: someone reported the zip of executables would not download. Why so is mysterious, WordPress foibles possibly: it could see the file as known but refused. Experimentally change web page filename to something non-existent, WordPress gives page 404. O k a y, this is silly.

    I uploaded another copy with a slightly different filename. Seems to work, at least it did.

    See next comment.

  16. tchannon says:

    Author: Sorry about this, I’m not at present very with it. Problem turns out to be WordPress refusing to serve the file. I’ve worked around this by locking the zip, details on the download page.
    I wouldn’t normally do a workaround like this in public but personal circumstances give me no time to find people to help set up a public software distribution system.
    I’ve done the best I can to make compiling from sources turnkey for one compiler which will run on XP through W10 but this means installing the compiler. (80MB download, c version w32, has its own sources with libraries, complete system)
    It is then double click on a batch file.

    Roger: If WordPress get upset please blame me, ask them to contact me.

  17. Paul Vaughan says:

    Maybe long-forgotten (?) …so a little timely review on terrestrial-surface barycentric J-S versus terrestrial-core heliocentric JEV and how they tie exactly together:

    SEV = 9.007246722
    JEV = 11.06964992

    2*(9.007246722) = 18.01449344
    2*(11.06964992) = 22.13929985 (Hale)

    harmonic mean:
    (22.13929985)*(18.01449344) / ( (22.13929985 + 18.01449344) / 2 ) = 19.86503587

    (29.447498)*(11.862615) / (29.447498 – 11.862615) = 19.86503587 = J-S

    It’s clear there remains some kind of a miscommunication hang-up on the whole bidecadal thing. (The nature of the misunderstanding isn’t clear.)
    It’s not either-or but rather a core-surface heliocentric-barycentric geometry distinction.

  18. tchannon says:

    Work with Synth and solar data is unlikely to produce much in the way of useful results, clues maybe.

    I had a vague intention of extending the software so that useful interconnections could be done and more external data fed in, such as modulation.

    What I had in mind is some kind of switch (routing) matrix. This is a major development. I never found the time or inclination. An additional problem is how to interact with the matrix, a UI problem. This is where an interactive GUI would help.

    There is no reason why modulation could not be several internal factors, mixed.

    How would I do this?

    First is obviously getting a suitable scheme in mind.

    I think next develop a GUI and get it fully working with what is.

    Then add the GUI interaction for the scheme.

    Then deal with the changes to Synth, GUI would help, make debug easier.

    This would take months.

  19. tchannon says:

    No surprise to me there is some kind of problem with the rapidly prepared release software.

    At the moment I am baffled. Right now restoring the entire source etc. directory from backup so I can compare with old stable. Local SVN doesn’t give enough info.
    gens= on the cmd line is being ignored yet I thought it was working. Kind of thing hasn’t been changed in years.

  20. tchannon says:

    Ho ho, hole in own foot. A time shortcut on an horrendously long process caught me, failed to notice it was told to ignore what I put on the cmd line.

    See if I can leave the computer all day crunching that one. Hundreds of files. If this works a 60MB archive will get uploaded. Won’t be spot on. No time.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s