Figure zero
Above figure is the least important. I hope it will become clear the red trace is solar magnetics, the sun flips fairly regularly and so does the magnetic field coupled to earth. Evidence earth sees these magnetics is shown…
This is a poster from the AGU Fall Meeting (December 2012, San Francisco, USA), shown by Mikhail V. Vokhmyanin and Dmitri I. Ponyavin from Saint Petersburg State University, Russia. They are describing extending a reconstruction of the solar magnetic field further back into the 19th century to 1844 using terrestrial field measurement data from Helsinki observatory.
In other words from earth surface measurements. Optical solar is a red herring. TSI (Total Solar Irradiance) is neither accurate nor the whole energy story.
Minimal analysis of Hadcrut 4 global
Note: in all these plots example data is Hadcrut 4 global monthly, Y axis assume change in C, not shown for visual simplification. This article is dealing in concepts.
 |
 |
| Figure 1 | Figure 2 |
|---|
Figure 1, published Hadcrut 4 global temperature estimate, figure 2 with an approximation using a least squares fit minimal Fourier decomposition as a function generator.
Fourier decomposition into four components, fixed offset and three pure sines (if Figure 4 looks wrong, is an optical illusion, data provided if you want to check)
 |
 |
| Figure 3 | Figure 4 |
|---|---|
 |
 |
| Figure 5 | Figure 6 |
| Now recompose the decomposition, literally a point by point sum of the components. See Figure 2, same trace. This is a computer optimised least squares fit to the dataset. |
|
| Figure 7 |
Figure 8
Figure 8 is included but is unlikely to mean anything to general readers but I will try and explain.
Sunspot data, optical change on the sun is visually in ~11 year periods but in fact the underlying solar change is magnetic, by eye a polarity change of sunspots is invisible, can’t be negative light.
The magnetic cycle runs at half the speed, ~22 years. Decoding an encoded “signal”, in this case restoring the upside down sunspot periods can be done by a polarity switch, flip the data upset down or not. (=SIGN(ref to delayed) * data)
Here I am using the ~21 year signal automatically extracted from Hadcrut 4 data [A] as the switch, flipping at the zero crossing of the sine wave, but with a delay, is close to quadrature if that means anything to you.
If this is chance it is astounding it makes a good job of a difficult to do task. Curiously there seems a change 1850 … 1880 where the dataset mismatches the model.
Much more detail investigation is needed.
A). software turnkey, no human override, but to be fair I wrote the code which is stable general usage so you’ll have to trust me there is no cheating. The only guidance was choosing how many decomposition factors, gens=3, the rest is dataset driven.
Cross check for sanity
|
|
| Figure 9 |
|---|
The Hadcrut 4 monthly data has been low pass filtered at about 17 years. This compares well with the model showing there is no large component which has been omitted. Slight flattening at the ends of the filtered data is present and caused by end correction failing, not serious.
The obvious mismatch 1850 to 1860 is just that, a mismatch, model failure, however, the temperature data back then is very uncertain, curiously also what the sun was doing, circa 1870 is frequently mentioned in both the news and literature over magnetic storms. The infamous Carrington flare was 1859.
As a further check
Figure 10
Figure 10 is a windowed chirp z transform of data.
Residual is spectra after model subtraction.
Filtered shows the effect of the low pass, the accepted by the filter is the plotted.
Fourier model contains the causal of the difference between the blue and red traces, blue trace is about 10x larger signal. The filter has rejected the fast “noise” to the left and accepted everything below the red trace, shown Figure 10.
Statistics
Model and filtered, r2 = 0.99
More realistically, between the model and original dataset r2 = 0.73, more appropriate but meaning little to general readers RMSD = 0.15 a measure of least squares fit.
There are 1955 data points.
Reported sine periods are approximately 478y, 65y and 21y. See [1].
These parameters are far more accurately determined than can be done using DFT (Discrete Fourier Transform) even with interpolation, in part because of binning limits. (time resolution is limited by discrete data, values have to be put into “bins” and length of dataset, is ambiguous)
I cannot figure out how to put a figure on the match with solar.
Commentary
It is reasonable to assume the periodic data is about higher dimensionality[3] in the data as is common with natural data, unless there are definite causals.
Caution is also needed over dataset errors which I suspect are considerable given known severe sampling errors and the usage of simplistic statistical math to attempt automatic correction of known bad data)
Of the numbers only 21y has known candidate causals but the merit of fit is difficult to decide.
- Saros cycle, lunar, 18.6y
- dominant solar system gravitational ~19.5y
- solar magnetic cycle, irregular around 21y
The last is the most likely candidate with both solar and solar/terrestrial linkage. An actual mechanism is unknown regardless of various hypothesis.
A notional 60 year cycle is often reported and does seem to endure to a degree in long history. I am aware of no definite causal (eg. dispute the ephemeris related as aliasing), however this work suggests the two are linked, 3:1 periodic relationship and near 1:1 phase relationship, where 21y has a good phase relationship with solar data.
Circa 500y makes little sense, are reported but without a definite causal. Given the length of data long period factors are dubious. Perhaps notably there is no obvious curve matching human activity, no higher order deviation.
Figure 11 is a detail showing how two model terms add to produce the “no warming” from 2000 to date, in fact a curve. Given the tight data match the following can be done.
Figure 11
Figure 11, removing the long term leaves this curious construct of two terms.
Particularly interesting is the frequency relationship, very close to 3:1 (3.02), stranger still the phase relationship as though the two are tied together.
In nature odd harmonics are unusual, symmetric effects.
A narrow image visually reveals this is indeed a very strange effect for a real dataset which ought to be high H noise.
Figure 12
I find this shocking. The long period does not so relate but is also likely to be wrong. If there was 3 x 65 that would be in the DeVries region or solar repeat, goodness knows.
Worrying is that if the above is true it would have been long seen and clear in many datasets. It isn’t. At the Talkshop some do consider magnetics a likely link, eg. vukcevic, such as yesterday https://tallbloke.wordpress.com/2013/01/23/ex-nasa-scientists-the-science-is-clearly-not-settled/comment-page-1/#comment-41732
Decomposition wrong? No. If anyone knows otherwise please speak up. Also, independent confirmation I have not made mistakes would be nice.
Now I have to take the heat for sticking my neck out.
Errata
Figure number 9 onwards corrected (reported by Tallbloke).
1. Data here and here as .ods in .zip, ask if you need help. [pending, need to produce non-work version]
First file is source for most plots. Second file has been split off and is live for investigation of solar linkage (source for Figure 8).
Everything is in there but no filter software or dataset analysis software. Spreadsheet is used for convenient and portable post processing.
2. Underlying data http://www.metoffice.gov.uk/hadobs/hadcrut4/
3. Hurst exponent considerably different from 0.5 http://iahs.info/hsj/470/hysj_47_04_0573.pdf
Posted by Tim Channon
Tallbloke's Talkshop 
Sorry if there is garbled text, WordPress threw one of It’s format wobblies dumping everything tables included into a heap of no format, bad tags, missing tags, changing font etc. Getting it back is not nice. Let me know if anything is obviously wrong.
Looks like you’ve replicated Scafetta:
Great work Tim!
Minor nit: Two figure 9’s
The numbers do tie in well with the planetary hypothesis, but maybe you’d prefer to see what other suggestions come up first before we go into that?
I’ll only add that the Chiefio has a highly relevant post up today on Lunar Saros cycles
A very interesting post…
Thank you “for sticking my neck out”… don’t stop…
I really like the approach of just following the numbers…
Optical solar is a red herring.
TSI (Total Solar Irradiance) is neither accurate nor the whole energy story.
Totally agree with that.
My reservations are associated with the pedigree of Hadcrut 4:
It would be interesting to know whether the patterns are echoed in a “trusted temperature dataset”…but I suspect that concept is an oxymoron… so perhaps I will have to wait for some biographer to written: The Secret Temperature Measurements of an Eccentric Rural Clergyman… I live in hope… but I’m not holding my breath.
Roger A: The crucial difference between Tim and Scafetta here is that Tim has a long term underlying sine wave at 478 years whereas scafetta uses a quadratic fit. I’ll note in passing that 478 is around half of the period identified by P.A. Semi as the long cycle of Angular momentum in the solar system: 974 years.
TB: The difference is that Tim’s sine wave implies a natural cycle while Scafetta’s quadratic implies a long-term trend – maybe AGW, as Scafetta acknowledges. The problem is that over a 160-year observational period there’s no way of saying whether we’re looking at a trend or a 478-year cycle. Tim’s cycle and Scafetta’s quadratic plot pretty much on top of each other over this period.
Since there is an underlying warming over the last 300y , it may make more sense to add at least a linear increase of (as per Scaffeta) a quadratic.
This is also quite similar to what I did on HadSST3 over on J.Curry’s blog. Tim is aware of this, so I just put this in for other readers.
If you look at my triple cosine fits in figures 8,9,10 there, you will see that Hadley mostly destroyed the clear circa 160y periodicity with their “bias corrections”. Since 70% of HadCRUT4 is hadSST3 most of this will apply here also.
I also found a longer period in HadSST3 but it was not consistent between the time series analysis and looking at dT/dt , as it was for the unmodified ICOADS.
Fourier analysis also showed their processing had made significant changes to the whole frequency structure. This does not necessarily make the result “wrong” or worse than the original but means much of what you are fitting here is result of data manipulation as much as it is the original data.
Especially the early data (pre 1930) I showed that their processing removed the majority (>50%) of the climate variation from the data. And I’m talking about on a decadal scale, not the detail.
That is probably why you do not find a shorter century scale period. They removed it.
Ray Tomes suggested that the noise level in SSN data suggested it would be better for his CAT software to take the square root. I’d want a good excuse before doing that but the idea that sun spots are a physical response to the square of some quantity would explain the apparent rectification of the 21y cycle.
It would also suggest that any filtering should be done on the sqrt(SSN) , same for any cosine type analysis. The flattened sides of the SSN data look more like a sin^2 than a pure sine.
UPDATE: Two data files have been uploaded and linked.
“Exploring sqrt of sun spot number ICOADS and HadSST3”
Looks like there may be some mileage in sqrt(SSN) , again underlines my suspicions about Hadley’s post war poking. The spurious Folland and Parker 0.5 degree drop that was done in hadSST2 and reworked by being gently slid in instead of in one hit in hadSST3, but to the same effect.
This correlation looks interesting. Certainly matches early 1800’s freeze and most of 20th c.
Even a potential for a bit of AGW at the end. Something for all tastes.
Tim, I can confirm your findings. I did this kind of work last year for the preparations of our recent paper. The key cycles that have been self-selected by self-organizing modeling are: 21y, 22y, 67y, 68y (also 65y and 61y). The HADCRUT data are noisy and short so there is some uncertainty in the results, of course.
The long-term cycle, which is mistakably interpreted as linear or quadratic or what ever trend when looking at the past 150 years of temp records, only, is about 800 years long (not fixed, it is increasing) and is caused by the Earth Orbit Oscillation as one major cosmic climate driver. See also:
Another note on the sqrt plot, I labelled it arbitrary scale but it was not totally arbitrary. I just quickly gauged by eye what level of the SSN curve seemed to match warming and cooling periods in SST.
At a rough estimate I subtracted 6 , as can be seem from the legend. $2-6 means column 2 ( the sqrt SSN) had 6 subtracted.
Now 6^2=36. My quick and dirty “neutral” SSN . I recall someone else here estimated about 40.
@greg:
You say “Fourier analysis also showed their processing had made significant changes to the whole frequency structure. This does not necessarily make the result “wrong” or worse than the original but means much of what you are fitting here is result of data manipulation as much as it is the original data.”
To see how large these data manipulations were I reconstructed HadCRUT3 using the raw ICOADS SST data instead of the adjusted HadSST3 data (I assumed that no significant adjustments were applied to CRUTEM4) and subtracted it from HadCRUT3. The graph below shows the results. I don’t think there’s much doubt that these adjustments will have made significant changes to the frequency structure:
Whoops. for HadCRUT3 please read HadCRUT4. Altogether too many HadCRUTs.
It was Tallbloke who came up with the SSN=40 for the “neutral” SSN value. It seems to work.
knowledgeminer, and others
Ah, so you are finding (kind of or actual) doublets, which begs questions.
I did quickly check but nothing significant is there in the context of intent, which is minimal given things get messy if more is attempted. I’ll leave that as a subject for now.
I haven’t explained what happened leading to this post.
Over quite some time I’ve looked at these kinds of data but said little in public.
Quite recently I posted casual chit-chat which mentioned Hadcrut 4, data I have been ignoring. A few discussions took place some peripheral such as on my own rarely used blog.
I then decided that perhaps I ought to analyse Hadcrut 4 more seriously, not an interesting dataset. As an excuse to present banal findings a good move was turn it into a tutorial/explanation of a poorly understood subject, useful for later on when something important is going to be presented. (different subject)
Most of that was done, hence a lot of the content. It was only on wondering what to write about the various parts that I checked 21y against solar. Goose pimples time, not what I expected at all.
It was then a matter of reworking the article, with Rog doing rapid fire on posts, moving the goalposts, trying to fit in is fun. (grin, amused)
From my point of view the discovery of a definite sun/earth linkage, no hand waving, is critical for getting serious focus on solar linkage. If that can be firmed up a lot of other things will flow from that. Worth noting IPCC only accept TSI. This is hard to unearth out of datasets (okay so they are awful).
I’m in a position to do more than rough period relationships, phase is included, a whole different game. This is the critical area to do with planetary influence, it is exact, if not, why not. (won’t go into caveats etc)
A good instance of a problem is the sunspot ~1790 matter, a discontinuity which wreaks posits about planets affecting the sun, it has to be explained very solidly, otherwise, no dice, no acceptance in the general community.
The long term stuff is the subject of interminable arguments, we have opinions, does it get anywhere? I tend to keep off certain subjects.
It’s only science if independent controlled experiments are done, how reality is decided. Almost everything in climate is posit only, even where there could be experiment.
It’s interesting to note that sqrt(SSN) looks more like Lean et al that Svalgaard but does fall at the last decade , which was Svalgaard’s killer argument against Lean and others until now adding the mean SSN back in under SSN.
Since the noise pattern suggests what we are “measuring” with SSN may be the square of some physical quantity it may be easier to find a justification for doing sqrt than for adding in a (distorted and a badly) low-pass filtered SSN.
I’ve plotted sqrt(SSN) against Lean and Svalgaard TSI to see how it compares.
I have to an extent fiddled with the filter length to adjust sort term variance to be about right at the end but this looks interesting.
If this holds water, long term TSI has been far more variable than is currently thought and would go a long way to explaining temperature changes over the last 200-300 years.
I’ll have to look at this compared to actual measured TSI.
PS. That last plot aligns the “neutral” SSN to TSI=1360
greg said:
“If this holds water, long term TSI has been far more variable than is currently thought and would go a long way to explaining temperature changes over the last 200-300 years. ”
Confirmation of short term variability on that larger scale would be very helpful for a multitude of reasons.
If one assumes that atmospheric CO2 levels follow temperature and that temperature follows the amount of solar energy entering the oceans and that that in turn follows solar variations on such a scale then we can judge that the ice core CO2 records are but a pale reflection of real world short term atmospheric CO2 variability.
Greg, could you supply a verbal narrative to explain the nature and significance of sqrt(SSN) ?
Greg says: January 25, 2013 at 9:24 am
I’ve plotted sqrt(SSN) against Lean and Svalgaard TSI to see how it compares.
Amazing… that might just explain Eighteen Hundred and Froze to Death
Greg says: January 25, 2013 at 9:24 am
I’ve plotted sqrt(SSN) against Lean and Svalgaard TSI to see how it compares.
So the 5 w/m2 TSI change from the “old normal” of 1,366 to the “new normal” of 1,361 could be heralding a 200th anniversary re-run of the Eighteen Hundred and Froze to Death movie… chilling thought.
Excellent work gentlemen, keep it coming!
[snip] no need for condescension – tb
I suggest that instead of just investigating square roots, you parameterize the ladder of powers into your analyses. This is standard operating procedure for many of us who make every effort to do careful data exploration & diagnostics. Statisitical inference that ignores this issue often ends up being based on demonstrably false assumptions.
This discussion looks like a strategic opportunity to remind everyone of the hierarchy of patterns nested in the heliospheric current sheet (HCS) or heliomagnetic field (HMF) or interplanetery magnetic field (IMF) …or whatever one prefers to call it:
HCS Earth-Crossings
27d: http://i48.tinypic.com/2yydr92.png
1a: http://i46.tinypic.com/2qvwacn.png
HCS Earth-Crossing INTEGRAL (emphasizes shifts in persistence)
27d: http://i45.tinypic.com/2nbc3dw.png
I’ve pointed out the coherence many times:
Tsonis’ episodes of network synchrony are determined by the sun.
As Milanovic has patiently cautioned us: It’s a field, gradient, & flow problem (i.e. it’s spatiotemporal). And as Dickey has cautioned us: Temperature, mass, & velocity are coupled. There’s no way to avoid the absolute exploratory need to get a firm handle on aggregation fundamentals. Otherwise it’s just more mainstream modeling based on false assumptions — i.e. more mainstream failure to recognize statistical paradox.
It has been refreshing seeing some of the developments in this thread.
HCS… see the Parker spiral on the poster shown in the post?
Steven Wilde: Greg, could you supply a verbal narrative to explain the nature and significance of sqrt(SSN) ?
Ray Tomes suggested that the magnitude of variability meant that running his software on the square root would work better. I said I thought that would be unjustified without a physical reason. Just gratuitously doing operations like that on data goes a bit against the grain.
However, it got me thinking that sun spot number was counting spots of something we seem to have only vaguest ideas of the mechanism causing it.
Sun spots have to reach a certain area to be visible and counted, so there is already some kind of square law. It also seems quite possible that the physical manifestations we are counting are related to some square law, so I thought it was worth examining. I am not proposing a physical mechanism, I’m looking at whether the time series of the sqrt() suggested by the noise patterns indicates there is something to follow up on.
I have long been doubtful of the 0.5K post-war drop that Hadley insert into the data. I looked into this in considerable detail in my article on Climate Etc. last year.
This plot seems to confirm my suspicions if there is indeed a link between SST and TSI. Even mainstream are now invoking TSI to explain the plateau despite having said for the last 30 years or so, it was too small to matter .
The late divergence is also interesting since it may leave some evidence of AGW depending on how TSI affects SST and the time to equilibrium.
The SST data is not reliable further back but we know from other sources that there was a redoubtable cold period in early 19th c. that matches the SSN changes.
I don’t think my TSI v sqrt(SSN) is valid. The scaling is too dependant of the filter. I’ll come back on that.
Since some seem interested in the power law of some solar activity perhaps I can add something.
For me this is an old subject where if I recall there is a view out there it is ^1.4
My own looking at this suggests the figure is about right under some circumstances, really rather depends.
My assumption is there is some level of solar activity. Above that level is sunspot periods as a non-linear instability.
As it happens I deleted some content from the article here as overloading. One item was a comment about the possibility there is an inversion of the assumed solar characteristic.
This is a casual what-if posit that solar magnetics stabilise the sun.
My reasoning is the magnetic reversal occurs during maximum sunspot cycle activity when the magnetic field is at its lowest and the sun is quiet when the magnetic field is high.
Further confusion comes from the asymmetric solar behaviour, hence the ragged peaks and so on, been described as undulation [Eddy].
Note: I also deleted power law traces from the plot. Only confused things.
This plot : http://wattsupwiththat.files.wordpress.com/2012/11/clip_image017.gif
from here:
points out what I was suggesting.
compare to the following:
I don’t have the data for Svalgaard’s plot to overlay them but the it seems clear they are similar in form (except the vertical scaling).
SSN is a proxy for the square of helio magnetic flux as detected on Earth.
Ray’s suggestion of taking sqrt of the data before processing would seem to be justified. Indeed, before I passed the light gaussian shown here, the high frequency ‘noise’ seemed fairly uniform across each solar cycle.
Now Vukevic has been pushing the magnetic linkage for some time now. As I suggested with the following graph, we may be better looking at unadulterated ICOADS rather than Hadley “corrected” SST.
OK. I can now see the graphic [1]. It would be interesting to see that projected forward some years, we have a reasonable idea how the current solar cycle might pan out, so for one possible scenario if we assume that the next solar cycle is as low as this one, then we might get some guide as to the possible temperature drop courtesy of sqrt(ssn).
Is this another way of assessing the impact of possible magnification factors such as clouds ?
[1 snipped no see comment –mod]
“…. we may be better looking at unadulterated ICOADS rather than Hadley “corrected” SST.”
No “may” about it in my opinion.
PS. a couple of quick notes.
I note that my sqrt(SSN) matches the measured data (red) section of the Svalgaard plot but accepting that correlation it suggests his back projected IDV proxy is rather high. This is quite important since a lot of his recent work seems to be aimed a flattening out historic TSI rise.
The 0.34_adj on ICOADS is to subtract this WWII rise between which is clearly due to US Navy mobilisation affecting records. This was explained in detail in my Curry article:
The resulting post war drop in Hadley ‘corrected’ data seems spurious.
I also noted that short and long term cycles were disrupted. This is why, while I do not regard ICOADS as being without issues, I think it likely that Hadley’s correction are further muddying the waters and any work on looking for periodic patterns in climate would be better done on the unprocessed data.
I’d forgotten playing around with solar data straightening, Ray is right this is a good idea.
Reminded myself earlier, is tantalising, seems close yet never so far quite right.
I think there are several incompatible processes going on which makes decode fiendishly awkward.
When playing I tend to use short data, looking at predictive.
I’m continuing with where I left off on this one, uses restored bipolar and non-linear.
There is a hint the current solar cycle will have a double top, related I suspect to the unusual wind-down of cycle 23.
Roger Andrews says:
“…. we may be better looking at unadulterated ICOADS rather than Hadley “corrected” SST.”
No “may” about it in my opinion.
I would not be that categoric about whether all their processing is detrimental but they are making significant changes and some of it definitely seems detrimental.
They have clearly blown out the pre-1925 data. This is one of the periods I was highly critical of in the Curry article. They were/are intent on removing the pre 1900 warm period which shows natural variation close to that of 20th c. I showed thier “corrections” were removing about 67% of the variation in original data for a lot of this period.
When you look at auto correlation and see they also destroyed its structure, it’s hard to see it as improving the data.
Whether they have recovered some real structure in the late 20th c. or simply induced it I cannot really say.
Same for North Atlantic:
I really don’t think Hadley “corrections” are helping much with looking for correlations.
Interesting that N. Atlantic shows such a clear and durable repetitions. Looks more lunar than solar.
109.75/6=18.3
Square root is just one rung on the ladder of powers. Suggestion: Parameterize the whole ladder of powers into analyses as standard operating procedure. This has nothing to do with physics. This is about ensuring that statistical inference assumptions are NOT FALSE. This may present inconvenient challenges for those building physical models, but if those physical models are to underpin statistical inference then statistical inference assumptions canNOT be false; physics has NO say in this as it stems from elementary logic.
There has been so much brainwashing in the climate discussion about physical laws, it seems, that awareness of statistical laws has been suppressed. This should give pause to appreciate the challenges of cross-disciplinary work. The different paradigms might most productively be regarded as complementary rather than competing. It’s not an “either or”, but rather a matter of walking around something you can’t see very well to get different perspectives in order to better inform an overall image.
If understanding matures enough, there will be models free of issues with BOTH statistical AND physical laws. On many fronts in the climate discussion, we’re nowhere even remotely near that stage …so keep exploring, and don’t be intimidated by thought-policing authoritarian bullies who tell you to stop trying. We’ll get exactly nowhere if we allow ourselves to be coerced into not trying.
–
Separate minor note: A square term shows up in BV^2. Upthread in some of the comments this seems to have been overlooked.
–
The question we haven’t yet answered:
Why are Tsonis’ climate index network synchronizations coherent with the solar Hale cycle?
I identify this as an area where we are only seeing informative shadows. The shadows are crystal clear and crucially informative, but on this front we’ve not yet identified the universally constrained metrics casting the shadows.
I should have marked signif levels on that plot. For this number of points it’s around 0.2 @95% signif. , rising slightly towards the end (in the same degree as the plot peaks) as the length of data is reduced by the lagging.
So all those peaks are pretty sure to be true significant correlations, once Hadley has “bias corrected” the peaks are mostly under the 95% mark.
tchannon (January 25, 2013 at 4:36 pm) wrote:
“HCS… see the Parker spiral on the poster shown in the post?”
Certainly the “Fall” & “Spring” panels of the “Reversals of solar magnetic field” graph near bottom center are informative.
Click to access Poster_A0.pdf
Nicola Scafetta’s paper “Empirical evidence for a celestial origin of the climate oscillations
and its implications” looked at JPL ephemeris tables and found a spectral peak of 9.1 +/-0.1 years in the Earths velocity (hence distance) relative to the sun. He showed that this was due to the presence of the moon.
The length of the autocorrelation peaks I noted about give a cyclic repetition of between 9.1 and 9.15 years duration.
This may indicate a bulk water movement is resonating with this lunar induced movement of the Earth.
I’ve pulled this out of a file from a couple of years ago.
All I can remember is this came from work on solar asymmetry showing in the nasa/greenwich dataset. The lack of circa 11 year was a surprise but my main interest was the circa 45 year which matches a number of terrestrial datasets, such as satellite sea level and temperature.
I was not happy with the solidity of the data extraction so I left it as a curiosity. If true this put a sequence on solar, temperature, sea level, in that order.
Take as dodgy but 8.9 vs. 9.1?
[update] I might need to explain more, much of this is not sustained waves.
Greg:
“I would not be that categoric about whether all their processing is detrimental but they are making significant changes and some of it definitely seems detrimental.”
A few years ago I reviewed the HadSST2 corrections in some detail.
Click to access final-sst.pdf
What I found was that the enormous HadSST2 bucket-intake “bias corrections” had been applied basically to make the SSTs match the NMATs. Certainly they were unsupported by metadata. The corrections were in fact so slavish that a lot of time and effort could have been saved by using the NMATs as an SST proxy and throwing out the SSTs altogether.
Which raises the question of whether the NMATs are a valid SST proxy (or vice versa). There’s no compelling evidence that they are and a substantial amount of evidence that they aren’t.
“They have clearly blown out the pre-1925 data.” This period contains the Folland and Parker insulated-uninsulated bucket bias corrections. Folland and Parker theorized that a gradual change from insulated to uninsulated buckets during the second half of the 19th century and the first half of
the 20th had introduced spurious cooling into the raw SST record, but they were unable to quantify it directly because there were no bucket type metadata. To get over this difficulty they used SST-NMAT to define bucket type, with higher SST-NMAT values defining uninsulated buckets and lower SST-NMAT readings insulated buckets. Then they used the results to adjust the bucket bias out of the raw SST data. So it’s not surprising that the pre-1925 data are blown.
Tim: Take as dodgy but 8.9 vs. 9.1?
Those peaks are fairly broad and 9.1 would be very near the peak. It is also the strongest peak below 45y.
Shame you can’t recall what the data represents.
Rog.A. Yes, HadSST3 was based on the same speculative thinking. They just tried to make the 1946 step change less obvious by fading it in. Involved more ignoring metadata: changing data clearly identified as being bucket readings into engine room intake to suit thier proposed quotas of buckets for the grid box concerned. Basically rigging the data to conform to their suppositions.
I’m pleased I’ve found that Atlantic data that showed such a clear cyclic repetition before their processing. Makes the point nicely.
Greg, If Tim’s plot is sunspot production asymmetry from the Greenwich dataset, the peak at 8.9 won’t be lunar related. the Peak at 5.92 will be related to the fact Jupiter spends that amount of time above the solar equator, followed by that amount of time below the solar equator. 45 years is the inner planets realignment period. 120 years is the long period harmonic of the peaks in the MEM periodogram found in Bart’s analysis. those peaks are Jupiter/Saturn related. Ian Wilson showed how the 45 year period is also related to the gas giants. 8.9 is mysterious, but may be someone else could enlighten us. Ulric maybe.
Something to do with sunspot area and latitude. How I did it given the messy data is the problem, reproducible result. Probably something like sine(lat) * area
Hence my interest in asymmetry and in magnetics.
There is also the probably cyclic north/south earth temperature shift, which connects in with the terrible sampling of earth surface and sst/land measurements, in sat data, mostly in surface data.
Looking at hadSST3 and ICOADS in N.Pacific.
I had previously been using hadSST3 and had found an apparent beats pattern of 5.4 and 6.1 years, at least over the last 60 years or so it seemed quite a clear pattern. I was interested to find I had already detected 5.42y in Arctic ice cover:
But was a bit surprised that there was no apparent sign of the circa 6y cycle.
Well may be that is because there isn’t one. It’s a part of the fiction generated by Hadley processing.
If I look at the icoads line in the N.P. plot I see what looks like a beats pattern phase inversion at -53.9y . When two harmonic oscillations superimpose there is usually a 180 phase reversal at the node of the beat pattern and the pattern reflects about this point. What we see at -53.9 looks very much like that.
The first positive correlation peak is again my 5.4y. Equally from the end: (100-78.76)/4=5.31
Looks like there’s 18y in there too. Needs some digging.
key thing is there doesn’t seem to be 6.1 I was seeing in the Hadley rewrite of climate history.
New data, from NASA-Greenwich-Hathaway sunspot dataset.
https://tallbloke.files.wordpress.com/2013/01/hath-sine-2012.zip
Inside the zip is a csv computed by me from the raw data. This involves fairly involved code and is an interpretation of the data. Data is a catalogue of all sunspot groups for a day, problematic because information, standards etc. have changed over time. Hathaway has done his best.
I have computed a daily dataset from this, that is the csv. Runs from 1874
Column sine area seems to produce the same spectra as above so I must have recreated the original time series when I was reworking the decode last year. I’m still nervous of the provenance.
Take as unofficial.
Greg:
The ICOADS and HadSST3 data from 1941 through mid-1946 or thereabouts are heavily contaminated by observational biases. I don’t know whether this might be corrupting your results, but I mention it FWIW.
Thanks Roger, but you don’t seem to have read the article I wrote on hadSST3 that I linked to above and linked a graph showing that period and it’s problems. I am well aware of the problems and have analysed them in depth and discussed them in comments of that article with John Kennedy of Hadley.
That’s one reason to look at dT/dt. Any data collection bias ( or bias correction bias !) will be localised and not shift the rest of the record.
In fact, much of the distortion is due to the Hadley regridding algorithm rather than the dubious assumptions of the bucket ‘corrections’. Prior to 1925 it’s both. That is why that period looses all structure.
The problem with 41-46 was a simple offset (in ICOADS 2.4) which I estimated to be 0.36-0.40K . Unfortunately , rather than resolve this, in v2.5 more data was added from (british) admiralty records that ended up just muddying the waters. The problem was diluted rather than being resolved.
Hadley are to this day simply fudging the issue by applying a once sided ‘correction’ to and up-down bias. That is why they have a spurious post war drop of about 0.5K.
The war time offset was a step change from one month to the next when US got invovled in WWII until demobilisation in 1946.
The original Folland’s folly was to correct one side of this and not the other. The current fix is do the same thing but blend it in over 20 years so its less obviously wrong.
Tim: Column sine area seems to produce the same spectra as above so I must have recreated the original time series
What is the asymmetry being investigated here? Solar N/S , odd/even cycle no. , rising falling edge?
Sunspot activity is often biased to one hemisphere, butterfly diagram, derived from the same dataset probably helps. http://solarscience.msfc.nasa.gov/images/bfly.gif
This seems to be related to magnetics.
The csv can be used in various ways provided daily is okay. I don’t have a Carrington version, bit artitrary anyway, we can only see one face.
Data’s there if it is any use.
Already working with it Tim, thanks.
Greg:
If you’re happy I’m happy. 🙂
“The war time offset was a step change from one month to the next when US got invovled in WWII until demobilisation in 1946. The original Folland’s folly was to correct one side of this and not the other.”
I don’t have the data to hand, but as I recollect Folland’s bucket bias “correction” stood at around 0.4C by 1941 or thereabouts. He then reduced it abruptly to zero on the basis of the assumption that the mix of insulated and uninsulated bucket, intake, hull sensor etc. measurements after 1941 would give unbiased results. There was of course zero support for this assumption. It was just a convenient way of getting rid of an inconvenient upward blip in the data.
But as you note, he didn’t get rid of the inconvenient downward blip on the other side. And 28 years later the Hadley Centre Data Correction Corps still haven’t gotten rid of it.
Yes RogA , this all seems to be part of an overall strategy that has been going on for about 30 years at Hadley (in fact about since it was opened they Thatcher). Play down late 19th c drop (nat. change). Knock 0.5 out of early 20th c. rise (nat. change), makes late 20th change look more like unnatural change.
In layman’s terms: a bit like trimming your pubes to make your dick look longer. All part of the AGW porn.
I think I feel a post coming on ….
Uh huh.
BLATANT REQUEST
If you want to help raise the profile of the talkshop and improve our chances of getting sponsorship for the September conference, please nominate us for the Bloggies awards.
Nominations close later today.
Thanks for your help.
It occured to me this morning that the whole discussion with Ray Tomes was about applying his spectral analysis software to SSN. Having realised that SSN is proxy for magnetic flux his sqrt suggestion makes sense. It may help remove spurious harmonics if we look at the physical property that is causing the effect rather than its square.
The obvious problem is that to do this we need to be able to resolve the +/- ambiguity in sqrt(). SSN does not contain this information.
Now since sin^2=2sin.cos with half the period , using a spectral analysis technique that uses both sine and cosine on the squared property may cover that. Needs thought.
The other observation, that relates more directly to the poster linked at the top of this article is the fact that SSN cycle is not really either of a stable period nor harmonic.
I looked at the panel on field reversal and it just did not ring true for me. Maybe I just missed the point but I was not convinced.
Looking at the profile of each (11y) cycle I see it more like an exponentially decaying rise (1-exp) and an exponential decay, with possibly a short stable plateau.
However, this means the system is not bipolar, which seems to be the idea put forward in the poster but tripolar. There is a significant zero polarity phase, not just a N/S flip-flop bistable.
It is recognised that longer cycles tend be the lower activity one, this further suggests that the lower polarised state is more stable. 1905 1925 1970 cycles appear to have longer plateaux.
I would suggest that for almost half the active part of each cycle the driver is ‘off’. I’m not talking about the keyhole view get by looking at outside, but the deep motor that is driving the cycles.
This is not my field, so maybe this has all been said and done, but all I’ve ever seen and what appears to be shown in the poster, seems to regard the quiet periods as simply the transition of zero as the poles flip.
I would suggest a tristate approach would better describe the data.
Greg: The cycles tend to be alternately pointy and plateau’d. This ties in with the planetary motion of Jupiter, Venus and Earth where the conjunction-opposition cycles reflect this ~22 year cycle. Both Ray and Ian Wilson have published on that. I looked at the alignments along the Parker Spiral (dynamically corrected for solar windspeed variation) and got this result:
Greg says: January 27, 2013 at 11:04 am
his sqrt suggestion makes sense
Makes a lot of sense to me…
I would suggest a tristate approach would better describe the data.
I really like that line of thinking… definitely worth pursuing…
My research on Length of Day oscillations is heading in that direction: Positive driver, Neutral [with some sort of “frictional” decline] and Negative driver.
It’s beginning to feel like a car with two gears [Forward, Neutral and Reverse] and a very powerful engine that really does some “shake, rattle and roll” when you hit the throttle…
Makes sense:
1) If the “bearings” aren’t fixed to something solid:
2) If the “magnetic” Continuously Variable Transmission really “slips” a lot:
Well the forward and reverse going through idle is more an accelerating and braking effect according to Ian Wilson’s tidal torquing model.
TB: This ties in with the planetary motion of Jupiter, Venus and Earth where the conjunction-opposition cycles reflect this ~22 year cycle. Both Ray and Ian Wilson have published on that. I looked at the alignments along the Parker Spiral (dynamically corrected for solar windspeed variation) and got this result
Any refs to those papers?
That plot is impressive. At least upto 1940 it fits remarkably well. Then it starts to drift out of phase a bit, plus amplitude. Something significant is missing. Why no Saturn?
How to reproduce your planetary index?
Greg: Wilson’s paper: http://www.gsjournal.net/Science-Journals/Research%20Papers-Astrophysics/Download/3812
See figs 3a and 3b particularly
Here’s the later development of the tidal torquing model:
Agree there is still much to do with my plot. Amplitude may be more a gas giants thing, whereas inner planets are more a timing thing.
The planetary index used is Roy Martin’s replication of Ching Cheh Hung’s specification:
Click to access Reconstruction_of_Hung_Fig4_01.pdf
Roy used it in direct linear graviational configuration.
I modified it to look at alignments along the Parker spiral, corrected for solar windspeed using a composite of the Svalgaard reconstruction plus an earlier study which went further back. I also reduced the Venus weighting to allow for the fact that Venus doesn’t have a magnetosphere.
To address your further point about goodness of fit. I found that by making small adjustments to parameters, the point of best fit moved along the timeline. Also, we might consider Charvatova’s periods of harmonious and disharmonious motion in this context.
Roy Martin:
Hung, is 3.0. The cutoff level was set at 1.88, which appears to capture somewhat more of the upper
part of the dataset than the top 25% of most aligned days as analysed by Hung. The number of
periods in the moving average was explored from twenty to forty two. Up to about thirty two
periods the form of the curve was somewhat erratic, becoming a progressively better fit as the
sampling period increased. It also tended to become erratic over about forty. The form of the curve
was fairly consistent between periods of around thirty four and forty months, so the mean value of
thirty seven was applied. The second smoothing period of twelve months was not found to be
critical.
========
OH NO! My old friend the runny mean again. When will the science world wake up to this?
The peak of the disastrous negative lobe is at 1.3371 times the base period. 40/1.3371=29.9
Having found that there was a frequency around and above 30m that he needed to filter out, by the time he gets to 40m he’s letting it back in. Not only that but he’s inverting it! I’m not surprised his results started getting “erratic”. Sure the second “smoother” of 12m would be pretty pointless after that.
Sounds like there’s a peak around 36m months that he needed to filter out.
He would likely get a lot more stable result by applying a properly designed filter rather than just stabbing around in the dark with a runny mean. He does not even seem to know why he’s doing all this , it’s just suck and see. The trouble with ‘suck and see’ is that you usually see it sucks. 😉
If you are doing anything similar to this, I suggest a clean filter like the gauss.awk script I posted here a while back, or for those not capable of anything apart from runny means do it in three passes with windows getting shorter by 1.3371 each time.
eg. 40:30:22 ; 36:27:20 These triple-mean filters do have the advantage of having a zero at the first window length. (40m and 36m in these two examples). Gaussian has a nice clean even profile that does let a bit of everything through. If there is fixed, known, strong period to be removed, the triple-mean may be more effective.
[ first min of sinc at tan(x)=-x: 1.3371 pi ]
I can only assume that Hung’s “aggregation” period of 3.46y (41.5m) was also averaging. This would thus suffer exactly the same problem. [Never use an average or runny mean to “smooth” periodic data, means are good for removing _random_ noise, not periodics.]
This kind of crappy filtering can be very misleading, creating false signals and breaking real ones.
Roy Martin:
A new data set for the index of planet alignment was generated with data points at intervals of one month. For this purpose the ‘month’ is one twelfth of the Julian year of 365.25 days, i.e.,
30.4375 days. To simplify calculations, the eccentricities of planetary orbits were ignored.
==========
I would have thought the JPL ephemeris would provide all the accuracy needed without such compromises being needed. Also this period is bang in the middle of the range of solar rotation periods (25-35 days) which must affect the SSN counting process done by viewing the visible side. This 30d period will cause an interference pattern when compared to SSN data. At least one of the two needs to be filtered and sub-sampled.
eg get daily data, run 30d , 3-sigma gaussian then resample at 30day intervals.
done with a bit more rigour, some of the oddities should disappear and any real relationships should become more apparent.
HTH. 😉
Greg says:
HTH
It’d help a lot more if you’d find the time to try it. 😉
Thanks though, some good tips there.
37m is also near 1/3 of the C20th average solar cycle length. If you smooth the surface temperature record at this period, the solar effect on global average temperature becomes more apparent.
http://woodfortrees.org/plot/hadcrut3vgl/from:1950/mean:37/detrend:0.6/plot/sidc-ssn/mean:12/scale:0.002/from:1950/offset:-0.4
Greg,
Welcome to a fun problem. Yes a lot has been written.
I look at the sunspot problem occasionally, largely a waste of time when there are more fruitful but ignored things to do.
A warning. The ~1790 problem kills most ideas stone dead.
Back then we are faced with lousy data reliability in combination with lunatic solar behaviour.
There is what to EE looks like phase reversal but also a “maybe” solar cycle, no solid evidence. Perhaps also a double high.
Here is a later quick look from earlier work
Click to access ssn-1780-anomaly.pdf
Greg,
I add, the sunspot regime looks like a chaotic oscillator, with some supporting evidence in phase behaviour. I suspect largely bi-modal but I can’t prove it.
TB: It’d help a lot more if you’d find the time to try it. 😉
I’m sure it would but I’m getting way too distracted from what I wanted to do on SST already.
If you try my triple-mean suggestion its even clearer:
http://woodfortrees.org/plot/hadcrut3vgl/from:1920/derivative:36/mean:36/mean:27/mean:20
http://woodfortrees.org/plot/hadsst2gl/from:1880/derivative:36/mean:36/mean:27/mean:20
Looking at derivative provides an alternative to subjective detrending. What is superbly clear here is the presence of both the solar cycle and the circa 9y I pointed out in N. Altantic SST.
The two are more or less in sync in the latter half of the 20th c. being aligned about 1980, and we can see how they diverge in phase further back.
It has always been a problem for those pushing solar cycles that there is a phase crisis further back. Obviously WWII is screwed as we have already commented. Further back they seem to be about equal in magnitude and 180 out of phase. Hence the phase crisis when looking just at temps and solar.
I’ve never seems such a clear demonstration of this though. It was only when I stopped looking at HadSST3 that I realised how pure the 9y signal was in N.Atl.
It would be nice to see the same plot for ICOADS, of course WFT.org does not want you to see that.
http://woodfortrees.org/plot/hadcrut3vgl/from:1920/derivative:36/mean:36/mean:27/mean:20/from/plot/sidc-ssn/mean:12/mean:9/mean:7/derivative/scale:0.002/from:1900
Here’s another look at your comparison of SSN and HadSST.
Again the pattern matches well in 1980 and we can see some other oscillation is distracting from the solar peaks. Looking at the derivative we can see yet another demonstration that the 1942 rise and 1946 fall in SST has not been resolved. This is clearly an exception to the rest of the record and in total opposition to the contemporaneous solar peak.
Greg, thanks, that’s a cleverly done plot. Regarding SST record: Roger A recently posted on the El Nino which caused cold snowy weather in Russia 1940-42. Maybe the SST record is contaminated with NH land temps somehow?
Didn’t mean to cut that off at 1920
http://woodfortrees.org/plot/hadcrut3vgl/from:1900/derivative:36/mean:36/mean:27/mean:20/from/plot/sidc-ssn/mean:12/mean:9/mean:7/derivative/scale:0.002/from:1900
It looks like the two were in phase and passing through zero in about 1955 and out of phase in 1917 and 1997/8 as near as I can read of a WTF.org plot. The latter dates seem to be the same point in the combined cycle.
That gives a beat cycle of circa 80 years: 9×9 ~= 10×8
The two cycles seem about equal in early period and solar is stronger later, so pattern is not identical each side.
If this is Scafetta’s 9.1y lunar cycle that gives a nice luni-solar combo covering a large part of the cyclic variation.
Greg: The big El Nino following solar min in 1998 knocks things out of kilter, because it is followed by a big la nina. There doesn’t seem to be a similar spike in SOI in 1916, but there is a big spike in AMO around then. Hmmmm.
TB: Greg, thanks, that’s a cleverly done plot. Regarding SST record: Roger A recently posted on the El Nino which caused cold snowy weather in Russia 1940-42. Maybe the SST record is contaminated with NH land temps somehow?
Difficult to guess what that period should look like. It exactly when US mobilisation messed up the record. Though a reconstruction based on beats I’ve identified would give a better guess than what we see here.
It was the trough of max cooling trend that corresponds to 1974/5 on the other side (mirrored). So it’s possible this could have been predicted as a particularly cold few years on the basis of this beat cycle.
Now if that also tied in with the cold part of the 5.4y cycle in the N. Pacific as it appears from RogA’s thread. That makes it about the worse time to try to take over Russia.
Hitler should have considered himself lucky not to have conquered Britain first: he would have had the Met. Office to advise him to expect barbecue summers 😉
Here is autocorrelation of S. Atlantic. (60W-15E; 10S-50S)
Not much similarity to N. Altantic. I was rather expecting at least some sign of the steady 9y repetition I found up north. However, beautiful beats profile there. Looks like 70y, as well as I can pin point the null points. modulating 60/11=5.45y. That is equivalent to 75.5 beating with 64.5y.
That 5.45 is very close to what I found in N. Pacific and d/dt of Arctic ice cover (5.42y).
That raises the possibility of 70 modulation of the ice cover which would not be inconsistent with the 40y of observations on which that graph was built.
It is interesting how good the pre 1900 data is for this region. Most areas tend to get a bit messy that far back. However, before the Panama canal opened in 1914, a lot of shipping had to go around the Cape of Good Hope, so we have rather better coverage here. Nice.
BTW Those dates are relative to end of 2009, ie pattern decorrelates around 1910-1915.
As always a word of caution. Autocorrelation shows repetition not form, so this is a pattern of 5.45y interacting with a pattern repeating in 70 years. These patterns may or may not be harmonic.
OK, this is what the average 5.45y profile looks like. I did a “seasonal” average with a season of 65 months and it was surprisingly regular and roughly harmonic. Nice.
Just moving to 66 or 64 it goes quite lumpy so this must be pretty close to the correct period.
Now, unless I’ve messed up my numbers, this would seems to show S. Atl oscillating between 0.5K/century cooling and 1.2K/c warming, with an average of 0.2K/c over the full record used.
Before jumping to conclusions about what the latter may mean , I should repeat the exercise having trimmed the data to be just the 70y cycle.
TB & others — a reminder …
SOI 1916 July&August — see bottom panel p.1:
Solar-Terrestrial Resonance, Climate Shifts, & the Chandler Wobble Phase Reversal
Greg: The far northern Atlantic also has a 74 year component.
And a 45 year component:
Paul V: Noted. Timely reminder.
Interesting. I had intended to have a closer look at the kink about half way along my S. Atl autocorrelation plot:. Wanted to see whether it was an interference pattern or a possible data glitch.
It’s at 73.7 years and looks like an interference phase shift. This method is probably more precise than Moret wavelet at pinning down the frequency. I think there is a strong possibility these are the same thing.
The switch in chandler wobble would be at about 82y in this data. Nothing notable in this data at that point.
My impression from the autocorrelation work suggests that Tsonis , Scaffeta etc would do well to repeat their analyses using non “corrected” datasets. It may be very revealing.
Greg (January 28, 2013 at 10:20 am) wrote:
“This method is probably more precise than Moret wavelet at pinning down the frequency.”
Careful. There are an infinite number of ways to do the same thing. It’s just a matter of finding time (not always possible given competing essential pursuits) to customize algorithms for the job at hand. Tim’s software can be modified to do the same thing wavelets do. Wavelets can be modified to do the same thing Tim’s software does. That’s the neat thing about math. Everything’s related to everything else. Math’s a big world. It’s just an investment of exploratory time (not always practically feasible) to see all the interconnections. This could consume many lifetimes. A sign of intricate awareness is ability to arrive at the same place via multiple paths.
The most important cautionary note here is that a temporally-global approach leaves one at risk of missing crucial spatial info. Walking around and looking from many perspectives using many methods hopefully we eventually see a way to isolate the universal constraints we seek. On that note…
TB: Another reminder on 1916 & other dates:
SOI Integral
Ready for some door-opening mind-bending?…
I’m going to suggest you look here — I hope Tim Channon will too:
Chen, T.-C.; & Wu, K.-D. (1992). Semi-annual oscillation of the global divergent circulation. Tellus 44A, 357-365.
http://www.tellusa.net/index.php/tellusa/article/viewFile/14967/16781
It may not be readily apparent why I’m pointing at semi-annual land-ocean asymmetry in an equatorial heat & water pump, but it should be immediately apparent that Hale spacing lines up with the dates you’re emphasizing. You can easily check by looking at the spring & fall panels in the bottom-middle graph of the poster to which Tim Channon linked and by looking at the HCS graphs I linked to above.
Tsonis is pointing us in the direction of a universal constraint on aggregate properties of terrestrial circulation. This is very refreshing stuff, particularly in stark contrast with the mainstream approach to modeling, which totally ignores the need to bound aggregate circulation to conform with observed earth orientation parameters.
The Hale cycle shows up in residuals from mainstream EOP models. Why would something so simple have been overlooked? Spatial phase reversals (e.g. interhemispheric gradients) blind temporally-global & uni-extent time series methods that are the staples of the conventional mainstream. That doesn’t mean the methods are wrong. It means they need to be customized to extend their range of tuned vision to a different class of mathematical features. It’s actually quite simple, but it helps to be aware of the nature of the features when designing a microscope to detect them.
FYI: The Hale cycle can also be found directly in seasonal SOI via aggregate differencing.
Tim Channon: Absolute north-south solar asymmetry is directly proportional to solar activity. This is not reported in the literature, but it’s simple & straightforward to confirm. (You will find very high correlations if you look.) The role of solar asymmetry from a terrestrial perspective is all about how it resonates with what Earth already has going on in a big way (e.g. the year).
Here I have limited the assessment of ‘seasonal’ mean cycle to a 70.8y period (13 cycles of 5.45y). This ensures that where ever this hits the long cycle it is always peak-to-peak , trough-to-trough or some equivalent average over a full cycle. This avoids getting false ideas from having more of the rise phase for example.
This is , ultimately the aim of this kind of analysis, to identify dominant cycles in the data before look for ‘trends’.
So this plot shows the 5.45y cycles averaged over : the full record ; the first 71y ; the middle 71y and the last 71y upto end 2009.
Interpretation:
full: nice even harmonic, mean over cycle of 0.7K/century rise. No surprises there but confirms analysis is coherent with other estimates of long term warming. Also note no up/down trend in cycle ie no temperature acceleration in either direction. Stable longer term rate of change.
early: cycle average slightly negative. no trend, ie small const cooling. Flat floor to end of cycle (-2K/century)
middle: cycle mean very close to full record about 0.7K/c; no accel trend.
last 70y: cycle mean somewhat higher (est. 1K/century) ; notable deceleration (end of cycle low then start).
Need some more time to pull specific numbers but the last part looks significant.
That is a very solid deceleration over the full 70 years of the last cycle. If I’m interpreting this correctly that means that we are at the end of the long term 0.7K/century warming.
Anyone care to estimate that slow down in K/c/c ?
Fitting a cosine to those profiles to estimate the mean offset rate of change:
early -0.3 K/c (centred on 1912)
mid 0.42 K/c (centred on 1948)
late 1.0 K/c (centred on 1975)
full record (133y) : 0.55K/c
Looking at the residual of the late profile from the cosine shows there is NO obvious trend over the cycle. The start/end offset is simply part of the mismatch from the pure harmonic.
That is better, since any accel visible over 5.45y would be worrying.
Having averaged the rate of change over complete cycles of the detected periodicity, the acceleration in the earlier half of the record was slightly greater than that in latter half.
You mention ICOADS, this stuff http://climateaudit.org/2010/08/30/a-first-look-at-icoads/
I think the question from me has turned up before on where the derived data you are using came from? Somewhat difficult to find.
Tim, good point. I have been extracting certain grid coords from ICOADS v2.5 as delivered by KNMI climate explorer.
One must always have an healthy regard for the origin of the data and any possible errors, biases and corruptions. However, having found the degree to which Hadley processing is changing the frequency spectrum (which on the face of it could be either an artful improvement are a badly audited corruption) I have an even more healthy scepticism for data that has be ‘processed’ to the point where a regular pattern in the autocorrelation gets broken or where new ones pop up that weren’t even present in the original.
I’ve just taken a look at S. Pacific expecting maybe some similarity of the beats I found in S. Alt.
Think again. More resemblance to N. Atl than South.
again underlining the importance of this 9y cycle when looking for correlation with SSN.
This is an important result:
Good work Tim.
Paul Vaughan says:
January 28, 2013 at 12:55 pm
Greg (January 28, 2013 at 10:20 am) wrote:
“This method is probably more precise than Moret wavelet at pinning down the frequency.”
Careful. There are an infinite number of ways to do the same thing…Walking around and looking from many perspectives using many methods hopefully we eventually see a way to isolate the universal constraints we seek.
==========
I agree, looking at a problem for diffenent angles brings better understanding. I was in no way suggesting there’s anything wrong with Morlet, it’s a valuable tool.
I have a high degree of confidence in the 5.45y period since varying by just one month produces notable distortion in 5.45y average. I do not have reason to be so confident in the 70y estimate of the other period. I hope to get a better look at that later.
What I find a little surprising is the lack of anything looking like 11 or 22y in autocorrelation. This may well be because this is far too irregular to produce a clear autocorrelation signal.
Yes that [Paul] is the intended take home but most blog subjects tend to get highjacked by argument over longer temperature wander. Hurst ought to be sufficient to stop that stuff dead as a useful topic, nothing is going on.
As I see it the claims of solar/terrestrial linkage are very weak but just one definite would open the door to more. In this case I assume magnetics somehow influence the earth and can be seen globally. I further assume a magnetic effect would be polar, precisely where there is a lot of unknown involving magnetics indirectly. Since the poles primarily shed heat this implies a modulation of heat loss.
Greg,
Fun starts. Sometimes an absence is so strong it leads to suspicion. An off topic example is the 33y period in some terrestrial proxy data yet is missing from all normal time series.
I have tried to explain this, including via the arctic ice example, rotating systems causing frequency doubling. You will get a yawning null.
Sunspots do 11y, nothing terrestrial, uh huh. What is double that frequency?
Perhaps double 33y period is what? (mental gymnastics for any casual readers here, period is reciprocal frequency)
Greg,
Some madness.
If you can open this I’ve faked up a 5.4 y from a crude run down ssn data. Column L
Phase is taken from ssn but anything is possible. Does this relate at all to the 5.45 you are seeing?
You can fiddle with the period and phase without too much difficulty.
https://tallbloke.files.wordpress.com/2013/01/had4-aux-3.zip
Tim, I’m not quite sure what you are aiming to show here. There’s still far more 10.8 than 5.4 so the 10.8 should be clearly visible.
sin^2(10.8) would have 5.4 period and that is equal to 2SinCos or 2Sin.d/dt(Sin) , so if the response was to the square of SSN or SSN.d/dt(SNN) it would have a period of 5.4 but that is getting a bit wild.
SSN is already a proxy for the square of the flux.
Also if Hale and Swabe were inter-modulating (ie some response to SSN x some response that is sensitive to polarity) that would give 16.2 and 5.4 but I was not getting any sign of 16.2 either.
The 5.45y ‘seasonal’ signal was surprisingly sinusoidal, I was rather expecting something more complex that would indicate more complicated origins. If this is related to Hale , something is ‘distorting’ it back to being a nice harmonic. 😕
It think the discerning eye may pick up a hint of some variation in period that may be similar to 10.8 not being fixed.
Though 5.45 is an obvious pointer to circa 10.8-11y Hale cycles it would be hard to make a convincing argument for it being the sole manifestation.
I’m not sure either. The clue I am after is timing/phase, what came first.
I’d look but no data. Climate Explorer defeats me.
No mention of ICOADS.
Detail of srqt(SSN) . Nice demonstration of Ray Tome’s suggestion the sqrt would even out signal variability. It does appear that this better represents the physical cause of what is being monitored by SSN counting.
Now it may be interesting to do the spectrum of that. My guess is that there will be less ‘red noise’ down the low end and some existing peaks with a stronger showing.
http://climexp.knmi.nl/select.cgi?id=someone@somewhere&field=coads_sst 😉
Sometimes it is hard to find something you know it there. If I get stuck I get Google to find it for me.
Ah good. This crazy inside out, nothing ever straight drives me nuts. Can it be trusted?
Good grief, what are they on?
“X axis: whole world in 180 2.00° steps, first point at 1.00° E, last point at 359.00° E”
KNMI point at NOAA who don’t say either, can’t trust the import is right anyway.
and “Degrees west or south are indicated by a negative number.”
But there is no West.
So zero is the Greenwich meridian or the date line?
A point incidentally is you can’t just average grid cells but no mention of that. Cells are variable area, must be a cosine weighted mean. Is it? (this stuff is why I usually do my own data extractions, can’t trust anything)
Yes, their coord input is not the easiest to deal with. All data scans eastwards. So if you want 10W to 20W do -20E to -10E or else you get most of the world included. 😉
“Cells are variable area, must be a cosine weighted” . I share your mistrust but I hope they are not that bad. I had some communication with Geert Van Oldenborgh who maintains it and he seems competent.
Looking at this again, 11y is present , though it’s a junior partner to the lunar 9y cycle.
N. Atl: correlation is reduced around 30y, peaks around 57, also low at 81 and 120. Beats of 9 and 11, with 9 dominant. Est 2:1 ratio in magnitude.
similar pattern in S.Pacific but some other element also playing.
Another very stable period seems to be coming out of this, remarkably similar in all basins:
All 53 month , amplitude 1.3 to 1.35 K/ca . Have not got relative phase yet.
Means of cycle between 0.35 and 0.65 K/ca overall warming dT/dt
4.42 years any obvious significance?
Phase plot. http://i48.tinypic.com/2rfpij7.png
Three major basins showing same periodicity of 53m
S.A – N.A – S.P lag of 5.3 months at each step.
N.Pac has slightly longer period 55m and is perturbed by WWII glitch. May need to take a subset.
4.42 years any obvious significance?
Possily also ill-defined period of circum polar current.
Greg, the lunar apse cycle is 8.85 years — see for example work by Ian Wilson & others.
I give up on kmli. Asked for data, jumped sideways, their autofilename differed and the file note inside made no sense.
Instead I decided to do some bad hacks, copy and paste. Bit of luck, shortly afterwards gridded hadsst3 is now in a common format in the sqlite database here. Yank official global and compare, close enough. (I use straight math on all gridded, they do statistics stuff)
I couldn’t remember what the lat/long function does so I have to work it out. Wants a list of co-ords. Wrote a quickie to make a list from area. Try it, get an error followed by mean data processing
On looking to see why I followed the trail, where to my complete surprise it points at plot. Go look, huh? Must have come from my code, not that I remember doing this.
Got a starting point. Need to dig more and find out if mean calc uses this data.
tchannon (January 29, 2013 at 5:47 pm) wrote:
“As I see it the claims of solar/terrestrial linkage are very weak but just one definite would open the door to more. In this case I assume magnetics somehow influence the earth and can be seen globally. I further assume a magnetic effect would be polar, precisely where there is a lot of unknown involving magnetics indirectly. Since the poles primarily shed heat this implies a modulation of heat loss.”
Tim, the weak thing I see is conventional mainstream exploratory vision.
Our fundamentally differing views on solar-terrestrial relations are almost certainly unbridgeable. I mention this to clarify for the record, but I’m content to skip right past it and make an effort to move forward on small subsets of the problem where we might eventually be able to find some semblance of agreement. You’ve brought something very important to the discussion, so I’m going to consider why you’re thinking what you’re thinking and what you might do next to explore more deeply your line of speculation.
Since you can’t deduce key locations of important dynamics from a global average time series, how might you further explore your suspicions about heat leaking from the poles? Well, the strongest geomagnetic manifestation of the Hale cycle occurs in fall & spring. So you could maybe look at spring & fall OLR at the poles, for just one example …And there are plenty of other things you could do. I’m sure you have plenty of ideas…
Something else to consider with due care is that the Hale amplitudes in the Tsonis framework are much higher than those in global average temperature. Global average temperature is just one of an infinite number of statistics. What reason do we have to believe that it should be the most optimal &/or sensitive marker? The evidence suggests its structure only allows it to capture a weak 1-dimensional shadow of something more multidimensional. Recall that Tsonis’ conceptualization is based on an infinite network of coupled oscillators. I encourage you to consider aggregate constraints on downscale spatiotemporal turbulence in such a bounded network. I accept & respect that you might decide to leave this sort of work to others, given that your very fine tools are designed for completely different types of jobs. (We might as well collectively move with an efficient division of labor.)
If you decide to explore your line of speculation further, I hope you’ll keep us updated. I suspect many Talkshop readers will be appreciative if you do.
In closing, I hope I can presume to speak for all Talkshop readers when I say:
Thanks for bringing something very important to the table.
Paul Vaughan says:
January 30, 2013 at 2:42 am
Greg, the lunar apse cycle is 8.85 years — see for example work by Ian Wilson & others.
Thank you, I was sure someone here would know what it corresponded to.
I never cease to be amazed that this kind of pattern can be extracted from such crudely obtained data with such marked variability in spacial coverage.
I’ve been convinced for some time that El Nino cycles are celestial in origin. That variations in the thermocline depth in equatorial Pacific are inertial rather than wind driven which seems to be the current explanation.
Here I’ve done seasonal average for circa 9 and 11y cycles for S. Atlantic. I’ve flipped one to allow more direct visual comparison, the relative phase in this plot is that for 2009 but is essentially arbitrary, being dependant on where the data starts.
It is evident from the autocorrelation plots that I did, that the shorter cycles I posted earlier do not show up. They are just part of longer the 9 year pattern. The period of the longer repetition shows the short one is just the first harmonic. I need to fit some exact values but it seems to be about half the amplitude of the long period.
Cleanest forms are obtained at 106 and 173m = 8.83 and 11.4y
The origin of the former is fairly obviously lunar apse, the latter is perhaps more intriguing.
It seems rather long to be the solar cycle and yet too short to be Jupiter. Their may still be some perturbing effect of the lunar pattern on this analysis.
I’m more inclined to see this as a direct jovian influence, although it could be jovian dominance of the solar cycle.
Also relative magnitudes , lunar effect is slightly stronger but not by much.
Remembering that I’ve flipped the green ‘jovian’ line , it is somewhat similar to the artificial SSN series Tim posted as the zipped up spreadsheet above.
I will try to find time later to subtract the lunar ‘seasonality’ and see how this leaves the jovian repetition pattern.
PS Tim, from your experience with ephemeris data, what is period of Jupiter’s gravitational influence on Earth?
Don’t recall any difference.
Thanks, but wouldn’t it have to be about a 1/11.8 years less that it’s orbital period?
As I recall the dominant gravitational sum is around 19.8y same as sun. A component of that has limited meaning.
A very quick check, geocentric. jupiter alone distance 11.857y and a touch of 5.9/6y which is in climatic, story there. It’s where the 3 year warm cool comes from. Effect seems to be episodic.
(humour) episodic, variation of certain laws, if it can sod you up it will sometimes (/humour)
J is in opposition every 398.9d and there’s probably no reason why J’s orbit would modulate that, though it would happen roughly a month later each year and possibly interact with Earth seasonal changes, magnetosphere or tides.
Earth’s perigee is in boreal summer. If J is in opposition at this time 6.5 months later E will be closest to J with NH inclined towards it. How often does this cycle reproduce itself, is what I was trying determine.
Greg: j’s orbit has high eccentricity and perigee forms ax pattern in relation to solar cycle. Details later.
I suppose two runs could be done for different points on the earth and take the difference.
Need to be super careful over aliasing.
.
“Need to be super careful over aliasing.”
Yes, this is a concern. I need to eliminate the possibility that this pattern is not due to the prolific use of monthly means. If the sampling period of (sort of) 30.5d is interfering with natural lunar cycles such as 29.5d it could produce a pattern of about 30m . That is dangerously close to twice the frequency I’m seeing.
Now if that is happening, it affects ALL climate data that are being used in monthly average format !
That 8.83y cyclic repetition, that is slightly larger than the solar signal, and as I have shown here is what causes the phase crisis in early 20th c. That causes many to reject the solar hypothesis.
So either there is a real lunar driven infulence on climate that is comparable to the solar signal or there is a universal data processing error that is comparable to the solar signal and is out of phase with it in early 20th c.
Either way this needs to be accounted for.
I need to find some SST data in daily format and do some proper data processing to see if this signal is still present. Maybe some of the satellite data is available in that form.
Any suggestions?
John Christy answers email.
hmm, polar satellite orbits scan swathes and only get a full scan in several orbits. daily is only available in complete latitude bands. Unfortunately this does not allow splitting the difference ocean basins.
I suppose it would be possible to split it if I could get the full dataset but this would substitute weekly cycle for a monthly one. Could be informative but sounds like a lot of effort for something that would not really give a clear answer.
Needs genuine daily dataset.
Greg: You’ll be needing a network of satellites then. I’ll put out an appeal for funding for you. 😉
LOL. There must be some daily data set that could be used to find out whether taking monthly means will create this kind of periodic signal as an alias. It can’t be that hard to find something.
In the mean time I’ve just added Indian ocean to the plot. This is a longitudinal effect. The phase is clearly a function of longitude. Pacific and Indian are nearly opposite and are almost opposite in phase. N.Atl and S. Atl fall inbetween with N.A closer to Pacific, as it is geographically.
With a peak to peak of about 3K/century this is either a significant false signal that needs taking care of, or an important find.
Solar-Terrestrial Magnetic Polarity Weave
Picks up popcorn and smiles.
Satellite data is terrible, think they can walk on water.
Thunking. Has problems but how about trying Argos data?
Here’s a first cut on this using CET data. Comparing a gaussian filtered CET to the calendar month averages.
This does show a suggestion of the what I was suspecting. Ignoring the additional problem of unequal months, a monthly mean is identical to taking the 30d running mean and sampling every 30 days.
As we can see this is letting some high frequency detail through (the classic problem with runny means). It also seems every peak is outside the range of gaussian filtered result.
So does this let through some part of a monthly lunar signal that could be aliasing with the sampling period to produce annual scale pseudo cycles.
1999 and 2000 have a data peak before the centre of the filtered peak, 2003 and 2004 after.
Now that’s just one subjective snippet but it would correspond to the length of cycle I have been picking up.
It is a good example of why you should not use simple averaging on data containing cycle signals shorter or comparable to the averaging period. The average of a sine is only zero if you hit bang on its period. Otherwise, you will reduce it’s magnitude, but the remainder will get through with a totally new (aliased) period.
A non existent periodic signal has been born.
Jumping ahead on this , I suspect this is what is happening in the Hadley regridding algo, which uses spacial running mean “filters” inside its iterative loop that builds their “climatology”. But that’s story for another day.
Figure 1
There might be more.
comparing monthly average to 30d gaussian filtred CET:
difference : (monthly mean – 30d gaussian) and the 30d gauss are further both passed through a year long gaussian: 12m for the monthly subsampled data and 365d for the gaussian filtered daily data.
This shows the extent to which such ‘defective’ processing will affect the annual time series. The difference ‘error’ signal is divided by 100.
We can see in this case the error is of the order of 2 or 3%. Sometimes in phase or out of phase and with a marked century scale effect.
If I can reproduce this small scale of error on marine data that may be more affected by monthly tides, I would conclude the 4.42y pattern was significant. However, it is clear that this kind of data processing sloppiness can have decadal or even centennial effects.
The magnitude will depend on the magnitude of the monthly variations in the original data. I would guess that central England air temps bear little witness to the phase of the various lunar cycles.
It remains to be seen whether the same can be said for the oceans.
Greg, awkward situation. At the moment I don’t want to discuss CET on the public record. Something you are doing might be somewhat interesting for reasons you cannot imagine.
If you can figure out how to contact me I can explain enough and probably supply a completely different unpublished daily dataset which has a known parentage. Use that instead. Long data is very rare, valid long data is spaghetti trees. This comment is subject to vanishing.
[Reply] Use me as go-between if you wish. TB
TB, feel free to give Tim my e-mail.
Tim. I was not intending to ‘discuss’ CET as such , it was just an example daily dataset (it could have been stock prices of bananas).
I wanted to demonstrate the kind of errors that can happen through improper data processing, in particular, sub-sampling data without the necessary filtering to prevent aliasing.
It showed it can lead to multi-decadal shifts rather than all averaging out as some people seem to imagine.
The effect will be much more significant if the data set concerned has a significant periodic component of about the period of the re-sampling. This is very likely to be the case in SST.
Now I think I need to find at least a 30y record of daily SST of some kind to look at.
just looking back at that last plot and seeing what CET does, it shows a surprinsingly linear, long term variation rise since 1860 with a temporary almost binary state change between 1960 and 1980.
The 1960 drop is coincident with one of the anomalous diviations between Atlantic SST and accumulated cyclone energy.
We can also see then previous drop in CET was in 1950 , not 1945 as per Folland’s Folley and successive attempts by Hadely to maintain this error in the SST record.
I’ve reworked the phase plot of the circa 4.4y cycle I found. In checking I found I had not correctly synchronised to the common end of point of the time series.
The major difference is that the N. Atlantic is now about 180 out of phase with S. Atlantic. In this plot I’ve inverted it to make visual comparison a bit easier.
The bottom line is much the same: a cycle that is very close to sinusoidal in form is propagating around the globe. This seem similar to the ill-defined circum-polar wave, except that it is not just regional but a global phenomenon.
This may also account for the unexplained idea that the El Nino 3.4 region is somehow driving global climate. It may simple be that it is in synch with this global cycle.
The joker in the pack is N. Pacific.
The reason I have left it out so far is that the post WWII ICOADS record has a long period with little of the regular cycle pattern found in all other basins and in N. P. before the war.
The Hadley reprocessed N. Pacific does have cycles but do not have much confidence that they are not spurious cycles generated by the regridding algo rather than a true climate signal. I will want to see this pattern still present in the other basins in the Hadley data first.
That will be an interesting test of how damage the Met Office processing does to the structure of the data.
The length of the cycle is somewhat different from the legend, which show the length of the window that gave cleanest results. Subsequent fitting of a sine plus 2nd and 3rd harmonic plus a constant (dT/dt) usually finds the cycle to be a few percent different from the window length (necessarily an integral number of months).
actual cycle lengths were: 51.77, 52.2, 52, 53 months for NA SA SP and Indian, respectively.
All these cycles were on top of a constant dT/dt term of between 0.55 and 0.65 K/century . This represents the average rate of change over the full data period (some ocean’s records are longer in terms of reliable pre 1990 data).
Finally the glitch, which is most noticeable at the beginning of N.Atl and S. Pacific, is the notorious post war jump, that gets divided down by averaging the repeated cycles.
If one were to accept that there was not some anomalous climatic change at just the moment the US Navy demobilised, this analyse method may provide a means of estimating the correction needed to remove the jump and restore the cyclic pattern.
This may be more appropriate than hand-waving assumptions about buckets and engine intakes.
Noting that the magnitude is about the same in each basin, it is interesting to note that the four shown here would pretty much cancel out in a global average temperature. North and South Atlantic are out of phase seasonally and S Pacific and Indian are geographically opposed.
This would leave N. Pacific as the only basin which would manifest in a global average. If this affect also applies to other cycles it would explain the idea the PDO (which is the difference of N.P from the global average plus some EOF voodoo) is an important global climate indicator.
This also underlines the point the Paul Vaughan repeatedly hammers home, the need to look at spatio-temporal change not just global averages.
I am not able to comment usefully, I don’t understand what you are doing, the plots.
Sorry, I thought I had roughed out what I was doing above. Here’s more detailed account of the processing:
Having noted that lagged autocorrelation of the monthly time series of SST in each basin indicated strong regular repetitive patterns were present in the data, I decided to investigate the form of repetitive patterns, since autocorrelation shows repetition but does not tell much about form.
The plots I have been doing are similar to what is current practice in climatology to remove the average annual seasonal variation to create “anomalies”, ie over a window of 12 months the temp for that month is averaged over a certain period. This average pattern is then subtracted repetitively from the full TS.
I have essentially been doing the same thing with a “season” of , for example, 52m. In fact I’ve scanned from 30m to 140m. What I am plotting is average “seasonal” cycle, similar to the 12m seasonal average that climatologists prefer because they can’t use filters 😉
Several patterns emerge , this one seems to be common to all four basins shown.
All this was done on ‘rectangular’ long/lat blocks of ICOADS data representing each basin, that has been filtered with a 12m 3-sigma gaussian. I’m looking at rate of change, in part, because this reduces the WWII problem to glitches rather than step changes.
I’m still wary of the possibility of aliasing introduces by the use of monthly averages.
Taking the average length of the monthly average to be 30.4 days and a possible physical lunar effect of 29.53d , that could alias at around 30 months. There is a cycle in small 30’s but it is almost an order smaller than what I’m seeing here.
That does not remove the possibility that this is something else aliasing since anything shorter than the sampling period could produce some spurious cycle.
That is why I was looking for a daily time series from somewhere that I could process with correct filtering to see whether any such signal was still present.
http://www.ncdc.noaa.gov/teleconnections/enso/indicators/sst.php
El Niño (La Niña) is a phenomenon in the equatorial Pacific Ocean characterized by a five consecutive 3-month running mean of sea surface temperature (SST) anomalies in the Niño 3.4 region that is above (below) the threshold of +0.5°C (-0.5°C). This standard of measure is known as the Oceanic Niño Index (ONI).
See what I mean about not being able to use filters. Not one distorting 3m running mean but FIVE. Do you think they’d learn enough to use different lengths of runny means instead of compounding the error? All this done on top of monthly averages of “anomalies” of monthly means…
No wonder the index looks like crap, it is. What chance that any signal that may be in the SST to start with is still visible after all that botched processing?
“Warm and cold phases are defined as a minimum of five consecutive 3-month running mean of SST anomalies (ERSST.v3, 1971-2000 base period) in the Niño 3.4 region surpassing a threshold of +/- 0.5°C.”
OK, they are not using five consecutive running means, they just love saying running mean so much they can’t stop themselves. What they meant is that they use five consecutive 3-month means.
They are using ONE running mean, and test to see whether it remains higher than the threshold for more than 5 months.
Now anomalies are not filters. They will reduce the seasonal signal but because all years are not equal WILL be significant residuals of an annual variation in any one year. This WILL interact with three month running mean.
3m running means of monthly anomalies of circa 30.5d means … all without the slightest attempt at anti-alias filtering. What a mess.
They’d to better to call this the El NONO index. An example to students of how not to do data processing.
But I digress. Still looking for daily SST time series.
Looking at Nino 3.4 region 45m and 58m are very strong and clear, not so much 53m.
45 and 91 give fitted periods in very good agreement : 91*0.992/2=45.14 cf 45*1.0037=41.17
=3.76 years.
Also found in S. Pacific and Indian, not obvious in S. Atl.
Coincident phases of main peak may suggest common cause rather than Nino 3.4 running world climate.
Lag on minor peak is about 3mth.
Ah! This may do the job: http://www.pmel.noaa.gov/tao/data_deliv/frames/main.html
Good find there boy.
This is where working in a field pays off although I can recall plenty of cases where I was unaware of lots in a field.
At last ! A bit short , due to float changes and discontinuities this was the the longest continuous segment I could find directly.
This plot looks at the variation of the calendar monthly mean temperature form a monthly resampling of gaussian filtered temp data. The variance (date minus the average of the whole set) “anomaly” in climate talk, of the correctly resampled data is shown since the absolute average is not of interest.
This enables us to see the proportional change made by taking monthly averages.
Firstly individual monthly means can vary by as much as 30% more than the correctly filtered data, this is typical of runny means letting through h.f. that their users often assume they have taken out.
It would also seem that even when filtered with a light 2m guassian this error is still tending to exaggerate the variance by around 5% compared to correct resampling.
While on this data the effect seems fairly constant , sometimes like 2011, it is not. This means it will be injecting some error in to even the decadal time series. The good news is that this does not seem to be a large effect for this data.
The years of strong peaks in the correctly resampled data does seem to be consistent with the 43m cycle I detected in global monthly data series.
So this rather limited check does suggest that what I found is a true signal , not a result of aliasing.
Just to clarify the processing: monthly average is logged at the middle of the month’s number, ie 1.5/12 for February.
The gaussian filtered data is logged at day-of-year/366 and resampled at the date closest to the date of the monthly mean.
I see what you mean, I tend to forget the, lets invent one, the Gregorian shuffle.
If the data was valid (fs/2 doesn’t happen with these people), which it isn’t, it would be simple enough to come up with a time correcting filter tailored to the problem.
Note: Clarifying for general readers, assuming I understand what Greg is talking about.
In the Meteorological and Climatic fields crude maths technique are used reducing the number of samples in a dataset, such as daily data into monthly data. Unfortunately calendar months vary in length and that means the time between monthly samples varies slightly. This is bad.
If it would help I could probably write an article about this some time, there will though I assume already be published works on this somewhere. The main thing is be aware of the pitfall.
“Unfortunately calendar months vary in length and that means the time between monthly samples varies slightly. This is bad.”
Inequality of calendar months adds some extra noise. It is not the key point I’m making.
The worry is, though monthly averages seem to be an innocent and convenient “round number”, the length is not arbitrary. We have months because , historically, the lunar cycle was important. Our calendar month length is based roughly on the visible lunar cycle of 29.5 days.
However, moon affects tides and oceans. If there is any lunar signal included in the temperature record and we sub-sample at 30 or 30.5 d intervals this will ‘alias’ the 29.5d signal to appear as something much longer. It will _appear_ as a cycle or 3 or more years. My concern was, the cycles I had detected may be just such an artefact of improper data sampling.
The correct solution to aliasing is to filter out anything shorter than the re-sampling period, before doing the sampling.
Taking an average is what Joe Bloggs does, to reduce noise and reduce the number of data points, if he has too many. However, this is mathematically identical to using a running mean as the pre-filter. Since RM is a very bad filter , it is a very bad way of doing it.
Not only does RM let a lot of high frequency through it actually injects signals that were not there because it INVERTS certain frequencies.
Now if all this is happening before you even do any further processing you risk being lead astray.
I’m reassured by the TOA float data since even with more rigorous processing the strong peaks seem still to be present at about the right interval and are not seen in the deviation caused by the monthly means.
That leads me to think that global scale cycles I detected in ICOADS are real.
After sending a N. Atl extract of ICOADS to Tim, he provided me with chirp analysis periodogram which showed the strongest peak was around 8.9 years with slightly lower peaks around 11.1 and 13.4 y
I went looking for the form of a repetitive pattern around 13y and found this.
Fitted f7(x) is the base period plus harmonics up to 7th (5th has been notably absent so far).
Rember this is rate of change. What this appears to show is a repetitive cooling event every 13.5 years with what in engineering terms is a strong negative feedback and under-damped response showing considerable ringing and overshoot.
peak to peak = 8.065 K/century
window len=155; fitted period= 161.909m = 13.492a
The initial glitch seems to be due to the window length not being the same as the period. It shows the wrap-around change-over.
We can also see that with slightly heavier filtering this would be another case of bump-bump-dip.
There are several similar patterns around this base period, so further investigation is needed to discriminate , I post this one because it was very close to the peak in the periodogram provided by Tim but the peak was broad so this is only exploratory not conclusive.
The slightly shorter periods have larger ringing and more even decay but the form of a neg spike with strong negative feedback response is common to them all.
[…] data used was published as spreadsheet two of Talkshop article Met Office Hadcrut 4: solar linkage (supplemented copy linked at the end of the current blog […]