I thought it would be useful to look at and show some of the problems with UK climatic weather sites. A primary one is WMO 003377, located at RAF Waddington.
Figure 1
Figure 1 shows the new Google facility for different image dates. I’ve left the date selector slider in view, top left.
The yellow “pin” is pointing at the met enclosure. There is obvious construction work going on which became tarmac.
- top 2001
- centre 2005
- bottom 2007
From the RAF web page
“RAF Waddington – Met Office
The Met Office, situated within RAF Waddington, plays a vital role within our organisation.
The Met Office is a world-leading provider of environmental and weather-related services in the UK and around the world. It operates 24/7, 365 days of the year, providing a weather-warning service for the UK and – via the Foreign and Commonwealth Office – for UK citizens abroad. The Met Office’s Hadley Centre is at the forefront of international climate research.
The Met Office’s role, within RAF Waddington, is to understand the science behind the weather and the environment; and so to provide forecasts and information which allow the integral components on station to make informed decisions about how best to capitalise on, adapt to, or mitigate the impacts of weather and climate change”
– http://www.raf.mod.uk/rafwaddington/newsweather/rafwaddingtonmetoffice.cfm
The Met Office doesn’t provide (and probably doesn’t know) the exact co-ordinates of their enclosures so it is necessary to play hunt the thimble. For these particular sites with the additional problem of restricted military areas. To the best of my knowledge this is the met enclosure. A search of the later image shows no new enclosure so I assume this remains in use. A twist comes later, ridiculous data.
Overseas readers should note the UK Met Office are part of the Ministry of Defence, although the dislike of this by ministers tends to leak. Fundamentally the weather service is only there for the military and is much the same as Air Traffic Control which uses shared services, common computers for obvious practical reasons (each needs to know the location of all flights, shared airspace), but separate operators.
There is nothing wrong with locating expensive equipment is areas where vandalism etc. isn’t going to happen yet the other side of the coin is a lack of public scrutiny, daylight to keep things honest.
Location
Figure 2
Just to the north is the city of Lincoln (nice page done by the university), fairly flat, North Sea is not so far to the east. Quilt of farming.
Yellow marker on the met enclosure. Prevailing wind in the UK is from the south west, with small town grown up that way. Effective heating of buildings only came during the 1970s, 80s.
This looks a typical Met Office enclosure co-ordinates 53.175364 -0.523324
(no easy way to give a direct link here, copy and paste should work, GE, various browser maps)
History
Official http://www.raf.mod.uk/rafwaddington/aboutus/history.cfm
Traditionally, this started as a aerodrome during WWII, the enclosures were close to Ops, control tower, an easy walk for whoever was on duty. In this case it is a highly developed site, the control tower has been relocated to the centre. Met site is an AWS, power and comms probably along the concrete covered conduit, all standard stuff. The instruments can be taken as pretty good today but historically that is a whole different matter. The Met Office moved onto remote linked during the 1980s, early 90s and this often shows in the data as a change in character. (and in fingerprinting, not exact science)
I recall showing some history of Waddington but perhaps on a different web site. Found some air shots from way back.
Briefly, WWII, then very active in several major guises including a base for nuclear armed V-bombers during the cold war. Victor, Vulcan, etc. A static display Vulcan can be seen on the other side of the airfield. May as well say it, various munitions shelters are present. (see much the same near Jerusalem) Dotted around are also the usual older dispersal emplacements, makes it harder for an enemy to take out aircraft on the ground.
No history of runway, apron or building development seems to exist… of course not. Fairly obviously this has been piecemeal, including extensions of runways and aprons.
Almost certainly the met enclosure will have been relocated, hopefully done properly with bridging but the Met Office are coy about this stuff. I am certain that for some airfield sites this is not true and similarly moves have not been done. There are rational explanations
Historic control tower photos Recent and current with dates, 2001 again.
For those interested, wartime memories.
Just missed the 2012 international airshow. There is a photo gallery…
Found a picture on disk here. Image from 1945
Figure 3
Back to business.
Data
An objective here is seeing whether a change close to an enclosure shows in the data. In this case I suspect not.
As an international climatic station it ought to be in GCHCN and it is.
I have older monthly data, daily? I checked, datafile exists.
Best by ftp, anonymous is fine.
ftp.ncdc.noaa.gov off here /pub/data/ghcn/daily
Readme ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/readme.txt
Specific file ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/all/UK000003377.dly
I grabbed and had a quick look, nice, 1949 through recent 2012, however, experience tells me to beware chickens. Data is often in a mess.
Comprises lines for a month, precip, tmin, tmax and other stuff.
Got a decode. On first look ???
A couple months of tmax is missing 1973, which makes no sense with tmin present. ghcn at it again.
Look at the end, huh? Eyeball raw data, this turns out to be an unholy mess. 
6th June 2001 through 30th June 2001 missing and all that follows is a mess of missing data and corrupt data. Such as zeros marked as good.
Also, the data source is random, marked as European source or estimate. Basically is a complete screwup.
This needs reporting but in my experience it is pointless. ghcn is notoriously poor yet the UK Met Office can’t have checked their own data is being published correctly either.
Typical, my decode
y, m, d, tmin, tmax, tmean
2011 6 28 na na na
2011 6 29 8.8 18.5 13.65
2011 6 30 8.7 18.5 13.6
2011 7 1 na 20.6 na
2011 7 2 na 21.3 na
2011 7 3 9.6 24.4 17
2011 7 4 12.7 24 18.35
2011 7 5 11.2 25.1 18.15
2011 7 6 12.7 20.4 16.55
2011 7 7 na 21 na
2011 7 8 11 na na
2011 7 9 10.8 22 16.4
2011 7 10 11.9 na na
2011 7 11 11.7 21.2 16.45
Monthly? Data is there GHCN V1 V2 and V3 but I haven’t coded up a decode for V3 and it takes time anyway doing V2.
I have the Met Office 2010 data release on file. Monthly 1951 through end 2009 with no missing data.
Quick look
Figure 5
Figure 6
Figure 6 is filtered @ 2 years giving valid annual. Nothing showing 2001.
[UPDATE 2nd August]
Writing articles like this one take a lot of time and effort because they are based on new work. I knew I had datafiles for this station which I have now extracted from the archives here.
These are monthly data only. Several surprises coming.
Figure 7
Nothing surprising in GHCN-V1
Figure 8
You have never seen this before although this particular data is of no interest as such. A database was part of foia 2009. I am far more interested in information than social tittle tattle of emails words. There is a temperature database in there and I decoded it. A little later I told Warwick Hughes and he declared it was the missing Jones database.
The database is highly significant and remains only trial explored. Some interesting edits were found (not sinister) but perhaps most critically the easily overlooked Fortran style mask layer, which was found to be in a surprising state.
I’ll skip GHCN-V2 for the moment which is highly interesting and talk about the Met Office version first which is Figure 5 & 6 above. If you actually look at their released data you will find a mystery.
Here is my decode and headings. Notice YearJones.
The Jones column is mentioned in the notes as ignore, internal use.
It actually pretty much links back into the foia database, or more likely a later version of it, a database which apparently does not exist. You’d have to look to see what I mean. So far as I can tell the only meaning is a commonality between Met Office / Hadley / CRU with current data, they are not independent.
Moving on to GHCN-V2
Figure 9
V2 is the better data because it is more honest, it suggests changes in data by month gaps but very critically it separates WMO suffix stations, which are often in very different locations. In this case it reveals there is suffix 000 and 001. (usually 000 is the master station).
What we see here is a change major enough to warrant data bridging, running old and new stations in parallel. I am very nervous of these, especially perfect matches, sorry, doesn’t happen.
Two possibilities come to mind
- There was a station move to the new location
- The Met Office switched from mercury to AWS
Notice too there is a data break 2001, yet the Met Office data shows no gap. Did they make it up? If they have data why didn’t they distribute it to NOAA etc? Gaps, breaks, changes are very important as markers, warnings.
GHCN-V3 is much smaller, they have hidden suffix stations. Some are even at different altitudes! (lapse rate)
Figure 10
Here is experimental dataset fingerprinting where the general technique has uncovered station problems before. As such it means little but points at where to look for trouble. An ideal dataset would show three straight lines except A which will wobble regularly with the weather.
I conclude that nothing is showing in monthly data to do with the 2001 change.
There are likely to be serious problems circa 1964/5 and 1986/7
It wasn’t the winter of 1963 extreme as it was, doesn’t show. What happened?
The 1980s matter seems to have an obvious answer, regardless of bridging there really was a change in the data. This seems to co-incide with a near step in the longer term. I am very suspicious of these dataset butt joins and faking up, another name for homogenising.
Finally…
Figure 11
What the heck is going on here? All the later data is different from GHCN-V1. This is direct evidence of data change.
And that is all I know so far.
[/UPDATE 2nd August]
Not provided any data, ask if you want.
Unprocessed datasets.
As OpenOffice .odt (change suffix to .ods if needed)
Any suggestions on where to go from here or help offered? I suspect this is one to drop.
Posted by co-moderator
As a service to the general reader, could you clarify what role the weather station plays in compiling UK (and perhaps international) statistics?
Other than that, it looks like a good example of the carelessness with which weather data that is fed into computer models is compiled. I can’t think of any other branch of science where this kind of sloppiness would be even remotely acceptable.
Thanks for your work on this.
On eyeballing Fig. 6 there was a step increase in 1989-90, with only a very slight incline since. I would guess that a local environment change at the time accounts for the step.
Nice catch, a couple of thoughts from this. I just raised the question of whether we should have another go at something similar to Watts site review on UK stations on another website. Another poster indicated that this had been attempted but there were access issues. It occurs to me that this predated Google earth. Perhaps we could do a sweep using Google earth to identify possible UHI impacts on uk data?
Also a number of RAF stations are being mothballed Witterring, Cottesmore, etc, it would be interesting to review their stations data to see if there is a corresponding drop off in the temps following their fall in usage?
johanna,
Afraid I know very little, the great cloak of never specifically say is difficult to work around.
For the UK there are many Met Office stations in what seem to be various networks, many of which do not seem to be documented until some “record” or other makes it useful. The stations used for the public Met Office web site are the majors as far as they are concerned.
As part of an earlier work I put in many hours (whilst I was too unwell to do much else) using Google, Microsoft and other resources to locate the probable actual station. With a few other people visiting the site and photographed them. Scotland poses problems, poor coverage, weather, lack of habitation interest. Northern Ireland, military block on high res.
What does the Met Office use for “climate”? I don’t know.
CET gets used but I have severe things to reveal about that soon.
Internationally the stations used pose a problem. There are WMO lists but it is a mess, particularly how suffix stations are confused with primary (suffix -000) stations, with major location differences. GHCN V3 fakes this up by combining without making this clear (why V3 data is much smaller than V2)
The list of stations shown by the Met Office in the 2010 data release of monthly is a good bet to contain the UK stations used internationally. Waddington is one.
Most of the often mentioned record hot sites in the UK are not international sites, eg. Heathrow, Southend, Gravesend, park next to Westminster Central London (never says central London).
An obvious major problem for the Met Office is government reducing funding for military and other government premises. This country used to have an amazing number if airfields, a result of two world wars and a large government. As site close there is then change of use and site security problems, so met sites close or are relocated. In some cases I found some dire things, such as a good location turning into jammed next to a road, a favourite.
Guam,
I suspect usage traffic has a minor effect. Possibly more critical (which I forgot to write up) is how runway/apron extensions done in a panic for a war (such as the two Gulf wars) preclude met station bridging, leaving either do nothing, leave it, or bridge to a new station and fake it up.
There is no chance of getting access to detail Met Office data. (who can say if it is right anyway, seen too many “official” records which are plain wrong)
More stations? Dozens I could show.
I’ve added my corrections to this huge worldwide resource, here is Waddington, as was and there is a correction. Letters are international aerodrome call sign.
http://weather.gladstonefamily.net/site/EGXW
Search engine can handle this, maybe add a site name
site:weather.gladstonefamily.net
Makes the shoddy US records look pristine! The WWII development and subsequent abandonment and neglect seem to have knocked station record continuity and quality into a cocked hat.
That Waddington trend line suggests a novel kind of correlation: match with the number of tons of concrete and asphalt laid within a 2 mile radius over time!
Perfectly reasonable to suppose the record is fair. However… a major update is coming where I start to reveal an iceburg.
Emailed NSDC reporting the fault.
Tim,
Are you starting a UK version of Surface Stations quality audit?
For this RAF station, the GHCN data is appalling with basically nothing worth having past 2001.
Any chance of sharing the Met Office data?
I was helping what died. Watts showed zero interest or response either.
This would be a very difficult task for the UK, essentially impossible without the co-operation of the government.
All data obtainable by the public. I have it decoded into the form I use. (portable)
Put this up in due course, reason for a delay will become clear.
Huge update to article.
When do you suppose they began watering and mowing the new lawn?
Tim,
The GHCN dataset has lines that should hold Tmax missing and other places where there is either or Tmax/Tmin but not both. It’s a complete mess! Take a look around 1973 half way through the year [line 885].
Update, added file with unprocessed datasets, contains 5 versions. All unofficial decodes.
http://www.metoffice.gov.uk/public/weather/observations/waddington?tab=last24hours
They do produce hourly observations I wonder if they keep this data.
Yes.
Looks like there is a common warming of almost all current datasets relative to GHCN-V1.
I wondered about trying to programming a compare of some kind. Looks a very tough problems for a variety of reasons.
Dataset commonality is limited.
Missing data tends to be more of a mismatch.
I think this would be a major exercise taking at least weeks.
Seems reasonable others will have already done this.
Even if found to be so it does not hard prove anything, for that we need either direct admission or an inviolate independent data in sufficient cases.
===
Did some software tweaking. Never got around to being able to script compound processing, sources is already complex enough although provision for embedding Lua is there (and tested). Done a quick workaround, can suppress a header, enough to serial chain processing via files, making it practical to fingerprint files quickly.
Needs time experimenting but to date has been successful in various cases, pointing me to problems as well as confirming what look to be clean datasets.
And it works. Same settings as previously.
This is a dataset I consider unusually clean. Chico farm was an agricultural station over many years but different people took over and the met station degenerated. (as I understand it, and yes this is near Anthony Watts, pure chance)
B and C scales are deliberately the same as same triplet shown for Waddington, A is almost the same, no significance.
I don’t need to point out this is chalk and cheese.
The Chico record when looked at normally seems to show the features of US temperatures over this period, or at least to reputed unbent ones, including warm 1930s.
I live about 10 miles from Waddington.
It was opened in November 1916, grass at the time, obviously!
It’s concrete runways were built during the spring/summer/autumn of 1943.
Further reconstruction occured in August ’53 to May ’55, in preparation for use as V-bomber base.
It was closed when the Vulcan bomber force disbanded in 1984, then reopening in 1991 when the Boeing Sentries were added to the RAF’s inventory.
Might be worth comparing the station’s temp record to that of RAF Cranwell, which is also about 10-15 miles from it.
Cranwell is not in any dataset I have although it is in a comprehensive WMO station list. 03379
Scampton is the closest I have identified, Coningsby is a little further.
Found it, about the same distance as Coningsby. Also spotted the met. enclosure.
Problem is what is the object next to it? On using Google history, ouch, the clowns have put a high security fence around this object, much too close to the Stevenson screen.
53.031153° -0.503529°
Coningsby is poorly located but what fun… control tower there, pile of rubble, Wot tower?
Moved to a new location not far away.
53.093881° -0.172934°
I’ll be driving past Coningsby at the end of this month if you want some photos.
The nearest public access point is 0.9km one way and >2km the other, nothing to be gained.
Hi Guys,
Any chance of doing a similar analysis for the Weather Generator validation sites –
Details of sites here -
http://ukclimateprojections.defra.gov.uk/media.jsp?mediaid=87941&filetype=pdf
Mentioned in the Climategate emails here -
http://foia2011.org/index.php?id=2445
I can’t work out what it is either! Seems to have no vertical elevation on Google map’s groundview. It’s too far from the A17 to see either.
I picked Cranwell, because it’s the closest weather station to Grantham, on Wolfram Alpha.
http://www.wolframalpha.com/input/?i=weather+grantham
You’ll see jack-all of the weather station at Coningsby, from any publically accessible site, the public roads are well away and so’s the Battle of Britain memorial Flight’s hanger. But do drop in there, a propellor-head’s delight!
Marion,
Some of those I could, some not.
Seems a good suggestion for a subset. When? I tend to have a number of items progressing.
Marion,
Forget it, they are using old data. I wouldn’t trust their data over that period either!
“The performance of the Weather Generator in reproducing observed data
(1961–1990) is assessed by using half monthly and seasonal plots of the weather …”
Have you seen or know the history of eg. Ringway? (looks like the met enclosure has moved again, but was behind jet blast screens, if they worked, that bad) Been massive developments at some of those sites. All probably have had equipments changes. **for the period 1961..1990**
Some of those stations are defunct anyway.
Tim,
Thanks, simply what I expected – for ‘validation’ one would expect the Met Office to use the best sites/data possible.
Yet they have used sites such as these! Why?
And this is the data that our Weather Generator has been calibrated on.
Seems to me it’s far more about ‘telling a story’ than trying to achieve accurate forecasts.
“I can’t overstate the HUGE amount of political interest in the project as a message that the Government can give on climate change to help them tell their story. They want the story to be a very strong one and don’t want to be made to look foolish.”
http://foia2011.org/index.php?id=2445
No email reply about the corrupt data, must be a week now.
Probably doesn’t matter for this specific case but there is a larger issue: Met Office incompetence where the body either knows or ought to know its data is corrupt.
Ignore, make a mental note and move on.
[...] Waddington has been the subject of a previous post on dubious climatic station choice. See here which shows some recent [...]