Climategate’s Harry_Read_Me.txt: We All Really Should
Oh bum. But, but.. how? I know we do muck around with the header and start/end years, but still..
In the upside-down world of Mark and Tim, the numbers of stations contributing to each cell during the gridding operation are calculated not in the IDL gridding program – oh, no! – but in anomdtb! ..well that was, erhhh.. ‘interesting’…So there is no guarantee that the station number files, which are produced *independently* by anomdtb, will reflect what actually happened!!
I am seriously worried that our flagship gridded data product is produced by Delaunay triangulation – apparently linear as well. As far as I can see, this renders the station counts totally meaningless. It also means that we cannot say exactly how the gridded data is arrived at from a statistical perspective – since we’re using an off-the-shelf product that isn’t documented sufficiently to say that. Why this wasn’t coded up in Fortran I don’t know – time pressures perhaps? Was too much effort expended on homogenisation, that there wasn’t enough time to write a gridding procedure? Of course, it’s too late for me to fix it too. Meh.
CRU NEEDS A DATA MANAGER.
Not only do both databases have unnecessary duplicates, introduced for external mapping purposes by the look of it, but the ‘main’ stations (2 and 4) have different station name & country. In fact one of the country names is illegal! Dealing with things like this cannot be automated as they’re the results of non-automatic decisions.
What a bloody mess.
Now looking at the dates.. something bad has happened, hasn’t it. COBAR AIRPORT AWS cannot start in 1962, it didn’t open until 1993! Looking at the data – the COBAR station 1962-2004 seems to be an exact copy of the COBAR AIRPORT AWS station 1962-2004. And wouldn’t you know it, the data for this station has missing data between 12/92 and 12/99 inclusive. So I reckon it’s the old FORREST AERO station (WMO 9464600, .au ID 11004), with the new Australian bulletin updates tacked on (hence starting in 2000) So.. do I split off the 2000-present data to a new station with the new number, or accept that whoever joined them (Dave?) looked into it and decided it would be OK? The BOM website says they’re 800m apart.
Hope that’s right..
All 115 refs now matched in the TMin database. Confidence in the fidelity of the Australian station in the database drastically reduced. Likelihood of invalid merging of Australian stations high. Let’s go..
getting seriously fed up with the state of the Australian data. so many new stations have been introduced, so many false references.. so many changes that aren’t documented. Every time a cloud forms I’m presented with a bewildering selection of similar-sounding sites, some with references, some with WMO codes, and some with both. And if I look up the station metadata with one of the local references, chances are the WMO code will be wrong (another station will have it) and the lat/lon will be wrong too.
I am very sorry to report that the rest of the databases seem to be in nearly as poor a state as Australia was. There are hundreds if not thousands of pairs of dummy stations, one with no WMO and one with, usually overlapping and with the same station name and very similar coordinates. I know it could be old and new stations, but why such large overlaps if that’s the case? Aarrggghhh! There truly is no end in sight.
I honestly have no idea what to do here. and there are countless others of equal bafflingness.
I suspected a couple of stations were being counted twice, so using ‘comm’ I looked for identical headers. Unfortunately there weren’t any!! So I have invented two stations, hmm.
I have to admit, I still don’t understand secondary parameter generation. I’ve read the papers, and the miniscule amount of ‘Read Me’ documentation, and it just doesn’t make sense.
As I was examining the vap database, I noticed there was a ‘wet’ database. Could I not use that to assist with rd0 generation? well.. it’s not documented, but then, none of the process is so I might as well bluff my way into it!
Units seem to vary: <DO YOU SEE? THERE’S THAT OH-SO FAMILIAR BLOCK OF MISSING CODES IN THE LATE 80S, THEN THE DATA PICKS UP AGAIN. BUT LOOK AT THE CORRELATIONS ON THE RIGHT, ALL GOOD AFTER THE BREAK, DECIDEDLY DODGY BEFORE IT. THESE ARE TWO DIFFERENT STATIONS, AREN’T THEY? AAAARRRGGGHHHHHHH!!!!!
So the CRU discarded all their original data, due to “lack of space” they say. Or maybe it was such utter garbage that their own people could not operate on it, as this text file shows — so it was worse than useless. As the saying goes, “Garbage In, Garbage Out”.
Only now they want the whole world to swallow it!
Perhaps my understanding of the English language is lacking, or something. But, as I read this, I get the distinct impression that the entire data-set on which the UN IPCC reports and Copenhagen are based are completely and totally fictitious. If so, we have fraud on a scale which makes Madoff and Enron look like, well, pimples on a hog’s ass.
I met a Prof from the U of Calgary who, with three grad students, was visiting the automated climate stations in Canada’s Yukon, Northern BC and Northwest Territorys in 1998 while I was fishing along the Canol Rd in Yukon. As we sat having a beer over the campfire I asked him for his opionion about the % of global warming due to human influence. He said “Zero”. I asked him where his work was published because I wanted to see his data and he gave me a wary look. He explained that since Algore got into a position of influence in the Whitehouse the science was being hijacked. Hr blamed the Americans, of course a famous Canadian pastime, for cherry picking people to staff new climate study operations all over the world. He claimed his data, which runs from the mid 1940′s when the Alcan highway and the Canol Rd were built, doesn’t show any warming trend. He further claimed that if he subjected his data to peer review he could lose his job because of the influence of the cabal of Gore’s minions in the “new” scientific community.
At the time I was surprised but not apalled. No one in 1998 was talking about a “Global Carbon Tax” or a world government to save us all from doom. I find it very interesting that the old guard of scientists, the pre-Gorean money types, are much more skeptical than the folks who are on the grant treadmill today. This is purely anecdotal, but logical in so far as the Climate-gate scandal goes. Raw data is not conducive to the AGW theseus, add statistical manipulation + vast sums of money, eureka! doom.
The science may be hopelessly muddled, but the important thing to keep in mind is that among the Obamanite smart set the science is settled. It is grossly irresponsible to bugger the economy based on this science.
How anyone can take AGW seriously after seeing just the commented code is beyond me.
To quote Frank Barone: HOLY CRAP! I’ve seen a few of the HARRY.READ.ME comments before, but to see 5 pages of them really puts things into perspective. Great job of exposing this stuff. People should go to jail for this fraud–AND IT IS FRAUD. To think of all the money that has been wasted on this junk science make me question all of science. Is the ‘so-called’ Big Bang real?
As a scientist the funny thing to me is that these morons believe that a global temperature is something that can be accurately calculated from surface data. First, the individual temperature stations are only accurate to within several degrees because of urban factors. Second, any two scientists would make different corrections and summaries of the data so it is not reproducible. Third, the resulting calculated temperature can’t be any more accurate than the accuracy of the individual measurements. (That is, temperature changes of .5 degree can’t come from individual data only accurate to 1 or 2 degrees.)
The average global temperature is about as significant as the global average telephone number.
In terms of historical temperatures the data sets mix different types of data. You can’t do that and be called a real scientist. You can’t mix tree ring data with temperature data, or tree ring data with ice core data. All you can do is compare them.
There is a lot of voodoo science in the climate ‘science’ community.
I was frustrated just reading about this guy’s frustration! Could this guy be the whistleblower?
out
I swear this current apocryphal world political/religious/spiritual situation was all presciently foreseen in the Scary and Brilliant new Thriller book BLACK ROAD 2012, which has something in it to offend or rally everybody. I mean it—it’s almost really a Nostradamus paranormal event! http://tinyurl.com/Paranormal-Detective-Thriller
This reminds me of an old joke that follows the rapidly changing attitude of a man from Southern California who moves to Buffalo to enjoy the snow. By the end of his first winter he is more than half mad and resolved to return to SoCal immediately!
I sympathize with this poor schmo who is only trying to do his job. He’s been handed a box of crap and is expected to turn it somehow into something resembling a wedding cake. Needless to say, it’s an illusion he never quite pulls off!
Come on, it’s not about saving the earth, it’s about power.
Charles Krauthammer says it well:
“Socialism having failed so spectacularly, the left was adrift until it struck upon a brilliant gambit: metamorphosis from red to green. The cultural elites went straight from the memorial service for socialism to the altar of the environment. The objective is the same: highly centralized power given to the best and the brightest, the new class of experts, managers and technocrats. This time, however, the alleged justification is not abolishing oppression and inequality but saving the planet.”
Mmmmm, mmmmm, mmmmm . . . .
I read this when the story broke(actually much more than was presented here). I’ve been wondering when it would get more airplay.
At the very least it shows that all these quasi scientists should be forced to show their homework, and re-do under adult supervision.
I won’t remark on the obvious. Less obvious is the childlike, teen-like writing style used by Harris. It reads like something my son would have written when he was in Junior High School.
Substituting the work “fake” for “synthetic” is appropriate in order to comprehend the shenanigans these clowns were up to.
This is certainly damning, or bloody well ought to be. But please remember, folks, that this was never about ‘the climate’ anyway. It’s always been about redistribution, dictatorship and world government. Al Gore lies and knows he lies. As do they all. We need to keep our thumbs glued to their foreheads until they scream ‘uncle.’
Who is Harry? He is a hero. He will probably be looking for work soon – any way Pajamas Media can turn him into a correspondent?
And now for some proper science, The Tiger Woods Index a method of comparing how many people think the carbon evangelists are crooks and liars vs ho many think Copenhagen will produce a way to save the planet:
Climategate and the Tiger Woods Index
Those of us who practice the black art of computer programming learned long ago to not trust comments (i.e., read the e-mails) but rather determine what the actual source code is doing (here and here).
While arguments can rage all day between both sides, only the patched, screwed-up code can point to how fraudulent this effort to massage the data is.
Don’t worry, if climate change gets debunked, liberals will find some new imaginary problem to solve. It’s what they do and all they can do.
TRAVESTY-
Their data is fraudulent,
their faith in that data had become self sustaining.
Their belief in this global warming carbon god idol and
religion has even caused the testing and truths they once believed to no longer be true.
That which was good and true had now become evil,
and that which was evil had now become good and true.
By the way-I beleive the medias fixation on Tiger Woods marital woes is solely to hide Climategate. Funny, but when it was Bubba and Monica playing fellatio on 37 occasions, character didnt matter. But when paid scientific whores cook data and suppress evidence, the leaker is the problem. Id hate to imagine where we would be without the new media, since the old media is full of leftist proaganda shills. I dont overly worry-the commie left CANNOT SURVIVE without a boogieman of some sort to blame for there failed ideology. Problem is, the cant survive without our paying the cost of there failures, both past and future. As my dear wife says, God created Leftists to serve as tarmacs for flies to drop the feces on.
To Spinoneone: your understanding of English is indeed lacking, as is the case with anyone thinking that taking small excerpts of *anything* so grossly out of context in order to make some numbnutted point shows anything but malicious intent by the person making the excepts.
This whole CRU hacking stuff has made it clear that science education in this country needs to be seriously improved. The most hyperventilating and stupid comments about the CRU emails seem to come from people who never even had high school lab. Imagine a year long high school project where 100 thermometers are placed in and around a city, and every day the temperature from each one at both the beginning of the day and the end of the day is recorded on a form by a student. At the end of each week, all the student readings are collected and put into a computer, and at the end of the month, the information is evaluated, with obvious recording mistakes discarded along with results from obviously bad thermometers, and then all the final results charted and graphed. At the end of the school year, all the student forms are discarded and the charted and graphed results saved as part of a long term project.
See any conspiracy here? Idiots.
Dan W (7) writes:(That is, temperature changes of .5 degree can’t come from individual data only accurate to 1 or 2 degrees.)
That just isn’t true. It depends on whether the accuracy errors have a systematic bias or whether they are random. Averaging processes routinely produce much higher accuracy than the data they are averaging.
However, I’m really nit-picking. Your other comments on surface temp data are right on. Overall, it’s noisy, lousy data, with all sorts of systematic biases. Nobody should bet the world’s economy on it.
BC:
You forgot the Add-hard-coded-number-arrays (variable name ‘fudge factor’) part.
“with obvious recording mistakes discarded along with results from obviously bad thermometers”
Yeeeeeah. What could possibly go wrong with just tossing out the data we don’t like?
http://www.eyeblast.tv/public/checker.aspx?v=GdaGkUIrDk
http://www.youtube.com/watch?v=Gp4sMasX-_8&feature=player_embedded
#22 BC
I take issue with your assertion that I am taking anything out of context. In fact, part of the reason for pulling so many excerpts from this file was to convey the sentiment of the document. I challenge you to read the entire text (as I did) and come away with confidence in CRU’s methods.
http://www.anenglishmanscastle.com/HARRY_READ_ME.txt
But… In case you need more convincing of Harry’s position on the science, let me present one of his emails:
http://www.eastangliaemails.com/emails.php?eid=237&filename=994083845.txt
—————————-
Williams, Derek wrote:
>No, it’s very dangerous to make predictions like this and IMO doesn’t
>help the cause. Even without human activities, natural things like big
>volcanoes can easily disrupt the climate in such a way as to swamp the
>signs of global warming and indeed produce severe weather conditions as
>a casual glance at met records for the past couple of hundred years
>quickly shows…
(Harry replies:)
You’re hardly any better, Derek: this is hardly a ‘Nature’ paper, is it?…
…Even if it turns out to be
naturally-occurring, who’s willing to take that chance? We should be
trying to wean ourselves off of unsustainable energy generation and
use anyway.
Cheers
Harry
——————-
–Andie Brownlow
“This whole CRU hacking stuff has made it clear that science education in this country needs to be seriously improved. The most hyperventilating and stupid comments about the CRU emails seem to come from people who never even had high school lab.
See any conspiracy here? Idiots.”
BC: As a Rocket Scientist who has spent the last 25 years doing statistical data analysis, the answer is YES.
27. Clay Marley:
Takedown! Oooh, that’s gotta smart!
Golly, BC. Thank you for venting your sense of superiority on the rest of us.
Many of us have indeed had high school science labs, all three basic disciplines in my case. Many of us are ‘environmentally conscious.’ We’re not creationists or holocaust deniers either, and we’re not the result of the unholy union of brother and sister.
And yet here we are, spekpticizing and denying away. Whhheeeee.
@29 Don Rodrigo
/Agreed
BC’s attack on our mental faculties doesn’t help his/her case very much. Some of us have to work with raw data and/or programs directly and can easily see come up with some major concerns with this process. Using BC’s own example what the CRU have been doing is more like this.
The “Plan” is to put 100 thermometers around the city and check them every day at both the begining of the day and the end. Then the data is recorded on a form by a student. Then at the end of the week all of the readings are put into a computer and compiled weekly, monthly, and then yearly.
Unfortunetly students lose the forms and instead of getting docked points for the project just make the numbers up. Then they give them to the computer guy who doesn’t care and puts in the numbers wherever not even bothering to properly put the data into the right areas. Then after the first month they find out they’ve actually got readings for 112 thermometers. Eh, nothing wrong there probably just merged data we’ll just flag them as a new made up number. It’ll all balance out.
Then at the end of the year they look at all this data and decide it’s not hot enough so they add on some “fudge” numbers to all of the monthly data sets until the series looks like what they expect to find. For good messure they include the extra readings that have piled up to prove they’ve been improving their messuring accuracy with more stations!
So even though they started out with 100 stations recording data being meassured 2 times a day and entered weekly into a computer somehow when it’s all said and done they have 223 stations reporting four times a day and they all show things are getting hotter! Wow, amazing.
Yeah it requires a lot of scientific background to realize how messed up this whole situation is.
Forget the read me comments, forget the emails, all you have to do is to look at the source code. That’s where the real fraud happened. Here’s a good explanation for the inner workings of the code, line by line — interpreted so even a journalist can understand:
http://beforeitsnews.com/story/0000000000000790
To AndieBrownlow: Oooo…they mispelled “percentage” as “pewrcentage” about 1/3 of the way down. Those bastards, hang them all!!!
Do you know why they need stuff like this thing for climate research? Do you have any idea how much data has to collected and analyzed for *serious* climate research? Go check the specs for this little storage system. Even a branch of climate research needs more than a few boxes of data. Yeah, I kinda stick by my assertions about your excerptions.
Maybe the real problem is that you guys genuinely have no idea what a massive, complex endeavor it is to do climate research on a global level…hmmm…. Nah, you’re all just numbnuts.
Can someone please tell me exactly what harry_readme is? I know it’s a text file, but is it a compilation or extract of comments that were placed in some code?
I’m a software engineer, and I’ve worked with code written by scientists from one of the DOE labs in the past. Harry’s comments had a familiar ring to them. In my experience software written by scientists is some of the worst code on the planet. Scientists are arrogant enough to think that time-tested design and coding conventions are beneath them. They have too much ego invested to admit they need the help of real computer scientists.
Jefrois @ 9: Hawking your own crappy books on someone else’s web site is just bad form.
“L”*
Let’s see now. We here in the United States have only 400 to 700 years supply in known and economically recoverable deposits of coal, petroleum and natural gas to meet our future energy needs. So we really have to hurry to convert to far more expensive sources of “renewable energy”. Surely, we can’t expect energy technology to improve in the next 400 years any more than it did in the last 400 years, now can we? Energy technology as of about 1609 CE, Hmmmmm.
#7 Absolutely right, DanW.
A year ago last April, I posted this comment on the Riehl World View blog in answer to a BBC report using a figure like this to report an increase in the average temperature of the planet Earth: 0.75C.
“Moe, let me ask you a very simple question, with your permission of course. Here it is: How deep is the ocean?
Well now you may say, “Stupid question. I have to know where, what part of the ocean, you are asking about.”
I say, no, Moe, the entire ocean, the average depth of the ocean. Think about this and you will come quickly come to the realization that only a very rough estimate, but no accurate measurement to a centimeter say, is possible. The winds, the irregular bottom of the oceans, and the tides make such an accurate measurement impossible. Only a very rough average depth may be posited, not to mention measured. The same is true of the average temperature of the earth only more so. The sun, shade, winds, different heights of the Earth’s surface and the effect of time of day make accurate measurement to a hundredth or even a tenth of a degree impossible…
Posted by: Fred Beloit | Friday, April 04, 2008 at 04:12 PM”
#22 by that brilliant scientific experimenter and science educator BC:
“…every day the temperature from each one[thermometer] at both the beginning of the day and the end of the day is recorded on a form by a student.”
BC clearly seems not to understand that the beginning of one day is only a second away from the end of the previous day. LMAO.
BC – You’re always on here calling us idiots, numbnuts, and the like. What a gross superiority complex you are demonstrating!
When is it going to get through to you that a lot of people posting on this site are truly smart and educated individuals? When are you going to start showing some respect? If you want to refute their arguments, then by damn, refute them without the childish name-calling! What the hell is wrong with you, anyway? You may not agree with their opinions, but that doesn’t make them idiots.
It is time that you recognize some of these people to be at least your intellectual and educational equals and cease the childish name-calling. If you can’t do that, then go post somewhere else.
I, personally, am interested in your opinions, because you at least put forward some arguments, unlike the trolls, so I usually read your posts and links. However, you come off as a troll when you resort to ad hominem attacks. It is time for you to grow up and to act like an adult in a forum for adults. Else, go to another site, also known as, “Go sit at the kids’ table.”
Hey, thats a great photo of a Nuclear detonaton. I wonder if that was supposed to illustrate a super duper fast form of global warming…. probbblee not eh?
Great article and Merry Christmas.
It is truly amazing that reality has lost its grip on the masses. How can you even think this is NOT fraudulent? Well, more and more scientists are coming forth emphasizing that the IPCC was designed solely for the new world gov using FEAR tactics; the purpose: to ‘prove’ AGW is real, even if it isn’t.
Check out the Copenhagen Document leaks, this knowledge may save your life! Do what you can with your Government Reps in your country to have these infringements on your life eliminated! You will literally be fighting for your and your family’s existence, Click the videos below:
http://www.youtube.com/watch?v=dAqqAnUxACY
http://www.youtube.com/watch?v=94KH-WMZuw0
Request that PM Harper DOES NOT sign the Copenhagen Treaty, thereby causing Canadians to lose their Sovereignty and Freedom, email the PM at: pm@pm.gc.ca
Sign the petition to protest the Inaccurate Science measurements that are being used to base the Copenhagen Treaty off of: http://www.gopetition.com/online/32485.html
… and, topping everything, mystery e-mails, yearning for a religion, group-think, panicky populations, gross mind-control, UN racket, etc. – why in any public (& MSM-handled) presentation of the global warming situation the following two extremely important scientific discoveries are not mentioned and discussed:
1) the Milankovich cycles –
1) and the fact that (independently of the Milankovich cycles???), the surface temperature of Mars has, in the past forty years, grown at the same rate of this planet’s – yet no mufflers, no hamburgers, no fartin’ cows, no SUV-s, no nothin’ there -
Why the extremely intelligent & blindingly rational liberals prefere to stomp around in circles ululating, feathers in head, banging drums, eating non-farts-issueing algae, buying very expensive Gaia-friendly home compost squeazers (I just read a really scary one about a couple in New York doing this), instead of examining and trying to get a clue from astronomy and geology about the universe’s larger (and, alas! uncorrectable) rythms?
It is obvious that global heating is here – yet it is also well known that this planet went through this situation many times before, a few of those times being rather close to us, the Middle Ages, and the relatively recent (geologically speaking), Holocene peak that caused the final “Bosphorus” break & flood of the Black Sea (about 3700 BC – and that temperature peak was much, much bigger than the worst anticipation we have for the future!) -
And, while it is very unclear whether human activity is causing the current warming, something else is very striking in this affair – the stubborn passion liberals invest in blaming “the unwashed, needing guidance” masses of this situation –
Somethin’s amiss here -
BC:
While we certainly need more science education, I doubt it would make massaging data and dubious correction factors palatable. In fact, it would have the opposite effect. If I did CRU’s kind of work at the university where I got my masters or at the lab I work at now, I would expect to be run out on a rail, probably sued.
Science is a noble endeavor, and I’m sure there are plenty of climatologists doing accurate and precise work. Sadly, the emails show that was not the case at the CRU.
Check out the Copenhagen Document leaks, this knowledge may save your life! Do what you can with your Government Reps in your country to have these infringements on your life eliminated! You will literally be fighting for your and your family’s existence, Click the videos below:
http://www.youtube.com/watch?v=dAqqAnUxACY
http://www.youtube.com/watch?v=94KH-WMZuw0
Request that PM Harper DOES NOT sign the Copenhagen Treaty, thereby causing Canadians to lose their Sovereignty and Freedom, email the PM at: pm@pm.gc.ca
Sign the petition to protest the Inaccurate Science measurements that are being used to base the Copenhagen Treaty off of: http://www.gopetition.com/online/32485.html
Check out what Government is doing behind your back at: : http://www.youtube.com/watch?v=VebOTc-7shU
Well, it seems Mr. Ehrlich ( another of Obama’s Eugenicist Science Czars who works with John Holdren ) will have his way with humanity if the Copenhagen Treaty will have success. There is your global system, with a globally enforceable mandate- at first to impose carbon taxes upon humanity- later on to do what the eugenicists have always called and planned for: the orderly extermination of at least 80 percent of the global population.
Writer: Btok
So, a lot of these comments in the Harry.read_me.txt file seem to refer to the TS 2.1 gridded data. What a mess! That is pretty sad for those who routinely use this data to estimate ETo and/or open water evaporation. All I can say is Aaarrgghh!!