Climategate's Harry_Read_Me.txt: We All Really Should

One of the most damning pieces of evidence in Climategate (so far) is a text file called HARRY_READ_ME.txt. This file is supposedly written by Ian “Harry” Harris, a researcher at the University of East Anglia's CRU (Climatic Research Unit). In it he details the trials and tribulations of being tasked with creating a new climate information database from previous publications and databases. According to Harry’s documented struggle, he is confronted with missing, manipulated, and undocumented data that he has to use to try to piece together the newer TS 3.0 database.

Here are the brow-raising excerpts:


So, uhhhh.. what in tarnation is going on? Just how off-beam are these datasets?!!

Unbelievable - even here the conventions have not been followed. It's botch after botch after botch.

22. Right, time to stop pussyfooting around the niceties of Tim's labyrinthine software suites - let's have a go at producing CRU TS 3.0! since failing to do that will be the definitive failure of the entire project..

Nearly 11,000 files! And about a dozen assorted 'read me' files addressing individual issues…

(yes, they all have different name formats, and yes, one does begin '_'!)

How handy – naming two different files with exactly the same name and relying on their location to differentiate! Aaarrgghh!!

If the latest precipitation database file contained a fatal data error… then surely it has been altered since Tim last used it to produce the precipitation grids? But if that's the case, why is it dated so early?

So what's going on? I don't see how the 'final' precip file can have been produced from the 'final' precipitation database, even though the dates imply that. The obvious conclusion is that the precip file must have been produced before 23 Dec 2003, and then redated (to match others?) in Jan 04.

There is no way of knowing which Tim used to produce the current public files. The scripts differ internally but - you guessed it! - the descriptions at the start are identical. WHAT IS GOING ON?

So what is this mysterious variable 'nf' that isn't being set? Well strangely, it's in Mark N's ''. I say strangely because this is a generic prog that's used all over the place! Nonetheless it does have what certainly looks like a bug…

Where is the documentation to explain all this?!

Bear in mind that there is no working synthetic method for cloud, because Mark New lost the coefficients file and never found it again (despite searching on tape archives at UEA) and never recreated it.

DON'T KNOW, UNDOCUMENTED. Wherever I look, there are data files, no info about what they are other than their names. And that's useless..

So what the hell did Tim do?!! As I keep asking.

This is irritating as it means precip has only 9 fields and I can't do a generic mapping from any cru format to cru ts.

Then.. like an idiot.. I had to test the data!

It's halfway through April and I'm still working on it. This surely is the worst project I've ever attempted. Eeeek.

Oh bugger. What the HELL is going on?!

In fact, on examination the US database record is a poor copy of the main database one, it has more missing data and so forth. By 1870 they have diverged, so in this case it's probably OK.. but what about the others?

Oh GOD if I could start this project again and actually argue the case for junking the inherited program suite!!

Oh Tim what have you done, man?

Just another thing I cannot understand, and another reason why this should all have been rewritten from scratch a year ago!

am I the first person to attempt to get the CRU databases in working order?!!