Add in satellite records with the same baseline anomaly (1979-1998) and you will see that all are pretty damn close:
http://i81.photobucket.com/albums/j237/hausfath/FigureOne.png

http://i47.tinypic.com/16c9wn4.jpg

(Confirmation of this plot is here: http://krugman.blogs.nytimes.com/2009/12/09/a-picture-thats-worth-a-thousand-stolen-emails/ )

The only mismatch I now see is between short-running satellite data that shows a much larger 1998 spike (claimed by most to be El Nino related) than any of these ground based plots, which is not really very important trend-wise.

Therefore if GISTEMP can be validated in a way that is convincing then ground-based measurements in general will indeed be strongly supported.

I’ve drafted a longer response in a blog post which is awaiting approval from The Committee.

I just got off the conference call with Al and George. They say post away.

I need some time to make my own plots of the various global averages to support (or deny) my assertion that GISTEMP is diverging from the others. That is definitely another well-accepted idea on skeptic blogs that has rubbed off on me, that I must now finally clarify.

“I suppose it would be possible to produce a plot showing the anomaly with and without various adjustments, but that’s not a plot that GISTEMP produces right now, and our current phase of working is replicating GISTEMP in Python (which we’ve very recently pretty much done) and then making that code clearer. Eventually I expect that we will get round to the stage where we can produce a “before and after” plot for the adjustments, but it’s not where our resources are focussed right now.”

Indeed plots with only single (or no) adjustments added in would allow exactly the “adjustments-over-time” plot(s) that I inquired about, by merely creating difference curves. So my inquiry is indeed an appropriate one instead of inapplicable to how GISTEMP works, which I had somewhat feared.

The above clarification of your mission statement is only partially convincing since if your results eventually show that an improved GISTEMP shows less recent warming compared to past variation, then public confidence would only be increased in the very long term, while very much taking a hit in the near term, and that is assuming GISS itself accepts suggested modifications rather than minor bug fixes. If all the project succeeds in accomplishing is to reproduce the same black box in Python instead of Fortran then it will fail to meet its stated goal. Clarifying how GISTEMP works is what seems to be your real goal and this does require novel analysis above and beyond what GISS offers.

Lack of time and resources is understandable on a volunteer project, especially when extra income is not common these days. Part of the misunderstanding here is that you are obviously interested in a rigorous and utterly professional presentation, whereas I feel some impatience about just seeing a quick weekend’s analysis. At this stage I’d say your team is the most qualified around to provide it, however long it might take to finally focus on.

1) Do our goals prevent us from presenting findings that conflict?

I’ve already answered this briefly, I’ve drafted a longer response in a blog post which is awaiting approval from The Committee.

3) That GISTEMP is an “outlier” with respect to other temperature series.

Again (like the GHCN adjustments) this is really irrelevant to ccc-gistemp. Our aim is to make the GISTEMP code clearer, perhaps when it is clearer it will be easier to see how and why it differs from other temperature anomaly reconstructions. The reconstructions come with confidence intervals and they all seem to be within each other’s confidence intervals, it would probably take a statistician to say anything more definitive.

2) Adjustments over time.

I’m not exactly sure what you mean by “an adjustment-vs.-time plot”. Are you asking for a plot similar to Figure 13 of Hansen and Lebedeff’s 1987 paper “Global trends of measured surface air temperature”? (the paper is available for download here)

In that case it shows the (5 year moving average of the) historical temperature anomaly computed with and without the urban stations.

I suppose it would be possible to produce a plot showing the anomaly with and without various adjustments, but that’s not a plot that GISTEMP produces right now, and our current phase of working is replicating GISTEMP in Python (which we’ve very recently pretty much done) and then making that code clearer. Eventually I expect that we will get round to the stage where we can produce a “before and after” plot for the adjustments, but it’s not where our resources are focussed right now.

I fully appreciate your statement “Just because many of us are non-programmers doesn’t mean we can’t understand final results…*if* they are shown”. We’re not hiding results, we just don’t have the resources to produce something novel right now.

Since this part of GISTEMP interests you, I suggest you recruit a programmer to help you. All our code is publicly available on googlecode and we can certainly pointed interested people at the relevant bits (I’m not so boned up on this particular part, but I think it’s mostly steps 1 and 2).

4) The adjustments in the GHCN source data.

From the perspective of ccc-gistemp this is irrelevant. The GHCN is a source to GISTEMP and we use the same source. The terms of ccc-gistemp are to clarify GISTEMP, not to go delving into the arcane details of how its source data are produced. From the wider perspective of Clear Climate Code, I suppose the exact adjustment procedure might be of some interest (although I would not place it high on our list of things to do next after GISTEMP).

The GHCN adjustments are discussed in the 1997 Peterson and Vose paper that is available for download from NOAA. [ed: 2010-01-08: corrected Rose to Vose, oops]

However, GISTEMP, and so also ccc-gistemp, does not use the adjusted data. It uses the unadjusted data.

Forgive me for pointing you at the source code (I’m hoping that non-programmers can understand this bit): fetch.py fetches the GHCN v2.mean.Z dataset (follow the link to see the actual source code). As the readme for the GHCN data makes clear (sorry, it’s an FTP link to a plain text file which will probably download rather than display) v2.mean.Z is the unadjusted data, the adjusted data is in v2.mean_adj.Z and that is not downloaded (or, obviously, used).

1) Do our goals prevent us from presenting findings that conflict? This is important, so a full response will probably go in another post, but briefly: We say “increase confidence in climate science results”. We don’t mean “increase confidence in the current results”, we mean “make sure that the results are visibly correct, if necessary by correcting them”.

2) Adjustments over time. (I think this is GISS site where you can compare adjusted versus unadjusted station data, right?)

3) That GISTEMP is an “outlier” with respect to other temperature series (the ones I know about are listed at the bottom of the GISTEMP tab).

4) The adjustments in the GHCN source data.

5) Sundry other snipes, such as “Please post an adjustment-vs.-time plot (or equivalent) ASAP to help clear up any such suspicion”, and “you are not doing a balanced analysis but are only attempting to vindicate GISTEMP”.

The official statement here is not encouraging: “On some other occasions we have questioned whether the intended algorithm is a sensible way to compute a global surface temperature anomaly dataset. … Of course, there are many other approaches adopted by various climate science groups around the world, and one reasonable answer to such criticisms would be “if you don’t like ours, use one of theirs”.”

Indeed, GISTEMP is an outlier. It shows continued warming in the last decade while the other sets do not. It is exactly this “trust us or get lost” attitude that very much decreases confidence in climate science. Publishing a simple plot of adjustments-over-time would quickly clear up any suspicion that the black box contains bizarre instead of mundane mechanisms. That is, unless the adjustments do show a warming trend over time. Then there’s proper explaining to do or you will fail to increase confidence in climate science.

Since GISTEMP uses already-adjusted data (from GHCN) your task of increasing confidence in the results is even more complicated, for this must be addressed as well.

Just because many of us are non-programmers doesn’t mean we can’t understand final results…*if* they are shown.

My first question really is whether “adjustments-over-time” is a valid inquiry to make based on how GISTEMP works. If not, then what is the equivalent overall analysis you might use to convince me to have more confidence is GISTEMP? Realize that any answers must be taken in the context that you clearly state that you are not doing a balanced analysis but are only attempting to vindicate GISTEMP, giving the impression that the project will be indefinitely delayed if analysis shows that GISTEMP has been parametrized to maximize recent warming and minimize past warming.

Please post an adjustment-vs.-time plot (or equivalent) ASAP to help clear up any such suspicion.