How close are we to GISTEMP?

This close:
gistemp (red) vs ccc-gistemp (black).

The two graphs are almost on top of each other. I’ll add 0.02K to the black line to separate them a bit:
gistemp (red) vs ccc-gistemp (black), artificially separated.

We can now see the red series that the black series was hiding, and we can see that the differences between the 2 series are minute at most. 1 or 2 centikelvin here and there. Red is official GISTEMP, black is our ccc-gistemp code.

What exactly am I comparing? GISTEMP’s global temperature anomalies, one set from their website, one set from our ccc-gistemp code. I’m running the vischeck command:

code/vischeck.py -o 2 result/GLB.Ts+dSST.txt result/GLB.Ts.ho2.GHCN.CL.PA.txt

(the -o option is used to produce the offset graphs, bottom picture)

The first file is GLB.Ts+dSST.txt, that I download from NASA yesterday. The second file, GLB.Ts.ho2.GHCN.CL.PA.txt, is the result of me running ccc-gistemp yesterday.

But it’s not a very careful comparison. The inputs I am using are SBBX.HadR2 and v2.mean downloaded on 2009-12-04 and an hcn_doe_mean_data downloaded in June (!). Also, the version of the GISTEMP code we are coding against is quite old (about a year) and has been updated several times. For example, GISTEMP currently use USHCN version 2, ccc-gistemp does not (yet). The fact that we’re not keeping up with GISTEMP is Issue 7.

Furthermore the exact output may depend on the Fortran compilers being used, the architecture on which I’m running, and the Python versions we’re using.

The bottom line is that we’re already very close to the GISTEMP output, well with any meaningful error threshold. As we get closer we’ll need to be a lot more careful about keeping track of exactly what inputs and software tools are being used. We’ve requested from GISS a copy of the exact inputs and outputs for one of the runs, so that we have a fixed set for comparison purposes.

22 Responses to “How close are we to GISTEMP?”

  1. Nick.Barnes Says:

    Excellent. I suggest &chdl=GISS|CCC%20%28offset%29&
    (or similar).

  2. drj Says:

    @Nick: Done. Also added “&chdlp=t”.

    As you know (but other readers may not), the charts are produced using Google’s lovely Chart API, and I love how it’s so easy to fiddle with the URLs.

  3. dhogaza Says:

    Good job, guys. Impressive work.

  4. Boris Says:

    “it’s so easy to fiddle with the URLs.”

    CONSPIRACY!

  5. jkeyes Says:

    LOL @Boris.

  6. steven mosher Says:

    Excellent work chaps. You have way more patience than I do.

    What platform are you guys running on

  7. drj Says:

    @steven: A variety of platforms. Our unstated aim is “anything that has Python 2.4″. I use Mac OS X 10.4 with 3rd party Python 2.6, others use Max OS X 10.6 (with Apple’s Python?); FreeBSD 6.3 with Python 2.5.4. A couple of participants are actively trying to get ccc-gistemp running on Windows; we don’t (quite) yet, and this is issue 1.

    Anything with Python (and Fortran, until we eliminate it) should be okay, but the devil is in the details.

    In the past I’ve tried to run it on AIX, but my patience ran out after trying to build Python 2.6.

  8. steven mosher Says:

    thx drj.

    I wanted a good reason to learn Python. mac Os 10.5.4. I’ll keep checking back.

    Also kudos for taking the high road here. When JohnV did his opentemp i was a supporter. Well, u guys know my position on code availability.

  9. Leo Morgan Says:

    Congratulations!
    A very desirable project.
    May I suggest a glossary? With the rule of thumb ‘if it’s not in the dictionary, it needs to be defined’. It depends on your target audience, of course. Does it include those like myself, the honest science enthusiast who is nevertheless an unconvinced skeptic, but willing to do the work to be persuaded by compelling evidence? If so, I really strongly recommend the Glossary so we can know what you’re talking about. I admit my ignorance, but affirm my willingness to learn.

  10. sod Says:

    good work!

    thanks!

  11. drj Says:

    @Leo: what sort of terms did you want us to gloss? I mean, where should we start first?

  12. ligne Says:

    not entirely surprising you managed to replicate the data so closely, but a fantastic effort nonetheless. congratulations to everyone involved!

  13. NikFromNYC Says:

    Nowhere have I ever seen adjustments-over-time for GISTEMP. Since GISS has a site that indeed shows station data before/after adjustment it seems that individual stations are having their data altered in a way that could be analyzed as adjustments-over-time. That is one thing you have not yet done: reproduce individual station adjustments to validate those on the GISS web site. The stated goal of this project, “To increase public confidence in climate science results”, is incompatible with the idea that you intend to present any findings that conflict with said goal. Reproducing GISTEMP merely makes a carbon copy of a black box as far as us non-programmers are concerned.

    The official statement here is not encouraging: “On some other occasions we have questioned whether the intended algorithm is a sensible way to compute a global surface temperature anomaly dataset. … Of course, there are many other approaches adopted by various climate science groups around the world, and one reasonable answer to such criticisms would be “if you don’t like ours, use one of theirs”.”

    Indeed, GISTEMP is an outlier. It shows continued warming in the last decade while the other sets do not. It is exactly this “trust us or get lost” attitude that very much decreases confidence in climate science. Publishing a simple plot of adjustments-over-time would quickly clear up any suspicion that the black box contains bizarre instead of mundane mechanisms. That is, unless the adjustments do show a warming trend over time. Then there’s proper explaining to do or you will fail to increase confidence in climate science.

    Since GISTEMP uses already-adjusted data (from GHCN) your task of increasing confidence in the results is even more complicated, for this must be addressed as well.

    Just because many of us are non-programmers doesn’t mean we can’t understand final results…*if* they are shown.

    My first question really is whether “adjustments-over-time” is a valid inquiry to make based on how GISTEMP works. If not, then what is the equivalent overall analysis you might use to convince me to have more confidence is GISTEMP? Realize that any answers must be taken in the context that you clearly state that you are not doing a balanced analysis but are only attempting to vindicate GISTEMP, giving the impression that the project will be indefinitely delayed if analysis shows that GISTEMP has been parametrized to maximize recent warming and minimize past warming.

    Please post an adjustment-vs.-time plot (or equivalent) ASAP to help clear up any such suspicion.

  14. drj Says:

    @NikFromNYC: You make a large number of points, I’ll break them down as I see it in this comment, and then make separate responses in further comments, or possibly posts if it is merited.

    1) Do our goals prevent us from presenting findings that conflict? This is important, so a full response will probably go in another post, but briefly: We say “increase confidence in climate science results”. We don’t mean “increase confidence in the current results”, we mean “make sure that the results are visibly correct, if necessary by correcting them”.

    2) Adjustments over time. (I think this is GISS site where you can compare adjusted versus unadjusted station data, right?)

    3) That GISTEMP is an “outlier” with respect to other temperature series (the ones I know about are listed at the bottom of the GISTEMP tab).

    4) The adjustments in the GHCN source data.

    5) Sundry other snipes, such as “Please post an adjustment-vs.-time plot (or equivalent) ASAP to help clear up any such suspicion”, and “you are not doing a balanced analysis but are only attempting to vindicate GISTEMP”.

  15. drj Says:

    @NikFromNYC: At least one of your points is easy to clear up.

    4) The adjustments in the GHCN source data.

    From the perspective of ccc-gistemp this is irrelevant. The GHCN is a source to GISTEMP and we use the same source. The terms of ccc-gistemp are to clarify GISTEMP, not to go delving into the arcane details of how its source data are produced. From the wider perspective of Clear Climate Code, I suppose the exact adjustment procedure might be of some interest (although I would not place it high on our list of things to do next after GISTEMP).

    The GHCN adjustments are discussed in the 1997 Peterson and Vose paper that is available for download from NOAA. [ed: 2010-01-08: corrected Rose to Vose, oops]

    However, GISTEMP, and so also ccc-gistemp, does not use the adjusted data. It uses the unadjusted data.

    Forgive me for pointing you at the source code (I’m hoping that non-programmers can understand this bit): fetch.py fetches the GHCN v2.mean.Z dataset (follow the link to see the actual source code). As the readme for the GHCN data makes clear (sorry, it’s an FTP link to a plain text file which will probably download rather than display) v2.mean.Z is the unadjusted data, the adjusted data is in v2.mean_adj.Z and that is not downloaded (or, obviously, used).

  16. drj Says:

    @NikFromNYC

    2) Adjustments over time.

    I’m not exactly sure what you mean by “an adjustment-vs.-time plot”. Are you asking for a plot similar to Figure 13 of Hansen and Lebedeff’s 1987 paper “Global trends of measured surface air temperature”? (the paper is available for download here)

    In that case it shows the (5 year moving average of the) historical temperature anomaly computed with and without the urban stations.

    I suppose it would be possible to produce a plot showing the anomaly with and without various adjustments, but that’s not a plot that GISTEMP produces right now, and our current phase of working is replicating GISTEMP in Python (which we’ve very recently pretty much done) and then making that code clearer. Eventually I expect that we will get round to the stage where we can produce a “before and after” plot for the adjustments, but it’s not where our resources are focussed right now.

    I fully appreciate your statement “Just because many of us are non-programmers doesn’t mean we can’t understand final results…*if* they are shown”. We’re not hiding results, we just don’t have the resources to produce something novel right now.

    Since this part of GISTEMP interests you, I suggest you recruit a programmer to help you. All our code is publicly available on googlecode and we can certainly pointed interested people at the relevant bits (I’m not so boned up on this particular part, but I think it’s mostly steps 1 and 2).

  17. drj Says:

    @NikFromNYC:

    3) That GISTEMP is an “outlier” with respect to other temperature series.

    Again (like the GHCN adjustments) this is really irrelevant to ccc-gistemp. Our aim is to make the GISTEMP code clearer, perhaps when it is clearer it will be easier to see how and why it differs from other temperature anomaly reconstructions. The reconstructions come with confidence intervals and they all seem to be within each other’s confidence intervals, it would probably take a statistician to say anything more definitive.

  18. drj Says:

    @NikFromNYC:

    1) Do our goals prevent us from presenting findings that conflict?

    I’ve already answered this briefly, I’ve drafted a longer response in a blog post which is awaiting approval from The Committee.

  19. NikFromNYC Says:

    That GISTEMP uses unadjusted data is very pleasant news to me. I ran into the opposite claim early on, repeatedly, and regrettably failed to confirm it. That means if GISTEMP (or its update) is validated then I can have real confidence in such validation.

    I need some time to make my own plots of the various global averages to support (or deny) my assertion that GISTEMP is diverging from the others. That is definitely another well-accepted idea on skeptic blogs that has rubbed off on me, that I must now finally clarify.

    “I suppose it would be possible to produce a plot showing the anomaly with and without various adjustments, but that’s not a plot that GISTEMP produces right now, and our current phase of working is replicating GISTEMP in Python (which we’ve very recently pretty much done) and then making that code clearer. Eventually I expect that we will get round to the stage where we can produce a “before and after” plot for the adjustments, but it’s not where our resources are focussed right now.”

    Indeed plots with only single (or no) adjustments added in would allow exactly the “adjustments-over-time” plot(s) that I inquired about, by merely creating difference curves. So my inquiry is indeed an appropriate one instead of inapplicable to how GISTEMP works, which I had somewhat feared.

    The above clarification of your mission statement is only partially convincing since if your results eventually show that an improved GISTEMP shows less recent warming compared to past variation, then public confidence would only be increased in the very long term, while very much taking a hit in the near term, and that is assuming GISS itself accepts suggested modifications rather than minor bug fixes. If all the project succeeds in accomplishing is to reproduce the same black box in Python instead of Fortran then it will fail to meet its stated goal. Clarifying how GISTEMP works is what seems to be your real goal and this does require novel analysis above and beyond what GISS offers.

    Lack of time and resources is understandable on a volunteer project, especially when extra income is not common these days. Part of the misunderstanding here is that you are obviously interested in a rigorous and utterly professional presentation, whereas I feel some impatience about just seeing a quick weekend’s analysis. At this stage I’d say your team is the most qualified around to provide it, however long it might take to finally focus on.

  20. Nick.Barnes Says:

    I’ve drafted a longer response in a blog post which is awaiting approval from The Committee.

    I just got off the conference call with Al and George. They say post away.

  21. NikFromNYC Says:

    I have falsified my former assertion, based on blog lore that GISTEMP is an “outlier” that shows more recent warming than other data sets. It does not appear to show greater warming than the other two ground-based data sets:

    http://i47.tinypic.com/16c9wn4.jpg

    (Confirmation of this plot is here: http://krugman.blogs.nytimes.com/2009/12/09/a-picture-thats-worth-a-thousand-stolen-emails/ )

    The only mismatch I now see is between short-running satellite data that shows a much larger 1998 spike (claimed by most to be El Nino related) than any of these ground based plots, which is not really very important trend-wise.

    Therefore if GISTEMP can be validated in a way that is convincing then ground-based measurements in general will indeed be strongly supported.

  22. Zeke Hausfather Says:

    Nik,

    Add in satellite records with the same baseline anomaly (1979-1998) and you will see that all are pretty damn close:
    http://i81.photobucket.com/albums/j237/hausfath/FigureOne.png

Leave a Reply