ccc-gistemp release 0.6.0
Posted by drj | Filed under announcement
[ed: 2010-10-29: This was an announcement for release 0.6.0, but that has a bug (see comments). Please use release 0.6.1 instead. I've edited the article to update the links]
I’m pleased to announce ccc-gistemp release 0.6.1 (making the previous buggy 0.6.0 obsolete). ccc-gistemp is our project to reimplement the NASA GISS GISTEMP algorithm in clearer code (in Python). This release is the first release made under the aegis of the Climate Code Foundation.
Many of the significant changes in this release have already been previewd in earlier blog posts:
- There is a new major version of GHCN (version 3, in beta), and ccc-gistemp can now use GHCN v3;
- ccc-gistemp can make use of a land mask;
- More flexibility in mixing and matching input datasets, which means I can run ccc-gistemp without using any US temperatures (amongst other things).
Further details are available in the release notes.
We intend to carry on our work on ccc-gistemp, and we urge you to download our code, try it, and read it. We welcome contributions.
The work for release 0.6.0 was carried out by David Jones and Nick Barnes, and the work for release 0.6.1 was carried out by David Jones.
October 28th, 2010 at 12:24 pm
I just downloaded and tried to install ver. 0.6.0
It threw an error while installing.
I’m using python ver. 2.6.5
====> STEPS 0, 1, 2, 3, 4, 5 ====
No more recent sea-surface data files.
Load GHCN records
Load USHCN records
(Reading average temperature)
Load SCAR records
Correct the GHCN Hohenpeissenberg record.
Adjust USHCN records
1 USHCN records had no GHCN counterpart.
Traceback (most recent call last):
File “tool\run.py”, line 280, in
sys.exit(main())
File “tool\run.py”, line 261, in main
data = step_fn[step](data)
File “tool\run.py”, line 104, in run_step5
result = step5.step5(data)
File “E:\Python\Gistemp6\code\step5.py”, line 443, in step5
return annzon(zoned_averages)
File “E:\Python\Gistemp6\code\step5.py”, line 320, in annzon
(tdata, twt) = zoned_averages.next()
File “E:\Python\Gistemp6\code\step5.py”, line 183, in zonav
boxed_data.next())
File “E:\Python\Gistemp6\tool\giss_io.py”, line 1241, in step5_bx_output
for record in data:
File “E:\Python\Gistemp6\code\step5.py”, line 69, in SBBXtoBX
landweight,landsub,oceansub = zip(*itertools.islice(data, nsubbox))
File “E:\Python\Gistemp6\tool\giss_io.py”, line 1261, in step5_mask_output
for datum in data:
File “E:\Python\Gistemp6\code\step5.py”, line 415, in ensure_weight
for _,land,ocean in data:
File “E:\Python\Gistemp6\tool\giss_io.py”, line 1170, in step5_input
for land, ocean in data:
File “E:\Python\Gistemp6\tool\giss_io.py”, line 1204, in ensure_landocean
for series in data:
File “E:\Python\Gistemp6\tool\giss_io.py”, line 1147, in step4_output
for land,ocean in data:
File “E:\Python\Gistemp6\tool\giss_io.py”, line 1020, in step3_output
for thing in data:
File “E:\Python\Gistemp6\code\step3.py”, line 296, in step3
for box in box_source:
File “E:\Python\Gistemp6\code\step3.py”, line 162, in iter_subbox_grid
station_records = list(station_records)
File “E:\Python\Gistemp6\tool\giss_io.py”, line 1004, in step2_output
for thing in data:
File “E:\Python\Gistemp6\code\step2.py”, line 687, in step2
for record in adjusted:
File “E:\Python\Gistemp6\code\step2.py”, line 67, in urban_adjustments
rural_stations, urban_stations, all = annotate_records(record_stream)
File “E:\Python\Gistemp6\code\step2.py”, line 124, in annotate_records
station = record.station
AttributeError: ‘Series’ object has no attribute ‘station’
October 28th, 2010 at 1:52 pm
I deleted all of the program and data and downloaded the tar.gz again.
Reinstalled and the same error occurred.
Restarted the program and it appeared to successfully run all steps.
It once again displayed the following line, but didn’t error out.
1 USHCN records had no GHCN counterpart.
Evidently all steps can’t be run while installing the program and data files.
October 28th, 2010 at 2:34 pm
@Bos Koss: Thanks for this bug report (you should join the googlegroup). Someone else spotted the same bug earlier. As you observe, you have to run it once to download the files (and it will crash); and run it again and it should work. I’ve created issue 88 (ah, I think… googlecode is down), and I’ll fix it at some point.
Sorry.
October 28th, 2010 at 2:38 pm
@Bob Koss: re “1 USHCN records had no GHCN counterpart.”
There is a USHCN record that is not also in GHCN. GISTEMP had to change their code to cope with this (see 2010-10-14 http://data.giss.nasa.gov/gistemp/updates/ ).
ccc-gistemp copes just fine in this situation (it retains the USHCN record and carries it through the analysis) [edit: 2010-10-29: but only if you use the updated v2.inv and ushcn2.tbl files from GISS, which you normally will], but it emits the message because it’s an unusual situation. Or was, until a recent change USHCN v2.
Perhaps when GHCN v3 is up to speed properly, we won’t have this USHCN / GHCN merge thing.
October 29th, 2010 at 6:10 am
Thank you for informing me on the status.
October 29th, 2010 at 9:56 am
@Bob Koss: There’s a new release that fixes this bug (hopefully, my test of 0.6.1 is still running, but it certainly gets further than it used to).
I’ve updated the main article.
November 24th, 2011 at 10:40 pm
Hi
I have wondered if it would be possible to replicate fig. 6 of Hansen et. al, 1999 using your program. The figure is a plot of the USA average temperature anpmaly. This figure looks significant different in later versions than in the original paper, see this gif anmation:
http://stevengoddard.files.wordpress.com/2010/10/1998changesannotated.gif?w=500&h=355&h=355
Would it be possible to replicate both versions? that is:
1) using data up to 1999
2) using most recent data
I understand that the reason why the versions differ is due to the way that the reference station method works. As new data come in the estimate of earliier anomalies change since the whole timeseries for he station are used to calculate the offsets.(This could probably be explained better by someone with more knowledge of the algorihm than me)
November 29th, 2011 at 12:22 pm
Søren: we don’t have the 1999 GISTEMP code or data. Given a 1999 dataset, and a suitable land mask, ccc-gistemp could be run to produce a similar chart (although it would not reflect the exact state of GISTEMP code as it was in 1999).
December 11th, 2011 at 2:35 pm
Thank you for the reply.
To be more specific, I would like to know if there is a parameter in ccc-gistemp that specifies the end year for the data that is used as input?