Comments on: On integers, floating-point numbers, and rounding http://clearclimatecode.org/on-integers-floating-point-numbers-and-rounding/ Sat, 04 Sep 2010 02:39:50 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Nick Barnes http://clearclimatecode.org/on-integers-floating-point-numbers-and-rounding/comment-page-1/#comment-2387 Nick Barnes Sun, 13 Jun 2010 18:49:54 +0000 http://clearclimatecode.org/?p=165#comment-2387 <i>Why are Fahrenheit numbers being used in scientific data anyway?</i> Because it is US weather data, which has always been, and continues to be, recorded in degrees Fahrenheit. Degrees Fahrenheit may not be my favoured unit, but they are a respectable scientific unit nonetheless. Any more dissing of Fahrenheit is off-topic here. Why are Fahrenheit numbers being used in scientific data anyway?
Because it is US weather data, which has always been, and continues to be, recorded in degrees Fahrenheit. Degrees Fahrenheit may not be my favoured unit, but they are a respectable scientific unit nonetheless.
Any more dissing of Fahrenheit is off-topic here.

]]> By: anon http://clearclimatecode.org/on-integers-floating-point-numbers-and-rounding/comment-page-1/#comment-2374 anon Tue, 08 Jun 2010 23:50:16 +0000 http://clearclimatecode.org/?p=165#comment-2374 Why are Fahrenheit numbers being used in scientific data anyway? Why are Fahrenheit numbers being used in scientific data anyway?

]]> By: Nick.Barnes http://clearclimatecode.org/on-integers-floating-point-numbers-and-rounding/comment-page-1/#comment-293 Nick.Barnes Wed, 24 Feb 2010 17:13:19 +0000 http://clearclimatecode.org/?p=165#comment-293 A comment to correct an error I made in the post. The USHCNv1 dataset has Fahrenheit temperatures to two decimal places, not to one as I stated in the post. For instance, a reading might be "66.83". Reading this produces 4702743173393285.2<sup>-46</sup>, i.e. 66.8299999999999982946974341757595539093017578125. Converting with multiplication first produces 387.2<sup>-1</sup>, i.e. 193.5, exactly the "right" answer (19.35 Celsius == 66.83 Fahrenheit). Converting by division first, or by multiplying by the factor (50.0/9.0), produces 6808175999188991.2<sup>-45</sup>, which is less by 2<sup>-45</sup>. Critically, this latter value rounds to a different integer (193 instead of 194). In this way, explicit rounding and truncation can amplify some floating-point rounding errors. A small proportion of the USHCNv1 dataset is prone to exactly this problem: some of them round up, some down. This problem does not occur with the USHCNv2 dataset, or the other input datasets. As it happens, even these amplified errors balance out in the later parts of the system, especially in the combination, averaging, and anomaly steps, and are not visible in the end results. But still, now that we no longer need to pass the dataset through plain-text intermediate files, the code is clearer without these explicit rounding and truncation steps. A comment to correct an error I made in the post. The USHCNv1 dataset has Fahrenheit temperatures to two decimal places, not to one as I stated in the post. For instance, a reading might be “66.83″. Reading this produces 4702743173393285.2-46, i.e. 66.8299999999999982946974341757595539093017578125. Converting with multiplication first produces 387.2-1, i.e. 193.5, exactly the “right” answer (19.35 Celsius == 66.83 Fahrenheit). Converting by division first, or by multiplying by the factor (50.0/9.0), produces 6808175999188991.2-45, which is less by 2-45. Critically, this latter value rounds to a different integer (193 instead of 194). In this way, explicit rounding and truncation can amplify some floating-point rounding errors. A small proportion of the USHCNv1 dataset is prone to exactly this problem: some of them round up, some down. This problem does not occur with the USHCNv2 dataset, or the other input datasets.

As it happens, even these amplified errors balance out in the later parts of the system, especially in the combination, averaging, and anomaly steps, and are not visible in the end results. But still, now that we no longer need to pass the dataset through plain-text intermediate files, the code is clearer without these explicit rounding and truncation steps.

]]> By: drj http://clearclimatecode.org/on-integers-floating-point-numbers-and-rounding/comment-page-1/#comment-291 drj Wed, 24 Feb 2010 12:21:35 +0000 http://clearclimatecode.org/?p=165#comment-291 @AndyB: Yes, there really is. @AndyB: Yes, there really is.

]]> By: AndyB http://clearclimatecode.org/on-integers-floating-point-numbers-and-rounding/comment-page-1/#comment-290 AndyB Wed, 24 Feb 2010 03:22:42 +0000 http://clearclimatecode.org/?p=165#comment-290 I've never known how floats actually worked - I supposed if pressed I would have guessed something like that given the nature of computing. But, c'mon 2^-45? Is there really discussion out in the blogs that claims floating point arithmetic is at all relevant to the thermometer record? That's just silly on the face of it. Thanks for the lesson! I’ve never known how floats actually worked – I supposed if pressed I would have guessed something like that given the nature of computing. But, c’mon 2^-45? Is there really discussion out in the blogs that claims floating point arithmetic is at all relevant to the thermometer record? That’s just silly on the face of it.

Thanks for the lesson!

]]>