High Quality weather station audit
Western Australia raw temperature trend audit of BoM HQ stations
Independent data verification auditing of temperature records across WA at the 26 High Quality (HQ) weather stations used by the BoM to monitor climate change shows that almost one in every three were originally recorded as whole .0 Fahrenheit degrees prior to September 1972, the month Australia's weather services were metricated and the Fahrenheit temperature scale was replaced by the Celsius scale.
Rounded .0 Fahrenheit and Celsius temperature recordings that collectively could influence trends and create a possible warming fallacy can be found throughout BoM temperature data that the Australian government accepted as accurate when legislating its 2012 carbon tax.
These influences are discussed in Australia's seminal study into the creation of an accurate historic temperature record:
A historical annual temperature dataset for Australia (Torok and Nicholls, BoM Research Centre, Melbourne, 1996)
Weather stations around Australia recorded an upward shift in their mean temperature plateau in the early 1970s.
This research addresses the Torok metrication issues by auditing the distribution of degree fractions for patterns at the 26 BoM High Quality (HQ) weather stations across WA, examining all years of daily temperatures with data as early as 1897 (click floating menu at left for stations).
The available BoM raw temperature data is examined for statistical patterns that an audit would flag as possible errors or biases requiring further research:
This audit of the 26 BoM HQ weather stations across WA shows that there were 1,066,506 minimum and maximum daily Fahrenheit temperatures recorded before September 1972. This includes both the 26 stations and their nearby preceding stations which the BoM uses to calculate the HQ temperature trends since 1910. This totals 52 BoM weather stations (see full list).
Among these daily minima and maxima, 294,401 were recorded as whole Fahrenheit degrees. This is 27.6% of the total record before Celsius conversion starting in September 1972. In an unbiased record where observers accurately logged the tenth of a degree as instructed by the BoM, all ten Fahrenheit fractions should be uniformly distributed.
That is, they should equal 10% each. Further, it can be shown that such a distribution, when converted to Celsius and rounded to one decimal place, gives numbers whose fractional parts are distributed 5.5% for each of zero and .5, and 11.1% for the remainder. A substantial deviation from that distribution indicates that:
The proportions of total Fahrenheit fractions recorded at all 52 stations before 1 September 1972 were:
The high proportions of .1 and .9 are due to the calculation of Celsius degrees that have been converted from Fahrenheit and rounded to the first decimal before conversion back to Fahrenheit, a mathematical conversion that is imprecise. The high proportions of .1 and .9 provide overwhelming evidence that many of these were in fact .0 Fahrenheit recordings and it is highly likely that the percentage of .0 Fahrenheit degrees was about 35%.
A study of conversion patterns between Fahrenheit and Celsius and confirmed by random Fahrenheit data shows that 5/9 of all Fahrenheit to Celsius degrees (to one decimal place) back to Fahrenheit conversions return to their original decimal, while 4/9 will convert one decimal point up or down.
This explains the high proportions of .1 and .9 in the results.
It can be assumed that the true percentage distribution of .1 and .9 for any given weather station is the mean of the distributions for .2 through to .8.
Based upon this tested formula, it can be stated that the true proportion of minimum and maximum .0 Fahrenheit recordings at the 26 HQ weather stations in WA (excluding their neighbouring stations) was 46.17%, which is commensurate with national audit results. The mathematics can be viewed here (Excel download, 90kb).
The higher proportion of .1, .2, .3 and .4 (35.4%) than .6, .7, .8 and .9 (30.4%) also suggests a slight tendency for lower recordings.
The proportions of total converted Celsius fractions recorded at all 52 stations before 1 September 1972 were:
It is important to note that due to the conversion of Fahrenheit to Celsius with rounding, the result will always approximate 5.6% for each of .0 and .5, and 11.1% for the remainder.
The bias is an inevitable consequence of any Fahrenheit to rounded Celsius back to Fahrenheit conversion, but this analysis shows that in many BoM stations there were far less than half the occurrence of .5C prior to September 1972.
Since 1 September 1972, there were 871,617 minimum and maximum Celsius temperature recordings at the 52 BoM stations, among which 156,193 were logged as .0C. This is 17.9% of all fractions recorded since 1 September 1972, suggesting a bias toward rounding of degrees and corruption of the record by replacing other fractions.
The proportions of fractions recorded at all 52 stations from 1 September 1972 to 31 December 2011 were:
This indicates a bias either through observer practice or database error for many .1C and .9C temperatures to be replaced by .0C, the bias decreasing up or down to .5C. The proportion of .1, .2, .3 and .4 (35.6%) compares with .6, .7, .8 and .9 (34.8%).
As an example, to the right is a snapshot of a typical BoM station record (Cape Leeuwin 9518 maxima) categorised by fractions to show the occurrence of 18.0C recordings, including pre-1972 Fahrenheit conversions to Celsius and direct Celsius recordings since September 1, 1972.
Note the far greater frequency of this .0C recording since Celsius metrication in September 1972, and note that the change begins as of that month. This pre and post 1972 pattern is common in many BoM weather station records.
More importantly, most stations since September 1972 have recorded a significantly greater number of whole .0C degrees than other fractions, either because of observation bias or a "bug" in the BoM database that creates profusions of .0C temperatures in the raw BoM records.
Among the 52 WA stations analysed, there were 9,119 days (25 years) of minimum and maximum temperatures that occurred in blocks of at least 10 days with consecutive, unbroken .0C and .5C recordings, all without exception since September 1972.
Note that there is no mathematical bias against .0 and .5 since 1 September 1972 because raw temperatures since then have been recorded in Celsius and there is no conversion from Fahrenheit.
1) This analysis finds that all BoM weather stations recorded a disproportionately high number of temperatures before 1972 that were whole .0 Fahrenheit degrees.
For example, many BoM stations have unbroken periods or are peppered with converted Celsius temperatures before September 1972 such as 20C, 20.6C, 21.1C, 21,7C, 22.2C, 22.8C, etc (= 68.0F, 69.0F, 70.0F, 71.0F, 72.0F, 73.0F, etc). This excludes or is biased against Fahrenheit temperatures to a tenth of a degree that may actually have been 68.4F (20.2C), 70.8F (21.6C), 72.5F (22.5C), 69.9F (21.1C), etc.
The most likely explanation, as noted above by Torok et al, is that large numbers of weather station observers recorded temperatures in whole .0 degrees Fahrenheit before September 1972 instead of recording to the nearest tenth of a degree.
If observers were rounding to the nearest Fahrenheit degree, it is more likely they would round down to the lower whole degree visible on the thermometer. As an example, it is feasible that an observer might truncate a thermometer reading of 72.7F to 72.0F but it is less likely they would increase a reading of 72.3F up to 73.0F.
A truncation test demonstrates the likelihood of rounding down, particularly by weather station observers who were unaware that thermometer readings of .5 should be rounded both up and down - not just down if they were ignoring the BoM instructions to record all Fahrenheit fractions.
The accuracy of all historic temperature records from manual observations is also questionable due to other influences such as thermometer resolution and mercury meniscus, and the impact of temperature screen design corrupting the accuracy of recordings.
A comparison of average annual minima and maxima at all ACORN sites in Western Australia also shows a rapid temperature increase starting in 1972.
Torok et al find no evidence of discontinuities between Fahrenheit and Celsius thermometers but fail in their discussion to consider a systemic non-instrument bias.
They acknowledge that if many observers "truncated" their measurements to the nearest whole degree below the actual measurement prior to metrication, this would result in an artificial warming in the early 1970s.
Their research indicated that examination of field books did not suggest the practice was sufficiently common to produce a major discontinuity in the temperature record, but this analysis suggests field books were an inadequate source for research and whole Fahrenheit rounding was in fact very common before 1972 metrication.
Further evidence of the pre-Celsius bias of truncated Fahrenheit temperatures can be found through analysis of the annual number of days on which the mean or a higher temperature is recorded. All WA stations are analysed here, where examples such as Carnarvon show about a month of average to above average days occurred immediately the temperature scale was changed from Fahrenheit to Celsius in 1972:
Many weather stations within the BoM HQ dataset have more than twice the proportion of .0C days occurring in their record after September 1972.
Temperature records at a majority of WA stations have unbroken months or years of .5C and .0C since 1 September 1972, and/or periods when these rounded degrees are random but in high proportion.
The BoM has explained in the case of Perth Metro 9225 (19 months of daily .0C minima in 1995/96) that the precise temperatures for the affected time period are available but have not yet been supplied to the BoM's national database.
The BoM explains that although in the case of Perth Metro the rounding can produce differences between 0.0C and 0.3C, the effect is negligible and the precise data averages to exactly the same as the rounded data. This claim requires further examination.
Australia's temperature trend rose to a warmer plateau after 1972, the same year temperature scales were metricated from Fahrenheit to Celsius.
All station records show that before decimal Celsius measurements forced a closer observation of tenths of a degree, rounding or truncating to whole Fahrenheit degrees was a common practice before 1972 when all thermometers were read manually.
In the years following 1972, BoM weather stations began to record large numbers of Celsius degrees rounded to .0C and/or .5C and these were recorded randomly or sometimes in unbroken blocks spanning months or years. Many but not all observations switched from being manual to electronic when the BoM began the installation of more than 550 Automatic Weather Stations around Australia. The AWS records air temperatures to an accuracy within .3C.
The .0 Fahrenheit rounding occurred before 1972 and the .0 Celsius rounding since that year. The two rounding phenomena are not related but each influences the accuracy of temperature trends before and after 1972.
With at least 27.6% of all WA temperatures recorded before September 1972 being whole .0 Fahrenheit, raw BoM temperature data overwhelmingly suggests "an artificial warming in the early 1970s".
The BoM might argue that this analysis considers raw instead of adjusted HQ temperatures and is thus invalid.
However, this analysis finds commonalities between the raw and HQ data and reasons that if the HQ dataset is adjusted from raw data, the latter should be investigated to ensure that all HQ adjustments have been made appropriately.
How does the HQ algorithm accurately adjust for almost a third of the raw data before 1972 metrication being whole Fahrenheit degrees that may or may not have been inaccurately recorded by many thousands of observers over many decades? Why are twice as many temperatures recorded to a whole .0C since September 1972 and does this bias recent trends up or down?
With the advent of a carbon tax that aims to fundamentally alter the economy and lifestyle decisions of Australians, the full BoM temperature record should be audited to ensure the data patterns identified in this analysis have not created an "artificial" warming bias.
In November 2009, the BoM adjusted all WA temperature recordings for August 2009 up by an average .4C.
When queried, the BoM explained that "... there was a bug in the Daily Weather Observations (DWO) on the web, when the updated version replaced the old one around mid November. The program rounded temperatures to the nearest degree, resulting in mean maximum/minimum temperature being higher. The bug has been fixed since and the means for August 2009 on the web are corrected ... the bug only affected averages for August 2009."
The .4C upward adjustment of temperatures for August 2009 remained (read more).
HI QUALITY audit|