Bureau of Meteorology database bug

With temperatures in Western Australia and across the country in 2009/2010 reported by the Bureau of Meteorology to be among the hottest ever recorded, it's worth being aware of database adjustments made by the BOM during that period.

BOM data indicates that Australia’s annual mean temperature for 2009 was .9°C above the 1961/90 average, making it the country’s second warmest year since "high quality" records began in 1910. WA's average temperature during the 12 months to the end of June 2010 was 23.55°C, matching the previous record.

Other analyses of the BOM's historic data for these 32 locations might calculate the averages a few points of a degree higher or lower, dependent upon different criteria, but it is irrelevant in the context of this discussion as 12.96C and 25.12C are the fixed baselines for all my minimum and maximum comparisons since 2008.

I update my website each month with the latest BOM data and when doing my update in December 2009 I noticed the mean minimum and maximum temperatures had all been adjusted up by about half a degree from the temperatures that had previously been recorded on the BOM website database for August 2009.

Below are the 32 locations with their original data in the top green row as posted by the BOM from September 1 to November 17, and their new adjusted data in the red row below:

August 2009:

I questioned the BOM as to what had happened and on December 7 2009 received this reply …

"Thanks for pointing this problem out to us. Yes, there was a bug in the Daily Weather Observations (DWO) on the web, when the updated version replaced the old one around mid November. The program rounded temperatures to the nearest degree, resulting in mean maximum/minimum temperature being higher. The bug has been fixed since and the means for August 2009 on the web are corrected."

The temperatures for all WA locations in the BOM web database remain at the higher adjusted temperatures for August 2009 and there was no change from the adjustment made on November 17 prior to my correspondence.

I was confused by this so I sent another question to the BOM, as below and with the response on December 9 ...

"Thanks for the reply but I'm still a bit puzzled. Below are August 2009 min and max data for 32 WA sites I keep an eye on, the top temps being those recorded on the BoM website from Sep 1 to Nov 17, and the bottom temps the updated version since Nov 17. The rounded temperature bug seems to have only affected August as there has been no adjustment to any other month including September. I've just checked the BoM site and there's no change to the August data. Can you please let me know which of the temps below are correct?"

Reply: "The bottom ones are correct. The bug only affected averages for August 2009."

Since then I have continued to update my site each month with BOM data to compare the ongoing 12 month average at 32 WA locations with my fixed 1900 baseline of minimum 12.96C and maximum 25.12C ...

The data shows averaged annual temperature across WA rose sharply from November 2009, the month the BOM software bug was corrected to adjust August 2009 average temperatures up by about half a degree. The above data is graphed below:

This baseline 1900s comparison shows the average maximum in the year to December 2011 remains about .4 C higher than before November 2009, with the average minimum about .7C higher.

BOM monthly data is available for all WA recording locations with historic averages which can be used to verify the trend I calculate from 32 locations compared to 1900 baseline temperatures.

The sharp upward trend compared with the 1900 average is reflected in BOM data below (BOM website source) which shows the overall WA mean temperature jumped by almost 1 degree C beginning in the spring of 2009 between August and November, broken into seasons to give some averaged time perspective.

The statewide data above shows temperatures dropped in the winter of 2010 to be just above the long-term average, although this moderating of temperatures is not reflected in the historic baseline comparison for the 32 major centres.

The BOM explains its baseline for the above anomalies thus: The climatological averages shown in the text and tables are generally long-term means based on observations from all available years of record, which vary widely from site to site. They are not shown for sites with less than 30 years of record, as they cannot then be calculated reliably. Climatological extremes are generally taken from available observations from all available years of record. The number of years can vary substantially from site to site.

The BOM also compares temperatures with the baseline average of 22.5C from 1961 to 1990. Western Australia's historic mean temperature anomaly trend can be seen below:

It is worth noting that, according to the BOM, the statewide mean maximum temperature for the three months October to December 2009 was the highest on record. August 2009 had the second highest mean maximum and the fifth warmest mean minimum since records began in 1950.

According to the BOM's Annual Australian Climate Statement 2009: An unusual winter-time heatwave occurred during August over large parts of inland Australia and resulted in Australia’s warmest August on record. A third prolonged heatwave occurred during November across central and southeast Australia, leading to a record 8 consecutive days of maximum temperatures above 35°C in Adelaide, and numerous maximum temperature records across southern and eastern Australia, especially in South Australia and New South Wales. Based on the analysis of daily (maximum and minimum) temperature data above and below set thresholds, there are clear upward trends in the number of hot events and downward trends in the number of cold events (over the period 1960 to date), consistent with the background of global warming.

The mean minimum and maximum across WA in autumn 2010 were the fourth and sixth highest respectively. January 2010 had the third highest average maximum and the second highest average minimum on record. February 2010 had the 4th hottest average minimum and maximum ever recorded.

The summer months of 2009/2010 were the hottest on record in WA and the national average in 2010 has been biased by temperatures well above average in the western half of the continent.

So has the weather been exceptionally hot across WA in the 12 months to July 2010 or did a BOM database adjustment in November 2009 influence subsequent temperature readings?

It is worth noting the frequency of adjustments on the BOM database as presented on the web.

In July 2010 I checked every month back to June 2009 in all 32 locations within my records against the BOM database as currently presented on the web. I found 43 adjustments by the BOM, the most consistent being January 2010 wherein most but not all locations saw a small reduction in their mean maximum:

When all the BOM adjustments are calibrated into my averages for the 12 months to July 2010, there is no change at all to the overall minimum and maximum averages ... i.e within adjustments by the BOM to its database since November to its monthly records since September 2009, the upward adjustments almost perfectly counter the downward adjustments. The consistency of adjustments made to January 2010 records suggests a database bug correction was again responsible for the "incorrect" original recording.

The increase in WA average temperatures coinciding with the month that a BOM database bug was corrected does not necessarily mean the Daily Weather Observations on the BOM website are inaccurate. It may well have been a very hot year across Western Australia.

However, the presence of software bugs for extended periods, as well as sporadic adjustments without a seeming pattern, raise questions about the reliability of data from the BOM website for research purposes.

Similar questions surround the reliability of Australian temperature recordings by the Goddard Institute of Space Studies.

This website produced by Scribeworks 2009/11