Statistics, rain, flooding and climate?

Discussing the causes and effects of climate change tends to be emotive.  Even more so on social media than in person, it seems.  A playful prance through the first page of results from Google searching “climate change blog” can convince you of that if necessary.  Although I normally stay out of such discussions – particularly when they resort to hurling polemic and ad hominem comments. However, I have been tempted to offer my opinion on one particular piece and so here goes.

A 5-sigma event (or not)

I happened to notice a comment on twitter which said “how can 2 x ave be a 5-sigma event?” and I dipped my toe in the water (admittedly without context at this point) and responded to the effect that “2 x mean of a random variable could be 5-sigma event, you need to know the variance”.  The exact verbatim tweets, ensuing discussion and my commentary are storified here.  If you need a reference for what sigma refers to here – Wikipedia is sufficient.

I realised that the prompt for the conversation was this blog post.  In it, the author makes the assertion that “The winter of 2013/4 has seen a 5-sigma event in southern Britain.”.  The article doesn’t actually say what that event was, so it is very hard to test the claim, but the context given in the article coupled with recent news stories implies that what it refers to is total rainfall  link it with the Somerset flooding and the met office chief scientist’s statement on the same [1].  The addendum to the blog post implies that the 5-sigma event in question is the total rainfall in January 2014. Before I investigate that claim, I’d like to make clear that I do not doubt whatsoever the motives or integrity of the blog writer (who I do not know).  I imagine the blog to be written from a perspective in which climate change is the largest problem that the world faces and that many perceived instances of “worsening” weather seem to indicate that it is happening apace.

That being said, there is no data referenced or provided to justify the very specific 5-sigma claim.  Much of what is written on the blog about distributions is reasonable – but it is generic and does not cover the application to the specific dataset in question – other than to point out that rainfall cannot be negative and therefore the assumption that the distribution is Normally distributed is just that – an assumption.  However, it’s a reasonable assumption – commonly made in first order analysis of the type that we discuss here.

If we assume that the variable in question is the total January rainfall, The Met Office provides the data by which we can test the claim. I looked at this page and calculated the mean and Standard Deviation for the  monthly total rainfall for January for some weather stations near to Somerset.  I’m aware I could and should consider even more data to test the claim exhaustively [2]. I’m also aware that I should probably not use data series of different lengths.  That’s not my point here – as the following section says.  This is provided only to verify the findings of the twitter debate.

It seems then, that the 5-sigma claim is wrong – experiencing twice the mean rainfall at these weather stations is roughly a 2-sigma event and observing 3 times the mean a 4-sigma event.  It seems unlikely that it would be right even if the exact months under consideration were slightly different to those I have tested – although I’m happy to rerun the analysis on any data provided. What is remarkable to me is how consistent these distributions are – with sigma being ~= half the mean value. This means that seeing 3x the mean value is about a 4-sigma event. Which is very rare.

The political debate

Even had the maths been correct, though, I think there is a deeper problem here. On the surface, the debate is about whether a statistical claim is correct. I contend that what is really at issue are differing (political) belief-systems and the use of statistics to inform (or at least bolster) their claims.

To be clear, the number being calculated and debated is the probability that a single extreme weather event will happen if no Climate Change is happening. The premise of using these statistics and terms is that the observations are drawn from an underlying “true” stationary random distribution.  The implied argument from the blog post, the Guardian articles and Zoe Williams’ tweets appears to run as follows:

1. If the observation is very unlikely, then the distribution must have changed.
2. The distribution changing implies climate change (and often the anthropogenic forcing element thereof).
3. If the climate is changing – that cannot be coped with using Business-as-usual methods.

These jumps are not dictated by the data, but rather depend on a number of judgements – all of which may be debated.

The first jump requires that observing a very unlikely event means the underlying distribution has changed.  This may or may not be the case. The point of probability is that rare events can happen, even with vanishingly small chance and that is why we put a number to that chance. Frequent occurrence of events that your statistical model says should be rare / unlikely may indicate that your statistical model is wrong. For those interested in when one should change ones model – see the Ludic fallacy from Nicholas Taleb’s Black Swan and the difference between frequentist (what we’re doing here) and Bayesian statistics – you could start here or, more humourously, here.

The second jump depends on more judgements.  Firstly, that climate change, would raise the total January rainfall, or increase the variance in total January rainfall, making the extremely unlikely event significantly more likely. I am not a meteorologist, so can’t judge this.  It’s worth noting that climate science appears to indicate that warming would increase the amount of water that can be carried in the atmosphere and therefore that we can expect “more intense daily and hourly rain events.” [3]. More intense events at short scale, though, does not necessarily imply increased total. Secondly. it also assumes that nothing else could account for a changing distribution.  In the case of the particular blog post under discussion, the article tacitly conflates the flooding in Somerset with increased rainfall (neglected some other causes which could confound the analysis – amongst others the much reported lack of dredging).

The last jump, it seems to me, is rarely debated in the public realm. I condense it as “whether the weather can be weathered”.  Some, no doubt, can: some not. The more subtle debate about what can and can’t be adapted to tends to be lost in hyperbole emitted by those with honestly held beliefs either that we are piling headlong into climate catastrophe, or that we are ploughing money and effort into something that is not a problem. From my point of view, this is the conversation that must be had.

I think that Climate Change probably is happening.  I base this on a judgement about the mechanisms I have seen described in scientific papers.  But it is a judgement and I try to keep it under constant review – in common with any of my scientific judgements and especially in the light of any new evidence and data.  I think that the analysis of weather and determining when it can be said to have changed significantly enough to indicate climate change beyond reasonable doubt is incredibly difficult.  As well as dealing with single measurements as described here, it needs to take account of a complex mix of observations across many measurements and the number and frequency of “out of the ordinary” events over time.  Rigorously.  It seems to me silly to get bogged down in arguments about statistical mistakes, or to vigorously sling mud back and forth.

The best we can say about individual observations is that they may add to the evidence for a changing climate.

We’re having the wrong debate

One observation, however unlikely, cannot prove or disprove a change in underlying distribution.

Fundamentally, the rarity or otherwise of individual weather observations cannot, in my opinion, provide conclusive evidence for or against climate change. Sophisticated analysis of multiple events is necessary – and this is what groups of scientists at the Met Office (and elsewhere) do.  It seems to me a waste of time and effort for us to quibble over stats, or to cite them as evidence for our arguments unless we’re absolutely sure of what they say and the argument that they can support.

Aside: possible further work…

As a nod to some useful further work, I think an interesting, different approach to connecting extreme weather events with climate change might be possible.  I think a Bayesian framework would help. As I’m not a meteorologist, I have no idea whether this is unusual, or done as a matter of course.  This is a useful technique to update a priori probabilities in light of observations.  It should be able to give a probability for whether Climate change is happening given observations if we have knowledge of the probability of making the observations given climate change is (or is not) happening.  I haven’t got an exact formulation of such an approach, but aim to investigate prior research or formulate my own approach (or both) and blog on this in the near future

Footnotes

1. Dame Julia Slingo is careful in her wording – saying

Dame Julia Slingo said the variable UK climate meant there was “no definitive answer” to what caused the storms. “But all the evidence suggests there is a link to climate change,” she added. “There is no evidence to counter the basic premise that a warmer world will lead to more intense daily and hourly rain events.”

2. I am tempted to write some software to take these data and produce distribution plots, mean and variance. Watch this space – I may eventually have time!

3. See footnote 1 above