Statistics, rain, flooding and climate?

Discussing the causes and effects of climate change tends to be emotive.  Even more so on social media than in person, it seems.  A playful prance through the first page of results from Google searching “climate change blog” can convince you of that if necessary.  Although I normally stay out of such discussions – particularly when they resort to hurling polemic and ad hominem comments. However, I have been tempted to offer my opinion on one particular piece and so here goes.

A 5-sigma event (or not)

I happened to notice a comment on twitter which said “how can 2 x ave be a 5-sigma event?” and I dipped my toe in the water (admittedly without context at this point) and responded to the effect that “2 x mean of a random variable could be 5-sigma event, you need to know the variance”.  The exact verbatim tweets, ensuing discussion and my commentary are storified here.  If you need a reference for what sigma refers to here – Wikipedia is sufficient.

I realised that the prompt for the conversation was this blog post.  In it, the author makes the assertion that “The winter of 2013/4 has seen a 5-sigma event in southern Britain.”.  The article doesn’t actually say what that event was, so it is very hard to test the claim, but the context given in the article coupled with recent news stories implies that what it refers to is total rainfall  link it with the Somerset flooding and the met office chief scientist’s statement on the same [1].  The addendum to the blog post implies that the 5-sigma event in question is the total rainfall in January 2014. Before I investigate that claim, I’d like to make clear that I do not doubt whatsoever the motives or integrity of the blog writer (who I do not know).  I imagine the blog to be written from a perspective in which climate change is the largest problem that the world faces and that many perceived instances of “worsening” weather seem to indicate that it is happening apace.

That being said, there is no data referenced or provided to justify the very specific 5-sigma claim.  Much of what is written on the blog about distributions is reasonable – but it is generic and does not cover the application to the specific dataset in question – other than to point out that rainfall cannot be negative and therefore the assumption that the distribution is Normally distributed is just that – an assumption.  However, it’s a reasonable assumption – commonly made in first order analysis of the type that we discuss here.

If we assume that the variable in question is the total January rainfall, The Met Office provides the data by which we can test the claim. I looked at this page and calculated the mean and Standard Deviation for the  monthly total rainfall for January for some weather stations near to Somerset.  I’m aware I could and should consider even more data to test the claim exhaustively [2]. I’m also aware that I should probably not use data series of different lengths.  That’s not my point here – as the following section says.  This is provided only to verify the findings of the twitter debate.

Station Data series dates mean Standard Deviation (sigma) sigma level (2 x mean) sigma level (3 x mean)
Hurn 1957-2013 88.6 42.6 2.1 4.2
Hurn 1957-2013 88.6 42.6 2.1 4.2
Yeovilton 1957-2013 70.5 31.6 2.3 4.5
Oxford 1853-2013 56.0 27.8 2.0 4.0
Southampton 1853-2013 81.5 43.7 1.9 3.7

It seems then, that the 5-sigma claim is wrong – experiencing twice the mean rainfall at these weather stations is roughly a 2-sigma event and observing 3 times the mean a 4-sigma event.  It seems unlikely that it would be right even if the exact months under consideration were slightly different to those I have tested – although I’m happy to rerun the analysis on any data provided. What is remarkable to me is how consistent these distributions are – with sigma being ~= half the mean value. This means that seeing 3x the mean value is about a 4-sigma event. Which is very rare.

The political debate

Even had the maths been correct, though, I think there is a deeper problem here. On the surface, the debate is about whether a statistical claim is correct. I contend that what is really at issue are differing (political) belief-systems and the use of statistics to inform (or at least bolster) their claims.

To be clear, the number being calculated and debated is the probability that a single extreme weather event will happen if no Climate Change is happening. The premise of using these statistics and terms is that the observations are drawn from an underlying “true” stationary random distribution.  The implied argument from the blog post, the Guardian articles and Zoe Williams’ tweets appears to run as follows:

  1. If the observation is very unlikely, then the distribution must have changed.
  2. The distribution changing implies climate change (and often the anthropogenic forcing element thereof).
  3. If the climate is changing – that cannot be coped with using Business-as-usual methods.

These jumps are not dictated by the data, but rather depend on a number of judgements – all of which may be debated.

The first jump requires that observing a very unlikely event means the underlying distribution has changed.  This may or may not be the case. The point of probability is that rare events can happen, even with vanishingly small chance and that is why we put a number to that chance. Frequent occurrence of events that your statistical model says should be rare / unlikely may indicate that your statistical model is wrong. For those interested in when one should change ones model – see the Ludic fallacy from Nicholas Taleb’s Black Swan and the difference between frequentist (what we’re doing here) and Bayesian statistics – you could start here or, more humourously, here.

The second jump depends on more judgements.  Firstly, that climate change, would raise the total January rainfall, or increase the variance in total January rainfall, making the extremely unlikely event significantly more likely. I am not a meteorologist, so can’t judge this.  It’s worth noting that climate science appears to indicate that warming would increase the amount of water that can be carried in the atmosphere and therefore that we can expect “more intense daily and hourly rain events.” [3]. More intense events at short scale, though, does not necessarily imply increased total. Secondly. it also assumes that nothing else could account for a changing distribution.  In the case of the particular blog post under discussion, the article tacitly conflates the flooding in Somerset with increased rainfall (neglected some other causes which could confound the analysis – amongst others the much reported lack of dredging).

The last jump, it seems to me, is rarely debated in the public realm. I condense it as “whether the weather can be weathered”.  Some, no doubt, can: some not. The more subtle debate about what can and can’t be adapted to tends to be lost in hyperbole emitted by those with honestly held beliefs either that we are piling headlong into climate catastrophe, or that we are ploughing money and effort into something that is not a problem. From my point of view, this is the conversation that must be had.

I think that Climate Change probably is happening.  I base this on a judgement about the mechanisms I have seen described in scientific papers.  But it is a judgement and I try to keep it under constant review – in common with any of my scientific judgements and especially in the light of any new evidence and data.  I think that the analysis of weather and determining when it can be said to have changed significantly enough to indicate climate change beyond reasonable doubt is incredibly difficult.  As well as dealing with single measurements as described here, it needs to take account of a complex mix of observations across many measurements and the number and frequency of “out of the ordinary” events over time.  Rigorously.  It seems to me silly to get bogged down in arguments about statistical mistakes, or to vigorously sling mud back and forth.

The best we can say about individual observations is that they may add to the evidence for a changing climate.

We’re having the wrong debate

One observation, however unlikely, cannot prove or disprove a change in underlying distribution.

Fundamentally, the rarity or otherwise of individual weather observations cannot, in my opinion, provide conclusive evidence for or against climate change. Sophisticated analysis of multiple events is necessary – and this is what groups of scientists at the Met Office (and elsewhere) do.  It seems to me a waste of time and effort for us to quibble over stats, or to cite them as evidence for our arguments unless we’re absolutely sure of what they say and the argument that they can support.

Aside: possible further work…

As a nod to some useful further work, I think an interesting, different approach to connecting extreme weather events with climate change might be possible.  I think a Bayesian framework would help. As I’m not a meteorologist, I have no idea whether this is unusual, or done as a matter of course.  This is a useful technique to update a priori probabilities in light of observations.  It should be able to give a probability for whether Climate change is happening given observations if we have knowledge of the probability of making the observations given climate change is (or is not) happening.  I haven’t got an exact formulation of such an approach, but aim to investigate prior research or formulate my own approach (or both) and blog on this in the near future


Footnotes

1. Dame Julia Slingo is careful in her wording – saying

Dame Julia Slingo said the variable UK climate meant there was “no definitive answer” to what caused the storms. “But all the evidence suggests there is a link to climate change,” she added. “There is no evidence to counter the basic premise that a warmer world will lead to more intense daily and hourly rain events.”

2. I am tempted to write some software to take these data and produce distribution plots, mean and variance. Watch this space – I may eventually have time!

3. See footnote 1 above

This entry was posted in Uncategorized. Bookmark the permalink.

11 Responses to Statistics, rain, flooding and climate?

  1. WarrenPearce says:

    “We are having the wrong debate”. Thanks for putting this so starkly. Good post.

    The reason why these weather events are perceived as so important, imho, is that they are regarded by the climate concerned/campaigners/convinced etc to be the best lever available to greater ‘action’, whatever that might entail. The ‘looking out the window’ test, as I unscientifically call it. And indeed, there is some evidence to suggest that recent weather events (or at least temperature anomalies) can make people ‘believe’ in climate change http://blogs.nottingham.ac.uk/makingsciencepublic/2013/01/25/weather-1-climategate-0/

    HOWEVER, expecting these weather events to lead to some kind of tipping point in terms of policy seems a bit fanciful. To be blunt, there have been plenty of extreme weather events over recent years where links have been made with climate change, but none of them have resulted in any policy change to my knowledge.

    In lieu of any cultural turn away from increasing energy consumption, the development most likely to result in any ‘action’ on climate change is the shape of the technology which will aid emissions reduction ie a broader, more effective set of renewable tech than we have at the moment. [sorry this is mildly o/t!]

    • jsnape01 says:

      I broadly agree Warren. I have looked at the impacts of direct behaviour (i.e. changing consumption habits / practices etc.) and they make a small difference – usually unsustained. This may change under greater economic pressure, but I think prices would have to become astronomic to see the effects.
      The effect of new technology is much greater. Greater still, in my opinion, is the effect of people’s willingness or desire to adopt / accept / install that technology. This is an argument I make regularly, but still have to refine.

  2. I’ll try again. Failed first time. I looked at the Met Office monthly data for SouthEast England. Average for December is 73.5mm with standard deviation of 37.2mm. Dec 2013 was 138.7, so a 1.75 sigma event. For January, the average is 67.2mm with a standard deviation of 32.3mm. I couldn’t find a definite number for Jan 2014. The only one I could find is for SouthEast and Central which was 175.2mm. That would make Jan 2014 a 3.3 sigma event. If I combine Dec and Jan, I get that combined it’s a 3.6 sigma event, so about 1 in 10000.

    As you say though, there are probably better things to discuss :-)

    • jsnape01 says:

      I am glad – these figures seem to tally almost exactly. 175.2mm is just under 3 x the mean you found (67.2). I estimated this should be just under 4-sigma – you get 3.3. It’s all about the same.

      I agree though – we should discuss other things. There is so much to question about the stats I’d rather leave it to the Met office and simply check when they are emitted (validity approximation to Gaussian / Normal, time over which to measure average / sigma)

      I am much more interested in what should be done. There is a line of argument amongst campaigners which seems to be implicitly: “Climate change is happening, so everyone must listen to me about how to deal with it. Quick, quick, you’re missing the boat, do what I say”. That’s a dangerous and unjustified line, IMO. I’d prefer to see “Climate change seems to be happening, what should we do about it?”

  3. Warren,

    “HOWEVER, expecting these weather events to lead to some kind of tipping point in terms of policy seems a bit fanciful. To be blunt, there have been plenty of extreme weather events over recent years where links have been made with climate change, but none of them have resulted in any policy change to my knowledge.”

    I never quite understand this argument. Certainly in my case, I don’t “hope” that an individual will cause policies to change. What I see is mounting evidence that climate change is happening. What I hope is that policy makers will start considering the evidence more closely and will start factoring the consequences of climate change into their policy decisions.

    You seem to be arguing that policy has never been influenced before and therefore expecting it to be influenced now is fanciful. Well it may be true that it’s never worked before, but if climate change is indeed happening (as I think it is) and if our policy makers continue to ignore the evidence then I think we are in potential trouble. I also think that the people who keep making these arguments aren’t really helping much.

    In lieu of any cultural turn away from increasing energy consumption, the development most likely to result in any ‘action’ on climate change is the shape of the technology which will aid emissions reduction ie a broader, more effective set of renewable tech than we have at the moment.

    I would agree with this. What I find it hard to understand is how we’re expected to develop this technology if we’re continually faced with people who argue that the only way forward is the further use of fossil fuels, that the evidence for climate change is weak, and that any attempt to move to a renewables based energy sector will kill people.

    In my opinion, we need policy makes to start considering the risks associated with climate change and to then start considering whether or not to prioritise investment in new technologies. I don’t see how we can completely separate the development of new technologies from how policy makers view the risks associated with climate change.

    • jsnape01 says:

      Warren can speak for himself, but my reading of it was the individual weather events have never influenced Climate Change policy before. Climate Change plainly has – e.g. we have a Climate Change act. I actually think at the high level, decisions are being made about how we should react to climate change (right or wrong). Just at our lower level, we’re quibbling about stats and science from a position of partial knowledge.

      I would not be so pessimistic about renewable generation development. I think fossil fuels will continue to become more expensive, less economic, thus weakening the only major argument for their continued use, IMO.

  4. jsnape01,

    Just at our lower level, we’re quibbling about stats and science from a position of partial knowledge.

    That’s certainly true (in my case, certainly, at least :-) ). I would argue, however, that there are some relatively high-profile people who use every opportunity to point out that a trend isn’t statistically significant or that an event wasn’t particularly extreme (happened before, for example). So, I’m not convinced that this type of framing isn’t making it to the policy makers. As you and I discussed briefly on Twitter, it would be interesting to see this looked at from a more Bayesian approach. I agree that policies are being made, but there’s also quite extensive pressure to change or remove these policies.

    I would not be so pessimistic about renewable generation development. I think fossil fuels will continue to become more expensive, less economic, thus weakening the only major argument for their continued use, IMO.

    I agree completely with this. I think that fossil fuels will get more expensive and renewables will become a much more obvious alternative. I still think that policy makers considering climate change in the context of the costs and benefits of more investment (both use and development) in renewables would, however, be something I would like to see more of.

    My point was simply that the apparent dismissal of the risks of climate change (that I seem to see happening) does not help – in my opinion – our policy makers to make sensible decisions about future policies.

Leave a Reply

Your email address will not be published. Required fields are marked *