Lies, damn lies and statistics… or the story of how 10 households using the Green Deal financing option is inspirational
Yesterday, DECC issued a press release announcing in triumphal tones that “New research shows the Government’s Green Deal is inspiring people across the UK to install energy saving home improvements.” The release stated that 47% of those households which had received a Green Deal assessment report had installed or were in the process of installing an energy saving measure and that a further 31% said they were likely to. This, of course, gives the impression that 78% of those having an assessment were on the cusp of installing energy saving measures. A more careful reading of the press release reveals that percentage to be taken from those that have received a report following the assessment – on which more below. The undoubted tone of the statement is that the Green Deal is working as an incentive to encourage the installation of energy efficiency measures. Whether the fact that less than half those having an assessment have actually acted on the advice is a success is debatable but not the main point of this post. My issue is with the way that the statistics have been presented to give a rather biased spin on the success of the Green Deal to date.
The press release was based upon this report. Admirably, this research (and a second study tracking progress) have been commissioned by DECC in order to provide useful information with which to evaluate the Green Deal. The raw data on which the report is based is available here.
Having read the methodology and downloaded the data, my rather quick and dirty analysis tells a rather different story to that told in the press release.
The report reveals that up to March 31st 2013, a grand total of 9000 Green Deal Assessments had been undertaken. The “official launch date” of the Green Deal is given as 28th January 2013 – this is in itself interesting, as the launch date has been somewhat difficult to pin down as shown by Guertler et al. (2013) (hat-tip to their excellent paper for the information behind the following sentence). As recently as September 2012, Energy Minister Ed Davey said “Most aspects […] will start on 1 October […] finance plans only from January 2013.” – although this was qualified by his colleague Greg Barker in October 2012 – “ [that] was not, in fact, a launch.”. One wonders whether the 9000 assessments were undertaken in the period 1 October 2012 – 31 March 2013, or between 28 January 2013 and 31 March 2013…
Of these 9000 or so, 900 were selected to be surveyed for the report – with 507 eventually responding to the survey. This seems a reasonable sample. Of these, the data sheet (Q10) shows that 64% had received their report following the assessment – with a further 13% unsure whether they had received it or had it sent to their landlord. Interestingly, though – the data sheet shows the base sample for those who had received a Green Deal Assessment Report (GDAR) to be 285 respondents (Q17 of data sheet) – which appears to include those whose assessment was arranged by a landlord. Which would be 56% of 507. The source of this discrepancy is unclear from the data sheet and worthy of further investigation.
Working with that sample of 285 – the data sheet then goes on to show which measures had been recommended to various householders, and for each of the improvements whether the respondents2:
- Have had this done
- In the process of doing this
- Definitely will do this
- Probably will do this
- Might or might not do this
- Probably won’t do this
- Definitely won’t do this
Questions 19-32 in the survey relate to subgroups of the respondents who have installed (A), are about to install (B), are likely to install (C&D), Might or might not (E) or won’t install (F&G) measures. The motivation for their decision and choice of financing option are reported. These are examined below, however a caveat is needed at this point. The percentages presented in Qs 19-30 are rather difficult to interpret. They appear to be related to questions asked per measure, but presented as a single percentage figure for each category. Thus, the percentages do not sum to 100% – often summing to, for instance, 150% or more. This may be due to two factors – one respondent may respond twice if they have acted on two measures from their report. In addition some of the categories are not mutually exclusive – for instance a respondent who paid for a measure from their savings may also take advantage of the Green Deal cashback scheme – both of which are options on the financing questions (Qs 19, 25 and 29).
From the base sizes given for questions 19-21 (those who have already installed a recommended key measure 1) and questions 22-25 (those who are in the process of installation) we can determine the number of households in each of those categories. These are 91 and 43 respectively. So, in all – 134 households have installed or are in the process of installing a measure recommended following a Green Deal Assessment. And 134 of 285 is indeed 47%. However – this figure makes no mention of the 222 respondents who have had an assessment but no report. This leaves almost half those who have had an assessment out of the calculation – given considerable uncertainty to the 47% figure – although as mentioned previously the press release is carefully worded to state that the 47% is of those who have received a report
Next, I looked at the methods by which those installing measures had financed them. This, surely, is an important consideration as the Green Deal is designed as a financing scheme to encourage investments and, as former Energy Minister Chris Huhne put it, “The Green Deal is about putting energy consumers back in control of their bills and banishing Britain’s draughty homes to the history books” DECC (2011), quoted in Guertler et al (2013).
On face value (taking account of the caveats above regarding presented percentages), I conclude that only 7 households have taken advantage of the Green Deal Financing scheme and got to the point where an energy efficiency measure has been installed. A further 3 households intend to use Green Deal financing to finance measures being installed currently. So, in total, it appears that 10 households are using the Green Deal financing option of the 507 who have had the initial assessment. Those who have not started any work, but are “definitely” or “probably” going to install measures are more positive about the financing option – with 31% of the 86 involved stating that they intend to use Green Deal financing – some 27 respondents. This is not quite the message that would be gleaned from the press release.
Understandably, the Green Deal Cashback scheme (which is effectively a capital subsidy for works undertaken) is more popular. It appears that ~15 respondents that have had work completed have used the cashback voucher scheme – with a further 9 of those currently having work carried out intending to use the cashback.
Tellingly – when asked why they did not or will not use Green Deal financing 18% of the 150 in this category said they “were not aware of it”, with a further 23% saying that they “Don’t like borrowing/taking out finance/prefer to pay up-front”. These seem like major issues for a government “flagship policy”. This should all be set against a backdrop where orders for energy efficiency measures have fallen through the floor (non-paywalled summary here), whilst the Energy Minister Ed Davey announces in this press release that ““This is great news for the energy efficiency industry as well, because this shows a genuine appetite among householders for more energy efficient homes.”
The government should be lauded for making data available to those who want to delve behind the headlines. However, the presentation of those data in a way that rather over-eggs the success of policies, whilst unsurprising, is less laudable.
The first official statistics on the Green Deal are due tomorrow. I await them, and how they are reported, with bated breath.
As always – comments are welcome.
References:
Pedro Guertler, David Robson and Sarah Royston (2013) Somewhere between a ‘Comedy of errors’ and ‘As you like it’? A brief history of Britain’s ‘Green Deal’ so far in proceeding of ECEEE Summer Study 2013 – available from http://www.ukace.org/wp-content/uploads/2013/06/1-306-13_Guertler.pdf
DECC 2011 “Homes and economy to benefit from energy and climate policies – Huhne” Department of Energy & Climate Change – available from https://www.gov.uk/government/news/homes-and-economy-to-benefit-from-energy-and-climate-policies-huhne
Footnotes:
1. Note that this does not mean that the respondent has installed everything that was recommended on the GDAR, but that at least one measure was installed.
2. It should be noted here that this information is recorded per measure recommended to the respondent. It is to be expected that some households had few (if any) measures recommended, whilst others had multiple measures recommended.
Nice stats!
Seriously a great piece of work there in unravelling the whirlwind of spin that are the Gov press releases. See my guest blog here written several weeks ago for EcoBuild via CIBSE published yesterday http://www.ecobuild.co.uk/news-social/ecobuild-blog/post/2013/06/24/the-green-deal.html
or the paper that we sent to DECC which they must be still reading as they acknowledged receipt and intention to respond weeks ago. http://greendealreport.info/.
Thanks for the comment Janet. I read your blog post and will have a look at the paper. It seems that the GD is looking a rather sickly shade of green from whichever angle you look.
it was a very common mistake to think of green deal as purely a finance mechanism. However, it is much more than that. The planet doesn’t care what finance is used to install an energy saving improvement – only that the improvement is installed.
[edit – reposting so it doesn’t appear to come from my twitter account. Note to self – Must get better at WordPress]
Hi @linniR. You make a good point – the Green Deal should not be seen purely as a finance mechanism. Absolutely the planet does not care and I am all for using as little energy as possible as efficiently as possible – I’m an engineer by background and that’s what we try to do 🙂
However, I think the financing mechanism is often touted as the innovative part of the policy by the government. I tried to devote the majority of the post to the way in which the statistics were reported, rather than a comment on the policy design itself. However, I do have views on that. I do think there is a structural problem with the policy in that it is presented as a financing option – a loan in effect – and many people do not want to take out loans. Particularly at the moment. The interest rate is not really attractive, even for those who do want to look into taking a loan, and I am not sure that people generally trust that the repayments will be met easily by what they save on energy bills. It’s particularly hard to convince people of that in the context of rising retail energy prices as you have to give the message that people seeing increased costs are saving compared to even higher bills they could have had. Not impossible – but not simple, either.
Although the assessment and report phases could happily stand alone and could encourage people to install measures with their own financing (and indeed the stats show that a number have), it is very difficult to say whether that effect is greater or less than advice campaigns or, for instance, the preceding CERT and CESP schemes. The fall in cavity wall insulation (referred to in the post) might suggest it is less effective – but a longer run of statistics should be allowed before that conclusion is robust.
The stats to June are out today – showing countrywide 4 pending (i.e. active) Green Deal plans with a further 241 new plans (which end up translating into GD financed improvements). This from 38,259 assessments. I don’t think we can work out how many of those resulted in some improvements being made with finance outside the GD. This is a problem in my view – but this is a political point (what isn’t?): I think that it would be useful for the government to have a handle on efficiecy improvements made to the buildign stock – I rather think the current administration would view that as none of the state’s business. Let us hope, though, that a fairly large number have been otherwise financed!