Summary

Lyme Bay Closed Area was a first for British waters.  The very first statutory closed area established for conservation reason, protecting fragile reefs and associated species from the effects of bottom-fishing trawls and scallop dredges.  It was a long process to get there, 16 years of surveys, reports and campaigning.  That it was established is an unqualified success.

Since it was established, annual surveys have been conducted and reports produced by Plymouth University’s Marine Institute describing the phenomenal re-growth that has occurred since the protection was introduced. I was directly involved in these studies, running a specific component (diving surveys 2008-2010).  The study was a DEFRA Science and Research project:

DEFRA Science and Research Projects. Lyme Bay – A Case-study: Measuring the effects of benthic species and assessing potential – MB0101.

The findings reported here suggest that the Closed Area has benefited the marine species living on the rocky reefs to a far greater degree than anyone could have possibly hoped. It seems too good to be true.

What if it is too good to be true?  What if much of the data is not real?

Increasingly concerned about aspects of the study, and after much deliberation, I wrote to the key scientists within Plymouth University’s Marine Biology and Ecology Research Centre and requested that my name as an author on the Lyme Bay study be removed from the final scientific report submitted to DEFRA. This was the first time in 25 years as a marine biologist I have felt it necessary to take such actions.

There are, in my view, three fundamental problems with the study:

• the study design is such that the comparative areas outside the Closed Area are in no way comparable; they were never likely to support similar species assemblages;
• the methodology used is highly unlikely to be capable of detecting the type of changes expected within the study timescale, or capable of detecting many of the species claimed to have been detected, indeed it is highly unlikely that some of the species reported as being recorded actually exist in the locations surveyed;
• the key changes highlighted simply could not have happened; they fly in the face of everything we know about the species and taxonomic groups involved.

I wrote to the key researchers and suggested the report be withdrawn until these points were addressed.  This was rejected.  Aspects of our own study have been incorporated into the final report.  The interpretation of our data is not one I or others working on the diving was involved in, nor one I would concur with.  Our own study was to be published by Natural England as a separate report.  Days before this was due to happen the process was halted.  When querying this I was informed that this was because they were too busy.  More than two years later it has still not been published by Natural England.  It is fair to say that our findings and recommendations do not all concur with those of Plymouth University’s marine biologists.

These issues are troubling. Of greater concern is the opacity of the underlying data on which these findings are based and the apparent lack of interest in both Natural England and DEFRA over very obvious flaws in the study.  This is best illustrated by examples; although this was an imaging based study (species counts through analysis of images collected by remote camera) almost no stills images have been made available, even as part of the study reviews.  Although never made clear, the photographs of species and habitats used in the study reports were not obtained by the camera system employed, nor collected as part of the study and some are clearly not from Lyme Bay.  Attempts to seek confirmation on issues such as the resolution of the camera system employed (which appears to be less than one megapixel) and clarification as to how claimed data extraction could have occurred given turbidity and low camera resolution problems have not been successful.  Comparative data sets between the camera system used by Plymouth University and divers have never been seen nor the results published. Statements made and comparative images depicting the findings of the study, shown on the Plymouth University website research page dedicated to this study are, at best, highly misleading.  Species reported as being recorded by the remote camera system on subtidal (approximately 20-27 metres depth) reef systems in the study area include species normally associated with rockpools and intertidal waters, small species normally requiring microscopic identification never previously recorded in Lyme Bay and small species normally found underneath rocks, yet many common species were not recorded.  It appears that further years funding was awarded to the study without any of these questions being addressed or any raw data ever being seen despite regular meetings with Natural England and DEFRA and interim reports produced.

Why does this matter?

From a scientific viewpoint a unique opportunity was lost.  The significant changes within species assemblages on the reefs within Lyme Bay are very unlikely to have occurred within the first 2-3 years; they will mostly likely occur during the 5-15+ years following cessation of mobile bottom fishing.  If we cannot trust the data collected in the first few years we have no benchmark against which we can measure change.  Currently there is no raw data available to provide this benchmark and even were the raw data made available we cannot be sure what is accurate and what is not.

This was a four year study, costing the best part of half a million pounds. Despite all meetings and interim reporting, it appears that there was a lack of critical analysis.  There is a danger that similar studies may be inclined to reach conclusions preferred by the client rather than one that reflect reality.   A number of marine protected areas (MPAs) have been designated recently in UK waters, with (hopefully) more to come.  These will require monitoring and initial consultations are already taking place.  It is stating the obvious to say that we need accurate and transparent data if we really want to understand the changes that occur in these protected marine habitats.

1. Tread carefully

DEFRA Science and Research Projects. Lyme Bay – A Case-study: Measuring the effects of benthic species and assessing potential – MB0101.

A cobble reef in Lyme Bay, approximately 22 metres depth. This was one of the diver monitoring stations. This illustrates typical visibility (~4-5 metres) in the central part of the bay. Note the survey diver (holding a white monitoring quadrat) only just visible, approximately 4 metres away from the camera.

For those of us working primarily on the conservation side of marine environmental monitoring and impact assessment, it is important that the work we do, and the data we present, is as robust, evidence-based and open to scrutiny as we expect that of developers, oil and gas industries and fishery industries to be. If not, how can we hold them to account should they fail to meet such standards?  As scientists, it is important that we bear in mind at all times that we are being paid to tell clients what they need to know, and that is not necessarily the same as what they want to hear.  Yet public criticism of the work of other scientists is not an action that should be undertaken lightly, and only after other avenues have been exhausted. One needs to carefully consider both the likely consequences of such actions and one’s own motives for doing so.   For those reasons I have deliberated for over a year before publishing this article.  I suspect I am unlikely to be offered another contract by Natural England in the near future.  That is a pity, but if contracts continue to be administered in this manner it is probably mutually beneficial if I am not.  I should also point out that, before deciding to publish this article, I consulted quite widely with marine biologist colleagues including independent scientists and others working within conservation organisations.

Study of the ecology of Lyme Bay has occupied a fair amount of my professional life. I have been diving and conducting surveys in Lyme Bay since the early 1990s; I ran the first studies investigating the impacts of scallop dredging on the reefs in the Bay and have run or participated in a great many since. So no-one was more pleased than I when statutory protection for the reefs in Lyme Bay was introduced in 2008 through a 60 square nautical mile exclusion zone for mobile bottom fishing.

To determine how well and how quickly the reefs would recover following cessation of disturbance by mobile fishing gear a three year study was commissioned by DEFRA (Department of Environment, Food and Rural Affairs) with the science being overseen by Natural England (NE) Marine Monitoring specialists.  The protection had been introduced due to the concern over (and evidence of) the destruction and decline in the more fragile and slow-growing species that occurred on these reefs, in particular the erect branching sponges, the octocorals Eunicella verrucosa (pink seafans) and Alcyionium digitatum (dead man’s fingers).

Once the statutory protection had been established, it was important to monitor the changes that occurred on the reefs.  There was little doubt that change would occur once such a major disturbance ceased, but how would it happen?  The important questions were how quickly would it occur, which species would re-establish first and how long would it take for species that were typical of undisturbed reefs (the erect sponges, seafans and dead mens’s fingers) to start to recolonise?

Between 2008 and 2011 I collaborated on a DEFRA /Natural England funded study to determine the changes that occurred on the deeper water reef communities within Lyme Bay Closed Area. Specifically I ran a study looking at changes occurring on boulder reef communities lying between 20 and 22 metres depth (chart datum). This work was conducted by a small team of highly experienced marine biologists/divers. The study was a sub-contract, conducted as a discrete, but ideally complimentary, study within the main contract investigating these changes. The main contract had been awarded to the University of Plymouth who were primarily using remote (towed) video (a camera system towed above the seabed, termed the ‘flying array’) to investigate these changes. Our work ran concurrently and was, at the start of the contract, to be published as one final report.

During the course of the study my colleagues and I became increasingly concerned about the reliability of the main towed video study.  So much so that, in late 2011 during the preparation of the final report, we requested our report be published separately as a stand-alone report. We also requested that we conduct our own data analysis rather than providing Plymouth University with our data, to be analysed by them. Our feelings concerning this were so strong that we conducted all analysis and write up on a unpaid basis.  In November 2012, after much deliberation, I wrote to the scientists within Plymouth University’s Marine Biology and Ecology Research Centre who had run the university’s three year study, and requested that my name as an author be removed from the final scientific report submitted to DEFRA. This was the first time in 25 years as a marine biologist I have felt it necessary to take such actions. Our own, separate, report was completed, reviewed, and then, after considerable pressure to modify certain conclusions (which I declined to do) accepted for publication.  It was never published by Natural England or DEFRA.

There were also other discrete components of the over-arching project, in particular a socio-economic study.  My comments here apply only to the benthos study, specifically the ‘flying’ towed video study which formed by far the largest part of the project. I have no knowledge of the socio-economic component or expertise in this area and have no reason to believe it is anything other than excellent.  Nor do I intended these comments as any criticism of Plymouth University’s science in general, which again I have no reason to assume is other than first rate.  I also stress that, before writing this, I wrote several times to the scientists involved in the Flying towed video study, explaining these problems in considerably more detail than below. I asked that any misunderstandings in my interpretation be corrected. After some delay I received a very brief response explaining that I ‘clearly did not like our study‘ and that they could not help me further.

2. Key problems

From Plymouth University website:

As soon as the SI was enforced in 2008 the team undertook the first baseline survey and have monitored the bay annually since then. At first the reefs were slow to respond but in 2010 the results were impressive (see videos from 2008 and 2011 below)

Presenting all the problems of this multi-year study cannot be done concisely, nor can the key problems, without first explaining a little about the ecology of Lyme Bay, the legislation introduced and the study design. However, a snap-shot of the study is presented on Plymouth University’s Marine Biology and Ecology Research Centre’s website, where a page is dedicated to the study. This provides an indication of the sorts of concerns I had. The page is titled Marine Protected Areas: monitoring the Lyme Bay exclusion zone and can be accessed here.

Plymouth University Marine Biology and Ecology Research Centre’s Marine Protected Areas web page.

The page summarises the importance of Lyme Bay, the aims of the study, the methods used and the findings of the three year study. Here is the study description from the web page:

The team developed a non-destructive, cost-effective and time-effective technique for monitoring vast areas of the sea bed in Lyme bay. The technique involves flying a towed HD camera above the seabed to capture video footage of the reef communities that can be analysed back in the laboratory. As soon as the SI was enforced in 2008 the team undertook the first baseline survey and have monitored the bay annually since then. At first the reefs were slow to respond but in 2010 the results were impressive (see videos from 2008 and 2011 below)

(The underlining of the last sentence describing the changes recorded are mine, not from the web page.)

Immediately below this text are two video clips, one entitled ‘Video footage of Lyme Bay reef taken in 2008’ the other ‘Video footage of Lyme Bay reef taken in 2011’. The differences between the two clips are indeed impressive; the 2008 clip (10 seconds) shows a rather barren area of rocky reef with little attached marine life; the 2011 (28 seconds) shows rocky reef supporting a range of larger marine species, the most obvious of which are numbers of large pink seafans (Eunicella verrucosa), a large yellow boring sponge (Cliona celata) and a large ross coral (Pentapora fascialis). Below are two, fairly representative, frame grabs I have taken from each clip, illustrating the differences between them.

Frame grab from video clip entitled ‘Video footage of Lyme Bay reef taken in 2008’

Frame grab from video clip entitled ‘Video footage of Lyme Bay reef taken in 2011’

The differences between the 2008 and 2011 clips are indeed striking, and if this is the change that has occurred on that area of reef between 2008 and 2011 it is quite spectacular. However, what you think you are seeing is not necessarily what you really are seeing. Reading the web page, one might assume that we are looking at the same area of reef at two different points in time. We are not. We are looking at two different reef areas. Nor are we looking at two areas representative of the change that occurred in this three-year time period; the large pink seafans visible in the 2011 clip are all at least 15 years old, most probably between 15 and 30 years old. We know this because of their size and ramification (degree of branching) and from what we already know about pink seafan growth rates from earlier studies conducted in Lyme Bay and elsewhere. Pink seafans grow slowly; estimates put this between one and three centimetres per year (e.g. Munro and Munro, 2003, Sartoretto and Francour, 2012). It is also highly unlikely that the Ross coral (Pentapora fascialis) and yellow boring sponge (Cliona celata) colonies visible in the 2011 frame grab could have grown to such size in three years.

Change at anything approaching the rate implied by the two video clips simply does not happen.  This is way beyond the growth rates for any known gorgonion species. To make a terrestrial comparison, this is akin to the 2008 video showing an area of barren wasteland following a major construction programme, with the 2011 video showing ‘the same’ area populated by 10 metre tall birch trees that have sprung up in the intervening three years. Unfortunately this is simply a rather graphic example of much within presentations given and the technical reports published on the DEFRA website.

The statutory closure of such a large area of seabed for conservation purposes was a first for England. It was high profile and highly contentious. It was also likely to be a trial, a test bed to see how well such areas worked in terms of facilitating regeneration and how quickly it could occur. This was a four year study, costing the best part of half a million pounds, easily the most expensive study conducted on the ecology of the reefs in Lyme Bay and probably costing significantly more than all the other studies in the previous sixteen years combined. It was also the most intensively scrutinised study; overseen by Natural England marine monitoring specialists, where annual interim progress reports were produced and quarterly meetings involving DEFRA, Natural England, Plymouth University and myself (as Marine Bio-images consultancy) were held with presentations on study progress given and questions asked. There are two obvious questions here.  Firstly, how did this happen, and secondly, could something so obviously wrong escape notice? Is it possible that no-one was aware these findings could not possibly be correct?

Fundamental problems

There were, I believe, three fundamental problems with the study.

1. The study design the study design is such that the comparative areas outside the Closed Area are in no way comparable; they were never likely to support similar species assemblages; thus differences between the still-fished and now protected areas could not be attributed to the differences in fishing pressure. Indeed it seems clear that this is not the most significant factor in differences between the treatments.

2. The image resolution of the the towed camera system employed is too low to accurately detect recently settled colonies of the species of interest on the reef, and the sled design exacerbates this problem. If this cannot be done then change and ‘recovery’ cannot be recorded.

3. Key changes highlighted simply could not have happened; they fly in the face of everything we know about the species and taxonomic groups involved, yet there has been no attempt to address this.

All studies have problems. The important thing is that they are identified and addressed. Thus the real issues here are not what the problems were but how they were, or were not dealt with. However before that can be properly discussed a little more detail on the nature of the problems is necessary. I am not going to attempt to describe all issues with the study; rather I will select one problem, image resolution, as the issues with this are simpler to explain in non-technical terms and it is fairly representative of how they were dealt with.  I will then briefly describe some of the study design issues.

Image resolution.

The prime means of data collection was by towed high definition (HD) video camera (as described on the University’s web page, image 2, above). The analysis is summarised in the biodiversity final report (Attrill et al, 2012)

Analysis of the video transects was conducted in two stages (Sheehan et al., 2010). Firstly, species counts were made from each entire video transect for infrequent organisms (all mobile taxa) and conspicuous sessile fauna. Secondly, frame grabs were extracted from the video to quantify the encrusting, sessile species, some abundant, free-living fauna and metrics of infaunal density and bioturbation such as burrow densities.

Thus pretty much all new growth and newly settled colonies’ data were gained from frame grabs from the video footage. However, there is a problem here. Video frame grabs are not a good way to produce stills. The maximum possible resolution of the frame grab stills extracted from the video is less than one megapixel (0.92 megapixels to be precise; the video format used was 720 Progressive scan, i.e. each frame was 1280 x 720 pixels, equalling 921,600 pixels). To put this in to context, this is only a fifth of the resolution of the cheapest smartphone camera one can buy and around a 1/20th of the resolution of a good quality digital SLR. In fact this is lower resolution than any consumer digital stills camera one can buy (or has ever been made; the first commercially produced DSLR, the Kodak DCS 100, released in 1991, had a resolution of 1.3 megapixels).  The system was adapted from one used in the clear and well-lit waters of the Great Barrier Reef where, for mapping corals such systems work quite well. The much darker and more turbid waters of Lyme Bay are a quite different scenario. One must also remember the task was not to map or count the presence large colonies, the aim of the study was recording colonisation and early growth of sponges, seafans, dead men’s fingers soft corals and ross coral bryozoans. These are all long-lived, slow growing species (the first three in particular) thus in the first 2-3 years one is recording colonies that are likely to be only millimetres tall. The problem was actually worse than the camera resolution alone would suggest. The camera system was designed to ‘fly’ above the reef. This again may work well in very clear bright waters; in the relatively turbid conditions such as those that prevail in Lyme Bay; the image resolution is further degraded by the considerable distance between the subject and the camera system, far greater than we would normally consider acceptable when collecting similar data using a high resolution camera. The camera to subject distance is around twice what would normally be considered the maximum one would try to extract such data from a much higher resolution stills camera.

Correlating the actual camera resolution with that required to record settlement and growth of small colonies would appear to be impossible; I have seen no coherent explanation as to how it was achieved. There is a bigger problem however when using such a flying towed camera system in Lyme Bay. Lyme Bay waters are far from gin clear; the seabed here is largely sedimentary with significant levels of suspended particles. It is also prone to strong plankton blooms which can reduce underwater visibility to less than one metre. However, and this is the important point, the level of turbidity is not constant. It changes constantly from hour to hour as tidal streams vary in strength, day to day and week to week as gales pass through and plankton blooms come and go. These changes in turbidity dramatically affect the amount that can be seen (and recorded). This is a problem for anyone working in this environment; the only solution being to avoid really bad conditions and to get close as possible to the seabed and the species of interest, thus reducing the amount of water (and suspended particles) between the viewer and subject. Ideally this distance should be no more than 0.25- 0.3metre (as is the case for diver surveys or most towed camera sleds).

The camera to subject distance of the ‘flying’ towed video used is around a metre or more from the subject (taking into account the angular distance as the camera is not looking straight down) these changes in turbidity will create enormous differences in what can be seen on the seabed. Individual organisms that will be clearly identifiable on some occasions will become completely invisible to the camera following relatively minor changes in turbidity. These changes in what is visible will almost certainly be of far greater magnitude than any actual changes occurring in species abundances. This means that improvements and changes in abundances that are not real will appear to occur. It also means that improvements and increases in abundance that are real may not be recorded and even if they are, there will be no reliable way to separate them from apparent, non-real changes.

Positional accuracy

This is not a resolution issue, but is a further confounding factor for image interpretation. The flying towed video’s position cannot be precisely controlled and is never accurately known.  Towed behind a slow moving boat (0.5kt) pushed by wind and tide, repeat surveys of the same transect will never cover exactly the same area of seabed, often being 10-20 metres off the previous year’s track.  The distribution of rocky reefs and associated life is very patchy in Lyme Bay and varies markedly over distances of only a few metres, thus two parallel track lines 10 metres apart (e.g. the same transect recorded at two sampling intervals) will most likely record quite different numbers of target species without any real change in numbers occurring. This is not a big problem for descriptive surveys, but is a huge problem for time-series monitoring.

Examples of image resolution

The problems associated with image resolution are probably best understood by showing examples. As mentioned earlier, recently settled seafans are extremely small, on average no more than 90mm tall three years after settlement. How difficult this would make recording recently settled seafans is clearly illustrated in the images below.  The image immediately below shows a full frame grab (1280 x 720 pixels) taken from one of the Plymouth Universities video tows (this is a typical image; I extracted several for comparison) reduced slightly in size to fit here. The white rectangle shows the area of 900 x 600 pixels on the full 1280 x 720 pixel image.

A 1280 x 720 (full resolution of the camera) frame grab from Plymouth University’s towed video, reduced to 900 x506 pixels for display purposes. The white box shows the area of 900 x 600 pixels at full resolution.

The next image (below) shows a fairly low resolution digital SLR camera image (a 6 megapixel camera; modern equivalents are 12-24 megapixel). Again, for the purposes of direct comparison, the white rectangle here also shows the area covered by 900 x 600 pixels on the full resolution image.

A still from an older, 6 megapixel, Digital SLR camera, reduced to 900 x 602 pixels for display purposes. The white box shows the area of 900 x 600 pixels at full resolution. Note also the differences in contrast, colour saturation and image sharpness with the previous image.

The next image shows shows the 900 x 600 (white rectangle) area from towed video sled image, displayed at full resolution.  As can be seen this is fine for recording larger conspicuous species such as common starfish (Asterias rubens), large seafan and deadmen’s fingers colonies.

A 900 x 600 crop from the above towed video still (white box) displayed at 100%. This illustrates typical resolution obtained from such a towed video system camera.

The next image is the 900 x 600 crop (white rectangle) taken from the 6 megapixel digital SLR image, also displayed at 100% so allowing direct comparison. Note the recently settled pink seafan near the bottom right of the image. This is likely to be that year or the previous year’s recruitment; I estimate it is 10-20mm tall. It would therefore seem unlikely that many such recently settled seafans would be recorded using the towed camera system, let alone reliable counts made.

900 x 600 pixel crop from the 6 megapixel DSLR image from Lyme Bay, displayed at full resolution. Note the recently settled (1st year) pink seafan in the bottom right of the image.

What happened when people became aware of this problem.

The camera resolution was not immediately obvious.  It was not mentioned in the paper describing the system, not in any of the technical reports.  No extracted stills from the towed video were ever shown at any of the quarterly meetings, nor were any used in any of the interim or final reports.  In fact images from other studies (including our own) were used to illustrate their reports, including some images that clearly did not originate in Lyme Bay.  I estimate that around 1000 or more stills were analysed.  This leaves one asking; given so much was made of the capabilities of this system why were none of the images captured by it ever shown?

I personally queried DEFRA, Natural England and the lead author of Plymouth University’s study, pointing out the resolution of the camera.  DEFRA made no direct response; Natural England’s response was that they didn’t know whether this was true (about the camera resolution).  Given that the study was totally dependent on the resolution of the images being good enough to reliably detect new growth amongst key species then Natural England and DEFRA’s indifference to this seems more than a little surprising, particularly so given that funding for the study was to be extended by another year shortly after. It would have taken around 30 seconds to confirm the resolution on Google, or they could simply have asked Plymouth’s team. I emailed the lead author at Plymouth University, including comparative images and my interpretation of the maximum resolution of the still’s extracted from the towed video, asking how they achieved detection of newly settled colonies. This was forwarded to more junior personnel within the team, and I received a reply that neither confirmed or denied my calculation of the camera’s resolution; instead I was informed that special ‘professional’ software was used. Unfortunately this software was not named, nor what it did explained. No examples of improved or enhanced stills were provided.  I am aware of no software in existence capable of enhancing electronic images to anything like the degree that would be necessary. To the best of my knowledge there has never been any clarification of the camera’s resolution, nor have Natural England or DEFRA asked to see any stills images .

How does the data stack up?

So how does the above assessment of the camera’s resolution square with the actual data recorded?  Evaluating the collected data is not straightforward; no raw data has been provided (fundamental rule of any monitoring study, you must provide raw data; only by comparing raw data can we understand what is going on when we find anomalies, without raw data apparent change can simply be subtle changes in analysis or interpretation when different workers are involved).  Reading the final reports we find that the frame grabs from the towed video produce some inexplicable identifications; e.g. rare encrusting sponges never before confirmed in Lyme Bay before and normally only identified after microscopic examination by specialists; small spider crabs that are normally extremely well camouflaged and difficult to identify at much higher resolutions; small crab species that normally live hidden under rocks and fish and starfish species that are normally found in intertidal rockpools rather 20-30 metre deep reefs. Many small, well-hidden species were identified by frame grabs that were never recorded by our dive team (a table listing a few of the anomalous records is provide at the end of this article.)  It is well beyond the bounds of probability that these species really were recorded during the study by flying towed video

How did the video data compare with that collected by our diver monitoring study?

Direct comparison is not possible.  Our study design and survey stations were different.  However, we were commissioned to undertake a a week’s diving survey work on small sections of a subset of of the flying towed video transects.  The purpose of this was to compare the species counts by divers with that obtained by towed video in order to calibrate the video.  So what were the findings of this comparison?  That is a good question.  We provided our data, and the comparison was never seen; the data simply disappeared.  When I queried this in meetings I received no answer.

How did the key, long-lived, slow growing species data look?

This is probably the most important consideration as this was what the study was all about.  If the flying towed video was accurately recording change then what we would expect to see is that the numbers of larger sponges, dead man’s fingers and seafans would remain largely unchanged (given their slow growth and longevity) in both the newly protected area and also in the older established voluntary protected areas lying within it.  We might possibly start to detect increasing numbers of small, recently settled  colonies as colonisation occurred on previously disturbed areas.  However, if the system was simply recording spatial heterogeneity in mature colony distribution (resulting from the fact the camera transects were never in exactly the same place from year to year) and variations in the number of large colonies detected (due to the system’s limited resolution and variable underwater visibility) then we might expect to detect very few small colonies and unexplained large random variations in the numbers of large, mature colonies recorded.  So what was found?

Interpreting the data is a little tricky, as no raw numbers have been provided. However, if one looks at the processed data provided in the final reports owe then essentially we find large, random variations in the numbers of these key species.  This include apparent dramatic increases and crashes between years in species we would expect to be extremely stable (e.g. dead man’s fingers in the longer-established protected areas) and dramatic fluctuations in the numbers of large (i.e. more than 10 years old) seafans within the protected areas.  How are these crashes and fluctuations explained? Well mostly they are simply not.  Increases that support the notion of rapid and dramatic improvements are described and identified as possible signs of recovery; population crashes (sometime larger in magnitude) are generally not explained.  For example, looking at the Alcyonium digitatum (dead man’s fingers) relative abundance data the most striking change is an apparent crash in numbers within the longer, established voluntary protected areas (2009-2010) with a simultaneous large increase in numbers within the new protected area. This was followed in 2010-2011 by the opposite, a rise in numbers within the older voluntary protected areas coupled with a fall in numbers in the newly protected area.  These findings are, at best, highly unlikely (especially when one considers that the longer established voluntary areas lie scattered within the newly protected area, see diagram below) and should prompt closer inspection of the data, particularly given that older voluntary areas all lie within the newly protected area, evenly scattered across it.

Diagram showing the location of the three different ‘treatments’, 1. the New Statutory Closure, 2. the Pre-existing Voluntary Closures and 3. the still-fished Nearby Sites.

If we turn to pink sea fans we see that the most pronounced change in numbers were in the pre-existing voluntary closures where between 2009 and 2009 the relative abundance dropped by about 2/3; at the same time the relative abundance appeared to be rising within the newly protected area. This was followed by an even larger change, a four-fold increase in relative abundance between 2009 and 2010 within the older voluntary protected areas.  At this stage someone should be asking serious questions about the data.  If we look at size class data it appears that the number of large seafans (<18cm or roughly <10 years old) increased by about 1/3 between 2008-2009, then decreased by about 1/3 between 2010-2011 in the newly protected area.   It would also appear that only tiny numbers of small seafans have been recorded (again, as we would expect given the resolution of the system); Frequency graphs are produced at very small scale so difficult to read accurately, but it appears that less than 20 seafans smaller than 60mm tall was recorded in every treatment in every year; with less than 5 recorded in most years in all treatments. Given that 60mm tall seafans are likely to be around 2-3 years old, this is an extremely small data-set and, even if accurate, little could be read in to it in terms of interpreting the changes that are occurring within the Bay.

Study design

I will briefly touch on the study design. The study was designed to test the following hypothesis.

‘Over time, species assemblages within sites in the new statutory closure but outside the pre-existing voluntary closures would change to more closely resemble those in the pre-existing voluntary closures but similar change would not occur within nearby sites where fishing by towed bottom gear was still permitted.‘

This means that the rocky seabed habitats within the new protected area would start out resembling the areas just outside, then gradually change to resemble the rocky seabed areas within the longer established voluntary protected areas.  This could not happen. The  seabed habitats and environmental immediately outside of the new statutory closure were very different from that within the new statutory closure.  That is the very reason that the boundary to the closure was originally set.  The light levels, current regimes, turbidity, amount of rocky habitat, structure of rocky habitats, amount of river water flowing in and the species present were all very different.  In short the areas outside of the protected area did not support the same species assemblages as the reef area within the protected area and never would irrespective of whether they were fished or not. As an example, the ‘still-fished’ control area to the west of the protected area has much weaker tides, has the River Exe and Otter flowing in (no major rivers flow in within the protected area) the water is markedly more turbid, the seabed much more sedimentary and composed of finer material, the rocks present are mostly low-lying sandstone as opposed the harder and more rugged-relief limestone within the protected area.  There are no seabed areas within the still-fished area to the west that remotely resemble the rugged and extensive reefs within the voluntary protected areas.  As a result of these differences the still-fished area to the west support very few sponges and dead-men’s fingers and almost no seafans.  This was explained to all,  in detail, several times at meetings.  I went as far as preparing Powerpoint presentations with graphics depicting current regimes, river inflows and seabed sedimentology, with supporting photographs of reefs from different locations.  Not a single question was asked or comment made at the end of the presentation.

What did the two 2008-2010 monitoring studies find?

As expected, our study found that there were possible, very early, indicators of recovery but that only future monitoring would identify whether these were real. We also (as predicted) found that the areas outside of the protected areas were fundamentally different from protected area itself  in all years, i.e. they started out very different and remained very  different thus these differences could not be attributed to the cessation of trawling and scallop dredging within the protected area (note, this is different from saying that changes within protected were not due to cessation of trawling and dredging, some almost certainly were, it is simply pointing out the irrelevance of making comparisons with non-comparable areas) .  The flying towed video study reported more positive indications of recovery.  Their frequency graphs for most key species indicated that the ‘still-fished’ controls were very different from the protected areas, in all years, however this was not noted in the report text and comparisons were made between both treatments that would suggest these were essentially similar treatments apart from one being fished and one not.  In our report we noted that the still-fished were not similar to the protected area and therefore could not be considered comparative controls.  I was repeatedly asked to remove this from the executive summary of our report. I declined as I considered that to do otherwise would fundamentally misrepresent the findings of the study.

Is it possible that these problems were simply not noticed?

For that to be possible we would have to accept that NE and DEFRA never noticed that, for three years they had seen no data and no extracted frame grabs from an imaging-based study.  One would also have to accept that, when NE and DEFRA were shown video clips of ‘new growth’ , growth that NE specialists recognised was at least a decade old, they assumed this was not somehow not relevant.  If one accepts that these lapses in critical analysis then I will simply state that all of these problems were explained clearly, verbally and in writing, and even in Powerpoint presentations to all relevant personnel.  I have little doubt that everyone was under great pressure from above and that may, in part, explain the reluctance to question the work in more detail.

Why did it happen?

This may be the most important question.  A number of factors combined here.  Firstly there was a lack of experience of working in these sort of conditions coupled with the use of equipment designed for a different purpose and quite different conditions. Secondly, there was an almost complete focus on statistics and statistical design, to the point that basic ecology was completely ignored.  Thirdly, the most obvious questions were never asked and key data, indeed almost all data, was withheld.  There was also very pronounced pressure for the studies to produce the ‘right’ answer.  Recall that there was no interest in whether the towed video camera system was even capable of detecting colonisation.  The findings of our own study differed significantly from that of Plymouth University and it was decided that this needed to be resolved before publication.  Now one might think that looking at both sets of data might be useful here; I proposed it several times however the data was never provided.  Instead it decided that our statistical analysis would be reviewed.  At the start of the meeting I was informed that Plymouth University’s report had already been approved and so was not open to discussion.  Thus in order to resolve the differences between the two studies review and changes to one study (ours) was permitted.

 Why does any of this matter?

Apart from the obvious,  that we should always aim for the best science, why does this matter?  One could argue that the changes described are almost certainly going to happen anyway, not in the time scale of this study and maybe not as neatly as the study suggests, but in 10 to 15 years it is perfectly possible that areas of recently protected reef will look as described in this study.  So what’s the harm?

From a scientific viewpoint a unique opportunity was lost.  The significant changes did not occur in the first 2-3 years; they will mostly likely occur during the 5-15+ years following cessation of mobile bottom fishing.  We do not know how exactly how the reefs will look then; nor do we have a benchmark against which we can measure change.  There is no raw data available to provide this benchmark and even were the raw data made available we cannot be sure what is accurate and what is not.

Talking in generalities for a moment, I think most scientists and environmentalists would agree that a client disinclined to question study findings provided the findings conform to a preferred agenda, and a contractor inclined to mold findings to suit the preferred agenda, is a toxic mix that is lethal to good science and to the development of policy that actually changes conditions in the real World.  Suppose for a moment that, once the closed area had been established, the eventuality of positive change  in the benthic environment of Lyme Bay was not quite the certainty that we believe it to be.  Would that have changed the findings of this study?  Given that the findings appear to be decoupled from what was actually happening on the protected reefs then the answer is ‘probably not’.  Suppose the client was a developer rather than a conservation agency, and that their preferred findings were for their development to have minimal impact and require minimal mitigation measures. This could be contentious, highly politicised issues (as was Lyme Bay Closed Area) for example dredging a shipping channel and removing live maerl beds in the process. If ever the central aim becomes to please the client by providing the ‘right’ answers rather than accurate answers then we are on a very slippery slope indeed.

It also matters in as much as this is being pushed, all flaws airbrushed out of the  presented material, as a shining example of how to monitor Marine Protected Areas in conferences, DEFRA reports and published articles.

Specifically addressing the Lyme Bay Closed Area, this was an experiment on a grand scale, the like of which we have not seen in UK waters before. It was a phenomenal opportunity to record how species colonise disturbed areas of seabed once this disturbance ceases. Data like this simply does not exist. Lyme Bay was in many ways the ideal candidate; it was easily accessible, all within diving depths, and had been well studied for nearly two decades so we already knew a great deal about the species that existed there, the habitats and locations where they were found and which ones had declined in number. Relatively rare within reef habitats, much of it was fairly level seabed, so the establishment of fixed monitoring stations was far easier than would be the case in many other areas. Unfortunately this opportunity has been lost.

There is also the question of what else has been missed. As mentioned earlier, we were commissioned to undertake a series of dives on a subset of Plymouth University’s towed video transect stations in order to provide comparative data (never seen). This was simply a snap-shot, one off visit where we looked at short sections of a few of the flying towed video transects.  However, at some of these stations we found significant damage, tracks several metres wide swept bare of life and almost certainly due to recent mobile fishing gear operating there. These were photographed and the images presented at meetings with DEFRA, NE, Plymouth University.  Perhaps surprisingly no-one asked why this was not being picked up by the towed video system. This would seem to be a fairly important question, as if it is not picking up fairly major signs of habitat degradation then how can it be detecting more subtle signs of improvement.

It is possible that Lyme Bay Closed Area monitoring was an anomaly, possibly due to the high profile and the level of political expectation as regards the outcome. The evidence suggests otherwise. The last Natural England contract I was invited to tender for, some 18 months back, was an extremely challenging survey requiring 3 dimensional mapping, counting and monitoring of very small animals (e.g. individual coral polyps only millimetres in diameter) living within submerged sea caves along an exposed stretch of coast where underwater visibility was normally very low. Moreover, the study was to be conducted in mid-winter, when storms were most frequent, underwater visibility at its lowest and darkness falling at 4 or 5pm. After carefully reading the tender and calling NE staff to discuss this with them I emailed informing them I would not be tendering and why. My reasons were that, in order to do the job properly (by that I mean to have any chance of generating data that was remotely meaningful) was simply far too dangerous at the time of year they required the fieldwork to be conducted and I would not consider risking the lives of survey team members in this way. Secondly, the weighting given to evaluation if different aspects of each bid was helpfully provided: 50% of overall weighting was given to cost; 5% was given to the expertise and experience of the team with regard to the work to be undertaken.  To put this in as polite terms as possible, this is madness. The message this sends out is pretty clear; you can have no experience whatsoever in this field, nor any expertise within the team relevant to the fieldwork, but provided you are cheap enough the contract is yours. This was hammered home further by a maximum of 15% weighting being given to the estimation of the survey actually being successful, so a starting assumption of ‘little chance of success’ even within the very loose adopted definitions of ‘successful outcomes’ is no barrier to being a successful bidder.  This is not only a pointless squandering of money, producing meaningless reports solely to meet targets, it is also dangerous and may lead to fatalities if this approach continues.

Perhaps the most important reason it matters is that a number of marine protected areas (MPAs) have been designated recently in UK waters, with (hopefully) more to come.  These will require monitoring and initial consultations are already taking place.  The worst possible scenario is that we continue along the path of devaluing expertise and placing ever more weight on low cost.  This is actually worse than doing nothing at all, as it inevitably generates bad data, and bad data we think is accurate is worse than knowing we have no data. This could also be a wonderful opportunity to not only monitor the condition of these MPAs; systems could be established whereby comparative data collected from locations around the UK could provide information about changes on a larger scale.  I am aware that there are many dedicated biologists within Natural England who would love to see contract survey and monitoring work conducted to the highest standard and generating real, useful and pertinent data.  I am also aware that front line staff are under enormous pressure from above and that many of these decisions are no longer within their power to make.

An alternative approach

A network of permanently marked fixed monitoring stations could have been established across the newly protected site and monitored annually for a fraction of the cost of the towed video study. High quality photographs from precisely the same position can be taken year after year. These fixed stations can also be used as structures from which to ‘hang’ physical data loggers (temperature, ambient light etc.) allowing correlation of change in the physical environment with biological change.

 It is true that data from permanently fixed stations are not amenable to testing by the most powerful statistical tests, but for working in the marine environment they have one overwhelming advantage, year on year data is directly comparable. Rocky seabeds in UK waters are simply too heterogeneous, in terms of spatial distribution of species, for random sampling. Moreover this spatial variability is evident over distances of tenths of a metres or less. We simply do not have the technology yet to relocate stations to this level of accuracy within realistic budgets unless we physically mark them. We actually do have a small number of marked stations within Lyme Bay closed area (created and surveyed for our diver-based boulder reef study). These are now abandoned and the data gathering dust. These stations could, with a little effort, be relocated regularly and comparative data collected in five, ten and twenty years time. The principle could also be expanded over a much wider geographical area. With a little imagination in the appropriate bodies a network of low-cost subtidal monitoring stations could be established. These could not only be used to collect data on the condition of marine protected areas but could also be collecting data time-series data that would inform us of local and regional changes in species and habitats.

Table 1. Some of the more unlikely species identified from video frame grabs

[table]Species,Taxonomic group, Comments,Recorded by

Grantia compressa,Sponge,A small flattened purse sponge generally found attached to other species or under overhangs,Video frame grab
Sycon ciliatum,Sponge,A small purse sponge up to 50mm tall generally attached to other species ,Video frame grab
Actinothoe sphyrodeta, Anthozoan, A small anemone found on rock faces identified by dark patches at the base of tentacles, Video frame grab
Ebalia granulosa, Crustacean, A small crab (around 10mm across) found on gravel seabeds, Video frame grab
Porcellana platycheles, Crustacean, A small flattened crab (around 15mm across) generally found on rocky shores underneath boulders, Video frame grab
Asterina gibbosa, Echinoderm, A small starfish (up to 50mm across) normally found in rock pools and shallow rock – never found on any dive surveys in Lyme Bay we are aware of, Video frame grab
Ocnus planci, Echinoderm, A rare sea cucumber – small (up to 80mm) difficult to identify without detailed inspection, Video frame grab
Aplidium elegans (?), Tunicate, Assume this is meant to be Sidnyum elegans – a small (about 50mm) red colonial tunicate requiring detailed inspection to identify, Video frame grab
Diademnum coriaceum, Tunicate, An encrusting colonial sea squirt – very difficult to identify from appearance alone (not previously known in Lyme Bay), Video frame grab
Lissoclinum perforatum, Tunicate, An encrusting colonial sea squirt – tricky to identify positively (similar to other Didemnid sea squirts), Video frame grab
Molgula manhattensis, Tunicate, A small (up to 30mm) solitary sea squirt often encrusted with sand and shells- very tricky to spot and identify with certainty, Video frame grab
Lipophrys pholis, Fish, Shanny – a fish normally found in intertidal rockpools, Video frame grab

[/table]

REFERENCES

Munro C.D., Munro L. 2003. Eunicella verrucosa: investigating growth and reproduction from a population ecology perspective. PHMS Newsletter 13: 29-31.

Sartoretto S. and Francour P. 2012. Bathymetric distribution and growth rates of Eunicella verrucosa (Cnidaria: Gorgoniidae) populationsalong the Marseilles coast (France). Scientia Marina, vol. 76(2): 349-355.