Comparison of image annotation data generated by multiple investigators for benthic ecology.

Durden, J. M., Bett, B. J., Schoening, Timm , Morris, K. J., Nattkemper, T. W. and Ruhl, H (2016) Comparison of image annotation data generated by multiple investigators for benthic ecology. Open Access Marine Ecology Progress Series, 552 . pp. 61-70. DOI 10.3354/meps11775.

[thumbnail of m552p061.pdf]
Preview
Text
m552p061.pdf - Published Version
Available under License Creative Commons: Attribution 4.0.

Download (408kB) | Preview
[thumbnail of m552p061_supp.pdf]
Preview
Text
m552p061_supp.pdf - Supplemental Material
Available under License Creative Commons: Attribution 4.0.

Download (204kB) | Preview

Supplementary data:

Abstract

Multiple investigators often generate data from seabed images within a single image set to reduce the time burden, particularly with the large photographic surveys now available to ecological studies. These data (annotations) are known to vary as a result of differences in investigator opinion on specimen classification and of human factors such as fatigue and cognition. These variations are rarely recorded or quantified, nor are their impacts on derived ecological metrics (density, diversity, composition). We compared the annotations of 3 investigators of 73 megafaunal morphotypes in ~28 000 images, including 650 common images. Successful annotation was defined as both detecting and correctly classifying a specimen. Estimated specimen detection success was 77%, and classification success was 95%, giving an annotation success rate of 73%. Specimen detection success varied substantially by morphotype (12-100%). Variation in the detection of common taxa resulted in significant differences in apparent faunal density and community composition among investigators. Such bias has the potential to produce spurious ecological interpretations if not appropriately controlled or accounted for. We recommend that photographic studies document the use of multiple annotators and quantify potential inter-investigator bias. Randomisation of the sampling unit (photograph or video clip) is clearly critical to the effective removal of human annotation bias in multiple annotator studies (and indeed single annotator works).

Document Type: Article
Additional Information: WOS:000379812200005
Keywords: R.V. Discovery; DIS377; Data quality; Expert knowledge; Multiple investigators; Quality assurance/quality control; Scoring; Visual imaging
Research affiliation: OceanRep > GEOMAR > FB2 Marine Biogeochemistry > FB2-MG Marine Geosystems
Refereed: Yes
Open Access Journal?: No
Publisher: Inter Research
Projects: IS2U
Date Deposited: 13 Jul 2016 07:52
Last Modified: 01 Feb 2019 15:03
URI: https://oceanrep.geomar.de/id/eprint/33351

Actions (login required)

View Item View Item