Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal

Emanuele Trucco; Alfredo Ruggeri; Thomas Karnowski; Luca Giancardo; Edward Chaum; Jean Pierre Hubschman; Bashir al-Diri; Carol Y. Cheung; Damon Wong; Michael Abramoff; Gilbert Lim; Dinesh Kumar; Philippe Burlina; Neil M. Bressler; Herbert Jelinek; Fabrice Meriaudeau; Gwenole Quellec; Tom MacGillivray; Bal Dhillon

doi:10.1167/iovs.12-10347

Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal

Emanuele Trucco, Alfredo Ruggeri, Thomas Karnowski, Luca Giancardo, Edward Chaum, Jean Pierre Hubschman, Bashir al-Diri, Carol Y. Cheung, Damon Wong, Michael Abramoff, Gilbert Lim, Dinesh Kumar, Philippe Burlina, Neil M. Bressler, Herbert Jelinek, Fabrice Meriaudeau, Gwenole Quellec, Tom MacGillivray, Bal Dhillon

Research output: Contribution to journal › Article › peer-review

143 Citations (Scopus)

Abstract

This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.

Original language	English
Pages (from-to)	3546-3559
Number of pages	14
Journal	Investigative Ophthalmology & Visual Science
Volume	54
Issue number	5
DOIs	https://doi.org/10.1167/iovs.12-10347
Publication status	Published - May 2013

Access to Document

10.1167/iovs.12-10347

Cite this

Trucco, E., Ruggeri, A., Karnowski, T., Giancardo, L., Chaum, E., Hubschman, J. P., al-Diri, B., Cheung, C. Y., Wong, D., Abramoff, M., Lim, G., Kumar, D., Burlina, P., Bressler, N. M., Jelinek, H., Meriaudeau, F., Quellec, G., MacGillivray, T., & Dhillon, B. (2013). Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal. Investigative Ophthalmology & Visual Science, 54(5), 3546-3559. https://doi.org/10.1167/iovs.12-10347

@article{0e06a36f0d1e4e57a9fe896ab87a97dd,

title = "Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal",

abstract = "This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.",

keywords = "Algorithm, Computer program, Conference paper, Consensus development, Eye fundus, Fundus image analysis, Human, Image processing, Ophthalmoscopy, Pathology, Reproducibility, Retina disease, Standard, Validation",

author = "Emanuele Trucco and Alfredo Ruggeri and Thomas Karnowski and Luca Giancardo and Edward Chaum and Hubschman, {Jean Pierre} and Bashir al-Diri and Cheung, {Carol Y.} and Damon Wong and Michael Abramoff and Gilbert Lim and Dinesh Kumar and Philippe Burlina and Bressler, {Neil M.} and Herbert Jelinek and Fabrice Meriaudeau and Gwenole Quellec and Tom MacGillivray and Bal Dhillon",

note = "Imported on 12 Apr 2017 - DigiTool details were: 086 FoR could not be migrated (80201 - ). month (773h) = May; Journal title (773t) = Investigative Ophthalmology & Visual Science. ISSNs: 1552-5783; ",

year = "2013",

month = may,

doi = "10.1167/iovs.12-10347",

language = "English",

volume = "54",

pages = "3546--3559",

journal = "Investigative Ophthalmology & Visual Science",

issn = "1552-5783",

publisher = "Association for Research in Vision and Ophthalmology Inc.",

number = "5",

}

Trucco, E, Ruggeri, A, Karnowski, T, Giancardo, L, Chaum, E, Hubschman, JP, al-Diri, B, Cheung, CY, Wong, D, Abramoff, M, Lim, G, Kumar, D, Burlina, P, Bressler, NM, Jelinek, H, Meriaudeau, F, Quellec, G, MacGillivray, T & Dhillon, B 2013, 'Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal', Investigative Ophthalmology & Visual Science, vol. 54, no. 5, pp. 3546-3559. https://doi.org/10.1167/iovs.12-10347

TY - JOUR

T1 - Validating Retinal Fundus Image Analysis Algorithms

T2 - Issues and a Proposal

AU - Trucco, Emanuele

AU - Ruggeri, Alfredo

AU - Karnowski, Thomas

AU - Giancardo, Luca

AU - Chaum, Edward

AU - Hubschman, Jean Pierre

AU - al-Diri, Bashir

AU - Cheung, Carol Y.

AU - Wong, Damon

AU - Abramoff, Michael

AU - Lim, Gilbert

AU - Kumar, Dinesh

AU - Burlina, Philippe

AU - Bressler, Neil M.

AU - Jelinek, Herbert

AU - Meriaudeau, Fabrice

AU - Quellec, Gwenole

AU - MacGillivray, Tom

AU - Dhillon, Bal

N1 - Imported on 12 Apr 2017 - DigiTool details were: 086 FoR could not be migrated (80201 - ). month (773h) = May; Journal title (773t) = Investigative Ophthalmology & Visual Science. ISSNs: 1552-5783;

PY - 2013/5

Y1 - 2013/5

N2 - This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.

AB - This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.

KW - Algorithm

KW - Computer program

KW - Conference paper

KW - Consensus development

KW - Eye fundus

KW - Fundus image analysis

KW - Human

KW - Image processing

KW - Ophthalmoscopy

KW - Pathology

KW - Reproducibility

KW - Retina disease

KW - Standard

KW - Validation

U2 - 10.1167/iovs.12-10347

DO - 10.1167/iovs.12-10347

M3 - Article

C2 - 23794433

SN - 1552-5783

VL - 54

SP - 3546

EP - 3559

JO - Investigative Ophthalmology & Visual Science

JF - Investigative Ophthalmology & Visual Science

IS - 5

ER -

Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal

Abstract

Access to Document

Fingerprint

Cite this