Interobserver variability in colonoscopy quality assessment: a retrospective standardized multicenter video-based study
https://orcid.org/0000-0001-5108-4740
Tudor-Stefan Rotaru
https://orcid.org/0000-0002-7022-2894
Mihaela Luca
https://orcid.org/0000-0001-8924-3161
Adrian Ciobanu
https://orcid.org/0000-0003-4473-6645
Cristian Gheorghe
https://orcid.org/0000-0001-7969-628X
Eugen Dumitru
https://orcid.org/0000-0002-7268-3723
Oana Bogdana Barboi
https://orcid.org/0000-0002-3179-9754
Diana Elena Floria
https://orcid.org/0000-0003-2727-9913
Vadim Rosca
https://orcid.org/0009-0008-5830-0918
Gheorghe Balan
https://orcid.org/0000-0002-2919-4940
Andrei Olteanu
https://orcid.org/0000-0003-3204-0878
Vasile Liviu Drug
https://orcid.org/0000-0002-7596-2656
Abstract
Background and Aims: Colonoscopy quality assessment is essential for adequate bowel preparation and complete examination, yet even validated tools such as the Boston Bowel Preparation Scale (BBPS) remain partly subjective. We assessed interobserver variability in expert evaluation using a standardized multicenter video dataset. Methods: This retrospective multicenter study included 64 anonymized complete colonoscopy videos from two academic centers. Eight experienced gastroenterologists independently evaluated recordings in randomly assigned pairs. Videos were assessed by five reviewer-pair combinations; individual reviewers evaluated between 10 and 33 examinations, and each pair assessed between 10 and 23 videos. Assessments included segmental and total BBPS scores, bowel preparation adequacy, and recognition of key anatomical landmarks: ileocecal valve, appendiceal orifice, hepatic and splenic flexures, and anal verge. Interobserver agreement was assessed using linear weighted Cohen’s kappa for segmental BBPS scores, intraclass correlation coefficient (ICC) for total BBPS score, and Cohen’s kappa with overall percent agreement for bowel preparation adequacy and anatomical landmark recognition. Because reviewer pairs varied across examinations, agreement measures were interpreted as pooled pairwise agreement across independent expert assessments. The Wilcoxon signed-rank test was retained as a complementary analysis for paired BBPS score differences. Results: Significant inter-reviewer variability was observed in BBPS scoring. Differences were found for the right colon, transverse colon, left colon, and total BBPS score: 2.42 vs 1.91, p<0.01; 2.47 vs 2.11, p<0.01; 2.44 vs 2.22, p<0.05; and 7.33 vs 6.23, p<0.01, respectively. Overall, bowel preparation adequacy classification did not differ significantly, although discordant judgments occurred in 34% of examinations. Anatomical landmark recognition also varied, particularly for the appendiceal orifice and colonic flexures. Conclusions: Expert-based assessment may show clinically relevant variability despite standardized review conditions, supporting the need for more objective and reproducible approaches to colonoscopy quality control.
##plugins.themes.bootstrap3.article.details##
colonoscopy , quality assessment, interobserver variability, anatomical landmarks, bowel preparation, Boston Bowel Preparation Scale, cecal intubation
[2] Rex DK. Key quality indicators in colonoscopy. Gastroenterol Rep (Oxf). 2023 Mar 10;11:goad009. doi: 10.1093/gastro/goad009. PMID: 36911141; PMCID: PMC10005623.
[3] Park SB, Cha JM. Quality indicators in colonoscopy: the chasm between ideal and reality. Clin Endosc. 2022 May;55(3):332-338. doi: 10.5946/ce.2022.037. PMID: 35656625; PMCID: PMC9178135.
[4] Tiankanon K, Aniwan S. What are the priority quality indicators for colonoscopy in real-world clinical practice? Dig Endosc. 2024 Jan;36(1):30-39. doi: 10.1111/den.14635. PMID: 37422906.
[5] Kastenberg D, Bertiger G, Brogadir S. Bowel preparation quality scales for colonoscopy. World J Gastroenterol. 2018 Jul 14;24(26):2833-2843. doi: 10.3748/wjg.v24.i26.2833. PMID: 30018478; PMCID: PMC6048432.
[6] Shine R, Bui A, Burgess A. Quality indicators in colonoscopy: an evolving paradigm. ANZ J Surg. 2020 Mar;90(3):215-221. doi: 10.1111/ans.15775. PMID: 32086869.
[7] Lai EJ, Calderwood AH, Doros G, Fix OK, Jacobson BC. The Boston bowel preparation scale: a valid and reliable instrument for colonoscopy-oriented research. Gastrointest Endosc. 2009 Mar;69(3 Pt 2):620-5. doi: 10.1016/j.gie.2008.05.057. PMID: 19136102; PMCID: PMC2763922.
[8] Lei II, Gaya DR, Robertson A, et al. Inter- and Intraobserver Variability in Bowel Preparation Scoring for Colon Capsule Endoscopy: Impact of AI-Assisted Assessment Feasibility Study. Cancers (Basel). 2025 Aug 29;17(17):2840. doi: 10.3390/cancers17172840. PMID: 40940936; PMCID: PMC12427401.
[9] Baker FA, Mari A, Nafrin S, et al. Predictors and colonoscopy outcomes of inadequate bowel cleansing: a 10-year experience in 28,725 patients. Ann Gastroenterol. 2019 Sep-Oct;32(5):457-462. doi: 10.20524/aog.2019.0400. PMID: 31474791; PMCID: PMC6686086.
[10] Kaminski MF, Thomas-Gibson S, Bugajski M, et al. Performance measures for lower gastrointestinal endoscopy: a European Society of Gastrointestinal Endoscopy (ESGE) Quality Improvement Initiative. Endoscopy. 2017 Apr;49(4):378-397. doi: 10.1055/s-0043-103411. PMID: 28268235.
[11] Hassan C, East J, Radaelli F, et al. Bowel preparation for colonoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Guideline - Update 2019. Endoscopy. 2019 Aug;51(8):775-794. doi: 10.1055/a-0959-0505. PMID: 31295746.
[12] Tang SJ, Wu R. Ilececum: A Comprehensive Review. Can J Gastroenterol Hepatol. 2019 Feb 3;2019:1451835. doi: 10.1155/2019/1451835. PMID: 30854348; PMCID: PMC6378086.
[13] Taghiakbari M, Hamidi Ghalehjegh S, Jehanno E, et al. Automated Detection of Anatomical Landmarks During Colonoscopy Using a Deep Learning Model. J Can Assoc Gastroenterol. 2023 May 2;6(4):145-151. doi: 10.1093/jcag/gwad017. PMID: 37538187; PMCID: PMC10395661.
[14] Moran B, Sehgal R, O'Morain N, Slattery E, Collins C. Impact of photodocumentation of caecal intubation on colonoscopy outcomes. Irish Journal of Medical Science. 2021 Nov;190(4):1397-1402. DOI: 10.1007/s11845-020-02469-z. PMID: 33471300.
[15] Zhou W, Yao L, Wu H, et al. Multi-step validation of a deep learning-based system for the quantification of bowel preparation: a prospective, observational study. Lancet Digit Health. 2021 Nov;3(11):e697-e706. doi: 10.1016/S2589-7500(21)00109-6. Erratum in: Lancet Digit Health. 2021 Nov;3(11):e696. doi: 10.1016/S2589-7500(21)00237-5. PMID: 34538736.
[16] Lee JY, Calderwood AH, Karnes W, et al. Artificial intelligence for the assessment of bowel preparation. Gastrointest Endosc. 2022 Mar;95(3):512-518.e1. doi: 10.1016/j.gie.2021.11.041. PMID: 34896100.
[17] Lee JY, Park J, Lee HJ, et al. Automatic assessment of bowel preparation by an artificial intelligence model and its clinical applicability. J Gastroenterol Hepatol. 2024 Sep;39(9):1917-1923. doi: 10.1111/jgh.16618. PMID: 38766682.
[18] Calderwood AH, Jacobson BC. Comprehensive validation of the Boston Bowel Preparation Scale. Gastrointest Endosc. 2010 Oct;72(4):686-92. doi: 10.1016/j.gie.2010.06.068. PMID: 20883845; PMCID: PMC2951305.
[19] Rostom A, Jolicoeur E. Validation of a new scale for the assessment of bowel preparation quality. Gastrointest Endosc. 2004 Apr;59(4):482-6. doi: 10.1016/s0016-5107(03)02875-x. Erratum in: Gastrointest Endosc. 2004 Aug;60(2):326. PMID: 15044882.
[20] Heron V, Parmar R, Ménard C, Martel M, Barkun AN. Validating bowel preparation scales. Endosc Int Open. 2017 Dec;5(12):E1179-E1188. doi: 10.1055/s-0043-119749. PMID: 29202001; PMCID: PMC5698009.
[21] Schelde-Olesen B, Bjørsum-Meyer T, Koulaouzidis A, et al. Interobserver agreement on landmark and flexure identification in colon capsule endoscopy. Tech Coloproctol. 2023 Dec;27(12):1219-1225. doi: 10.1007/s10151-023-02789-z. PMID: 37036637; PMCID: PMC10638147.
[22] Knudsen AB, Rutter CM, Peterse EFP, et al. Colorectal Cancer Screening: An Updated Modeling Study for the US Preventive Services Task Force. JAMA. 2021 May 18;325(19):1998-2011. doi: 10.1001/jama.2021.5746. PMID: 34003219; PMCID: PMC8409520.
[23] Jayasinghe M, Prathiraja O, Caldera D, et al. Colon Cancer Screening Methods: 2023 Update. Cureus. 2023 Apr 12;15(4):e37509. doi: 10.7759/cureus.37509. PMID: 37193451; PMCID: PMC10182334.
[24] Hsu WF, Chiu HM. Optimization of colonoscopy quality: Comprehensive review of the literature and future perspectives. Dig Endosc. 2023 Nov;35(7):822-834. doi: 10.1111/den.14627. PMID: 37381701.
[25] Ahmad A, Saunders BP. Photodocumentation in colonoscopy: the need to do better? Frontline Gastroenterol. 2021 Aug 2;13(4):337-341. doi: 10.1136/flgastro-2021-101903. PMID: 35722601; PMCID: PMC9186039.
[26] Heron V, Martel M, Bessissow T, et al. Comparison of the Boston Bowel Preparation Scale with an Auditable Application of the US Multi-Society Task Force Guidelines. J Can Assoc Gastroenterol. 2019 May;2(2):57-62. doi: 10.1093/jcag/gwy027. PMID: 31294366; PMCID: PMC6507282.
[27] Hanzel J, Sey M, Ma C, et al. Existing Bowel Preparation Quality Scales Are Reliable in the Setting of Centralized Endoscopy Reading. Dig Dis Sci. 2023 Apr;68(4):1195-1207. doi: 10.1007/s10620-022-07729-9. PMID: 36266592.
[28] Lee HJ, Keum B, Cho YS, Cha JM. Interobserver Variation of Bowel Preparation for Colonoscopy. Dig Dis Sci. 2023 Nov;68(11):4140-4147. doi: 10.1007/s10620-023-08114-w. PMID: 37740890.
[29] Massinha P, Almeida N, Cunha I, Tomé L. Clinical Practice Impact of the Boston Bowel Preparation Scale in a European Country. GE Port J Gastroenterol. 2018 Sep;25(5):230-235. doi: 10.1159/000485567. PMID: 30320161; PMCID: PMC6170922.
[30] Chen J, Wang G, Zhou J, et al. AI support for colonoscopy quality control using CNN and transformer architectures. BMC Gastroenterol. 2024 Aug 9;24(1):257. doi: 10.1186/s12876-024-03354-0. PMID: 39123140; PMCID: PMC11316311.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Archive of Clinical Cases is protected by copyright and may be used in accordance with copyright and other applicable laws. Content available at www.clinicalcases.eu and our digital applications is intended for personal noncommercial use.
Authors who submit a manuscript for publication in Archive of Clinical Cases agree to the following terms: a. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal. b. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal. c. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) only after the final version of the manuscript was accepted and published, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access). d. It is compulsory that before submission authors ensure that their work was not published in any other medical journals or pending acceptance for publication and that "Archives of Clinical Cases" is the only beneficiary at that moment if their work/case will be accepted by us.
Guidelines for linking to www.clinicalcases.eu a. The main purpose of the site linking to the Archive of Clinical Casess site should be educational. b. Links should be made to the Archive of Clinical Casess home page (www.clinicalcases.eu) or to the articles abstract. c. It is forbidden to use the Archive of Clinical Casess cover by outside organizations unless permission has been granted in advance, notifying our Secretary. d. Material owned by the Archive of Clinical Cases (including the name, logo, cover, and text) may not be used in any manner that may induce the idea or suggest that the Archive of Clinical Cases is in some way recommending a specific company, product or service. e. You must not use or allow others to access or use, all or any part of our Site or the contents and/or applications on it for commercial purposes without our permission. To seek permission to do anything prohibited by or not contained in these TERMS, or which requires our prior consent or agreement, you can contact us.