{ "id": "epab8yder6", "slug": "abstract-screening", "title": "Abstract screening", "updated": "2026-10-15", "topic_path": [ { "id": "physical", "label": "Physical world and mathematics", "api_url": "https://www.edgepedia.org/api/v1/topics/physical" }, { "id": "physical.science", "label": "General science and scientific practice", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.science" }, { "id": "physical.science.research_methods_design", "label": "Research methods and experimental design", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.science.research_methods_design" }, { "id": "physical.science.research_methods_design.systematic-reviews-and-evidence-synthesis", "label": "Systematic reviews and evidence synthesis", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.science.research_methods_design.systematic-reviews-and-evidence-synthesis" } ], "geo": [ { "id": "geo.nongeo.t1946.physical.science.research_methods_design", "label": "Non-geographic · 1946 to 2000: Research methods and experimental design", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.science.research_methods_design", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t1946", "label": "Non-geographic · 1946 to 2000", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946" }, { "id": "geo.nongeo.t1946.physical", "label": "Physical world and mathematics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical" }, { "id": "geo.nongeo.t1946.physical.science", "label": "General science and scientific practice", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.science" }, { "id": "geo.nongeo.t1946.physical.science.research_methods_design", "label": "Research methods and experimental design", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.science.research_methods_design" } ] } ], "excerpt": "Abstract screening is the systematic review step that assesses titles and abstracts against eligibility criteria, reducing thousands of search records to those read in full.", "snippet": "Abstract screening is the systematic review step that assesses titles and abstracts against eligibility criteria, reducing thousands of search records to those read in full.", "node": "physical.science.research_methods_design.systematic-reviews-and-evidence-synthesis", "markdown": "# Abstract screening\n\nAbstract screening is the step of a systematic review in which the titles and abstracts of records retrieved by literature searches are assessed against the review's eligibility criteria to decide which records proceed to full-text review. It sits between deduplication of search results and full-text screening, and it is the stage that reduces a search output of 1,000 to 70,000 references down to the set read in full.[1](https://www.ovid.com/journals/cesm/fulltext/10.1002/cesm.70098~artificial-intelligence-resources-for-the-screening-of) Screening is typically a multi-stage process in which potentially eligible studies are first identified from titles and abstracts, then assessed through full-text review and, where necessary, contact with study investigators.[2](https://pmc.ncbi.nlm.nih.gov/articles/PMC8005925/) The PRISMA 2020 flow diagram records the number of records screened and the number excluded at this stage.[3](https://doi.org/10.1136/bmj.n71)\n\n| Key fact | Value |\n|---|---|\n| Decision rule per record | Yes, no, or unsure; \"unsure\" records remain eligible and move to full-text review[4](https://pmc.ncbi.nlm.nih.gov/articles/PMC6771536/) |\n| Single- vs dual-reviewer sensitivity | 86.6% (95% CI 80.6–91.2%) vs 97.5% (95% CI 95.1–98.8%)[5](https://repositorium.meduniwien.ac.at/obvumwoa/content/titleinfo/6976676/full.pdf) |\n| Human error rate at this stage | 10.76% of screening decisions (95% CI 7.43–14.09%), about 1 error in 9 abstracts[6](https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0227742) |\n| Time per reference per reviewer | 0.9 minutes for abstract screening, versus 7 minutes for full-text screening[6](https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0227742) |\n| Scale example | 14,923 abstracts double-screened by a team in 89 days[4](https://pmc.ncbi.nlm.nih.gov/articles/PMC6771536/) |\n| Kappa interpretation bands | 0.61–0.80 substantial; 0.81–1.00 almost perfect agreement (Landis & Koch)[7](https://casrai.org/guides/title-and-abstract-screening) |\n| Governing reporting standard | PRISMA 2020 flow diagram, with an itemized exclusion-reason breakdown only at the full-text phase[3](https://doi.org/10.1136/bmj.n71) |\n\n## How it works\n\nThe purpose of the stage is to apply a deliberately conservative filter. Because a record excluded at title or abstract level never reaches full-text review, screeners are instructed to err toward inclusion: a record voted \"maybe\" or \"unsure\" is treated as included and moves to full-text review.[8](https://guides.lib.vt.edu/SRMA/screen) Overusing the unsure category is itself a cost, since it shifts burden to dispute resolution and full-text screening.[4](https://pmc.ncbi.nlm.nih.gov/articles/PMC6771536/)\n\nScreening at title-and-abstract level rather than title alone is an empirical trade-off. In a pilot of 2,965 MEDLINE and Embase citations, a titles-first strategy immediately rejected 86% of records while simultaneous title-and-abstract screening rejected 94%; both strategies identified the same 13 final articles, with recall (the proportion of truly eligible studies successfully found) of 100% for each.[9](https://pmc.ncbi.nlm.nih.gov/articles/PMC3604876/) The Institute of Medicine recommends the simultaneous approach.[9](https://pmc.ncbi.nlm.nih.gov/articles/PMC3604876/)\n\n## How it is done\n\nA typical workflow runs as follows. First, eligibility criteria are translated into concrete screening questions with a yes/no/unsure answer format.[4](https://pmc.ncbi.nlm.nih.gov/articles/PMC6771536/) Second, the team pilots the criteria on a small calibration set.[4](https://pmc.ncbi.nlm.nih.gov/articles/PMC6771536/) Low pilot agreement signals problems with the protocol, the screening form, or reviewer understanding.[8](https://guides.lib.vt.edu/SRMA/screen)\n\nThird, two people screen each record independently and blinded to each other's votes.[10](https://www.cochrane.org/ro/authors/handbooks-and-manuals/mecir-manual/standards-conduct-new-cochrane-intervention-reviews-c1-c75/performing-review-c24-c75/selecting-studies-include-review-c39-c42) Duplicating the process reduces both the risk of mistakes and the possibility that selection is influenced by a single person's biases.[10](https://www.cochrane.org/ro/authors/handbooks-and-manuals/mecir-manual/standards-conduct-new-cochrane-intervention-reviews-c1-c75/performing-review-c24-c75/selecting-studies-include-review-c39-c42) Interrater reliability is quantified with [Cohen's kappa](https://www.edgepedia.org/cohens-kappa), calculated as \\( \\kappa = (p_{o} - p_{e}) / (1 - p_{e}) \\), where \\( p_{o} \\) is observed agreement and \\( p_{e} \\) the probability of chance agreement; reliability should be reported separately for title/abstract and full-text stages.[8](https://guides.lib.vt.edu/SRMA/screen) Fourth, disagreements are resolved by discussion or a third, usually more experienced, reviewer; the Cochrane Handbook prescribes discussion among authors as the first step.[8](https://guides.lib.vt.edu/SRMA/screen) • [7](https://casrai.org/guides/title-and-abstract-screening) Exclusion reasons need not be recorded per record at this stage, but the total excluded must be reported.[11](https://library-guides.imperial.ac.uk/systematic-review/title_ab_screening)\n\n## Origin\n\nThe step appears within general review frameworks, placed after searching and before gathering information from studies. Its codification came through reporting guidelines. The QUOROM Statement, developed at a 1996 conference and published in [The Lancet](https://www.edgepedia.org/the-lancet) in 1999, addressed reporting of meta-analyses of randomized controlled trials, and the PRISMA Statement is a 27-item checklist with a four-phase flow diagram that records records identified, screened, and included.[12](https://journals.plos.org/plosmedicine/article?id=10.1371%2Fjournal.pmed.1000097) PRISMA for Abstracts, published in 2013 in PLoS Medicine by Elaine M. Beller and colleagues, extended reporting guidance to journal and conference abstracts of reviews.[13](https://doi.org/10.1371/journal.pmed.1001419) The empirical case for double screening rests on work such as the 2002 study by Phil Edwards and colleagues in [Statistics in Medicine](https://www.edgepedia.org/statistics-in-medicine) on the accuracy and reliability of screening records for identifying randomized controlled trials.[14](https://doi.org/10.1002/sim.1190) The current standard is the PRISMA 2020 statement by Matthew J. Page and colleagues, published in the BMJ in 2021.[3](https://doi.org/10.1136/bmj.n71)\n\n## Variants\n\n**Dual screening** uses two independent screeners who vote on every record, with conflicts resolved as above.[10](https://www.cochrane.org/ro/authors/handbooks-and-manuals/mecir-manual/standards-conduct-new-cochrane-intervention-reviews-c1-c75/performing-review-c24-c75/selecting-studies-include-review-c39-c42) **Single screening** is accepted by some bodies and not others: the University of York CRD, the US National Academy of Medicine, and the German IQWiG advocate dual-reviewer screening, while AHRQ and NICE view single screening as an acceptable alternative; Campbell and Cochrane guidance instead requires at least two people working independently to determine whether each study meets the eligibility criteria, with single screening at most a conditional shortcut at the title-and-abstract stage.[5](https://repositorium.meduniwien.ac.at/obvumwoa/content/titleinfo/6976676/full.pdf) A middle option is screening with verification, in which one reviewer screens and a second checks a random sample of screened records for consistency; automated tools can also serve as a second reviewer.[15](https://library-guides.ucl.ac.uk/systematic-reviews/screening) NHMRC guidance recommends this check when single screening is preferred.[16](https://www.nhmrc.gov.au/guidelinesforguidelines/develop/selecting-studies-and-data-extraction)\n\n**Liberal-accelerated screening** uses a single first-pass reviewer with a second reviewer checking only exclusions; it cuts reviewer-hours roughly in half but carries a real risk that the first-pass reviewer misses an eligible record the second reviewer never sees.[7](https://casrai.org/guides/title-and-abstract-screening) **Title-only screening** was implemented for 33 systematic reviews supporting the 2020 Dietary Guidelines Advisory Committee, trading sensitivity for speed.[17](https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-023-02374-3) **Machine-learning priority screening** ranks records by model-estimated probability of eligibility; PRISMA 2020's guidance names single screening, double screening, priority screening with machine learning, and classifiers calibrated to a given level of recall, and requires reporting whether records were excluded solely on machine assessment.[2](https://pmc.ncbi.nlm.nih.gov/articles/PMC8005925/)\n\n## Applications\n\nDedicated software structures the workflow. Rayyan, a web and mobile app for systematic reviews by Mourad Ouzzani, Hossam Hammady, Zbys Fedorowicz, and Ahmed Elmagarmid, was published in 2016.[18](https://doi.org/10.1186/s13643-016-0384-4) Abstrackr and Rayyan are free to use with logins, while commercial tools include [Covidence](https://www.edgepedia.org/covidence), EPPI-Reviewer, and DistillerSR, offering priority screening, machine-learning classifiers, disagreement display, and PRISMA flow diagrams.[16](https://www.nhmrc.gov.au/guidelinesforguidelines/develop/selecting-studies-and-data-extraction) Text-mining tools such as Abstrackr perform active-learning prioritization: they estimate the probability that each remaining abstract is eligible based on similarity to previously screened abstracts and sort the list accordingly.[4](https://pmc.ncbi.nlm.nih.gov/articles/PMC6771536/)\n\nSince late 2023, large language models have entered the stage. A 2024 evaluation in Annals of Internal Medicine by Viet-Thi Tran and colleagues tested GPT-3.5 turbo models for title and abstract screening.[19](https://doi.org/10.7326/m23-3389) Integration can be in series (autonomous pre-screening) or in parallel (the model in place of a second reviewer, halving initial screening workload); combining human and LLM decisions in parallel is recommended as the most reasonable integration because it can rescue relevant studies overlooked by human reviewers.[20](https://ora.ox.ac.uk/objects/uuid:98812ec8-931d-4530-9f89-fc2bfb4a15db/files/r9593tw967) • [21](https://www.cambridge.org/core/journals/research-synthesis-methods/article/compact-large-language-models-for-title-and-abstract-screening-in-systematic-reviews-an-assessment-of-feasibility-accuracy-and-workload-reduction/CB00FD70434780029EF6C027055331BA) A JBI-aligned scoping review of 174 studies published 2019 to 2026 found the most pressing recommendation (26 of the included studies) was persistence of the human component.[1](https://www.ovid.com/journals/cesm/fulltext/10.1002/cesm.70098~artificial-intelligence-resources-for-the-screening-of)\n\n## Limitations and alternatives\n\nHuman screeners make errors at a measurable rate. Across 139,467 citations and 329,332 decisions by 86 reviewers, the total error rate (false inclusion plus false exclusion) at abstract screening was 10.76%, and because dual erroneous exclusion removes a record entirely, the measured rate may understate missed studies.[6](https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0227742) Single-reviewer screening missed 13.4% of relevant studies in a crowd-based randomized trial, and missed more on the public health topic than the pharmacological topic (16.8% vs 10.5%).[5](https://repositorium.meduniwien.ac.at/obvumwoa/content/titleinfo/6976676/full.pdf) A second reviewer at the title/abstract stage identified an additional 6.6% to 9.1% of eligible studies in a complete dual review approach.[22](https://pmc.ncbi.nlm.nih.gov/articles/PMC6989049/) Some studies are falsely excluded because their titles and abstracts are uninformative; in one scoping review of 134 publications, 11 (8%) were excluded for this reason, and reference list checking recovered all 11, while contacting key authors, forward citation tracking, and hand searches recovered none.[23](https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-022-02109-w) The human \"gold standard\" is itself imperfect, which is why some authors argue automation tools achieving error rates similar to humans may be adequate.[6](https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0227742)\n\nOpen questions remain. No single kappa threshold is formally established as adequate; sources give the Landis & Koch bands, typical human inter-rater kappa of 0.82 to 0.90, and cautions against relying on kappa alone.[7](https://casrai.org/guides/title-and-abstract-screening) • [24](https://link.springer.com/article/10.1186/s13643-026-03111-2) For LLM screening, proposed governance includes depositing the full prompt history, the model version hash, and audit logs of reviewer–LLM disagreements, with human inspection of random samples sized to detect a 10% error rate with 95% confidence.[25](https://www.mdpi.com/2078-2489/16/5/378)\n\n## References\n\n1. [Artificial Intelligence Resources for the Screening of Titles and Abstracts in Systematic Literature Reviews: A JBI-aligned scoping review](https://www.ovid.com/journals/cesm/fulltext/10.1002/cesm.70098~artificial-intelligence-resources-for-the-screening-of)\n2. [PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews](https://pmc.ncbi.nlm.nih.gov/articles/PMC8005925/)\n3. [Matthew J Page and colleagues (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ.](https://doi.org/10.1136/bmj.n71)\n4. [Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses](https://pmc.ncbi.nlm.nih.gov/articles/PMC6771536/)\n5. [Single-reviewer abstract screening missed 13 percent of relevant studies: a crowd-based, randomized controlled trial](https://repositorium.meduniwien.ac.at/obvumwoa/content/titleinfo/6976676/full.pdf)\n6. [Error rates of human reviewers during abstract screening in systematic reviews](https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0227742)\n7. [Title and Abstract Screening: Workflow, Pilot Calibration and Disagreement Rules (CASRAI)](https://casrai.org/guides/title-and-abstract-screening)\n8. [Eligibility Screening - Virginia Tech Research Guide](https://guides.lib.vt.edu/SRMA/screen)\n9. [Titles versus titles and abstracts for initial screening of articles for systematic reviews](https://pmc.ncbi.nlm.nih.gov/articles/PMC3604876/)\n10. [Selecting studies to include in the review (C39-C42) | Cochrane MECIR Manual](https://www.cochrane.org/ro/authors/handbooks-and-manuals/mecir-manual/standards-conduct-new-cochrane-intervention-reviews-c1-c75/performing-review-c24-c75/selecting-studies-include-review-c39-c42)\n11. [Title and abstract screening - Imperial College London Library Guide](https://library-guides.imperial.ac.uk/systematic-review/title_ab_screening)\n12. [Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement](https://journals.plos.org/plosmedicine/article?id=10.1371%2Fjournal.pmed.1000097)\n13. [Elaine M. Beller and colleagues (2013). PRISMA for Abstracts: Reporting Systematic Reviews in Journal and Conference Abstracts. PLoS Medicine.](https://doi.org/10.1371/journal.pmed.1001419)\n14. [Phil Edwards and colleagues (2002). Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Statistics in Medicine.](https://doi.org/10.1002/sim.1190)\n15. [Screening studies - UCL Library Guide](https://library-guides.ucl.ac.uk/systematic-reviews/screening)\n16. [Selecting studies and data extraction - NHMRC](https://www.nhmrc.gov.au/guidelinesforguidelines/develop/selecting-studies-and-data-extraction)\n17. [Title-plus-abstract versus title-only first-level screening approach: a case study using a systematic review of dietary patterns and sarcopenia risk](https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-023-02374-3)\n18. [Mourad Ouzzani and colleagues (2016). Rayyan, a web and mobile app for systematic reviews. Systematic Reviews.](https://doi.org/10.1186/s13643-016-0384-4)\n19. [Viet-Thi Tran and colleagues (2024). Sensitivity and Specificity of Using GPT-3.5 Turbo Models for Title and Abstract Screening in Systematic Reviews and Meta-analyses. Annals of Internal Medicine.](https://doi.org/10.7326/m23-3389)\n20. [High-performance automated abstract screening with large language model ensembles](https://ora.ox.ac.uk/objects/uuid:98812ec8-931d-4530-9f89-fc2bfb4a15db/files/r9593tw967)\n21. [Compact large language models for title and abstract screening in systematic reviews: An assessment of feasibility, accuracy, and workload reduction](https://www.cambridge.org/core/journals/research-synthesis-methods/article/compact-large-language-models-for-title-and-abstract-screening-in-systematic-reviews-an-assessment-of-feasibility-accuracy-and-workload-reduction/CB00FD70434780029EF6C027055331BA)\n22. [The value of a second reviewer for study selection in systematic reviews](https://pmc.ncbi.nlm.nih.gov/articles/PMC6989049/)\n23. [Characteristics and recovery methods of studies falsely excluded during literature screening, a systematic review](https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-022-02109-w)\n24. [Machine learning-assisted abstract screening on learning analytics: a step-by-step tutorial](https://link.springer.com/article/10.1186/s13643-026-03111-2)\n25. [Large Language Models in Systematic Review Screening: Opportunities, Challenges, and Methodological Considerations](https://www.mdpi.com/2078-2489/16/5/378)\n\n---\n*Topic: Encyclopedia › Physical world and mathematics › General science and scientific practice › Research methods and experimental design › Systematic reviews and evidence synthesis*\n\n*Initially written Sep 29, 2026 · Edited: Oct 15, 2026*\n\n*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*\n\nLicense: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license\n", "same_as": [], "url": "https://www.edgepedia.org/abstract-screening", "markdown_url": "https://www.edgepedia.org/abstract-screening.md", "license": { "name": "Edgepedia Community License 1.0", "url": "https://www.edgepedia.org/edgepedia/license", "summary": "Free with credit, commercial use included. AI training is open to everyone. For other uses, organizations over USD 100M in revenue or 100M monthly users license separately.", "spdx": "LicenseRef-Edgepedia-Community-1.0" }, "credit": "\"Abstract screening\", Edgepedia (EdgeChat), https://www.edgepedia.org/abstract-screening. Edgepedia Community License 1.0.", "credit_md": "\"[Abstract screening](https://www.edgepedia.org/abstract-screening)\", Edgepedia (EdgeChat), [https://www.edgepedia.org/abstract-screening](https://www.edgepedia.org/abstract-screening). [Edgepedia Community License 1.0](https://www.edgepedia.org/edgepedia/license).", "credit_html": "\"Abstract screening\", Edgepedia (EdgeChat), https://www.edgepedia.org/abstract-screening. Edgepedia Community License 1.0.", "speakable": "Abstract screening is the systematic review step that assesses titles and abstracts against eligibility criteria, reducing thousands of search records to those read in full." }