Title and Abstract Screening in Systematic Reviews: A Comprehensive Guide to Best Practices

TAS serves as the first filter in the study selection process of a systematic review. Its primary purpose is to reduce the pool of potentially eligible studies to a manageable size for full-text review. TAS saves time and resources by quickly excluding irrelevant studies while focusing on high-quality research. This initial screening phase allows reviewers to eliminate a large number of articles that are clearly not suitable due to factors such as content, study type, language, or other predefined exclusion criteria. In fact, it is common for the majority of search findings to be rejected at this step. Studies have shown that, on average, only a small percentage (around 3%) of the initial search results are eventually included in the systematic review. This highlights the significant impact of effective screening on the final review and emphasizes the importance of a rigorous TAS process.

Practical Considerations for Screening

When conducting TAS, it's essential to obtain the most detailed version of the reference available from the database. This ensures that reviewers have sufficient information to make informed decisions about study relevance. Additionally, saving the references in a tagged format, where each piece of information (title, author, abstract, etc.) is clearly labeled, can facilitate efficient management and organization of the references during the screening process.

TAS is crucial for several reasons:

• Efficiency: It significantly reduces the workload by eliminating irrelevant studies early on, allowing reviewers to focus on potentially eligible studies.
• Minimizing Bias: By applying predefined inclusion and exclusion criteria, TAS helps reduce the risk of selection bias and ensures objectivity in the review process.
• Resource Optimization: TAS helps optimize resource allocation by reducing the time and cost associated with retrieving and reviewing full-text articles.
• Quality Control: It provides an opportunity to identify and address any discrepancies or uncertainties in the eligibility criteria, ensuring consistency in the screening process.

Before delving into the best practices for efficient TAS, it's helpful to understand the types of criteria used to evaluate study relevance. Screening criteria should be specific to the research question and clearly defined in the review protocol. These criteria are typically based on the PICO framework (Population, Intervention, Comparator, Outcome), which helps define the scope of the review and ensure that included studies address the research question.

Decision flowcharts, such as the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram, provide a visual representation of the study selection process. These flowcharts illustrate the number of studies identified, screened, included, and excluded at each stage of the review, ensuring transparency and facilitating accurate reporting.

Efficient TAS requires a systematic and well-defined approach. Here are some best practices to ensure a rigorous and unbiased screening process, incorporating the detailed guidelines from:

1. Develop a Comprehensive Screening Tool

Before starting the screening process, develop a clear and concise screening tool based on the review's prespecified eligibility criteria. This tool should include objective questions with a consistent sentence structure and yes/no/unsure answer options. Organize the questions hierarchically, starting with the easiest ones to facilitate quick decision-making. It's generally recommended to be overly permissive during the title and abstract screening phase, erring on the side of inclusion. This means that if there is any doubt about the relevance of a study, it should be included for full-text screening. Exclusion criteria can then be applied more stringently during the full-text review stage.

2. Pilot Test the Screening Process

Conduct a pilot test of the screening tool and eligibility criteria on a small sample of references 6. This pilot test helps refine and clarify the criteria, train reviewers, and ensure consistency in their application. It also provides an opportunity to identify any potential issues with the screening process and make necessary adjustments before proceeding with the full screening.

3. Independent Double Screening

Ideally, two reviewers should independently screen each title and abstract to minimize bias and improve the accuracy of decisions. This approach helps identify discrepancies and ensures a more thorough evaluation of study relevance. While single screening might be used in some cases, double screening by two independent reviewers is generally recommended to reduce bias and improve the accuracy of decisions. It's important to allocate sufficient time for full-text screening, considering the time required for retrieving full texts, especially through interlibrary loan.

4. Establish a Conflict Resolution Protocol

Before starting the screening process, agree on a conflict resolution protocol to manage disagreements between reviewers. This protocol may involve discussion and consensus or the involvement of a third reviewer to make the final decision.

5. Utilize Screening Software

Consider using screening software like Covidence, Rayyan or EviSynth to manage the screening process efficiently. These tools offer features such as:

• Automated deduplication: Removing duplicate records to save time and ensure accuracy.
• Independent screening: Facilitating blinded independent review by multiple reviewers.
• Conflict resolution: Managing disagreements and tracking resolutions.
• Progress tracking: Monitoring the screening progress and identifying potential bottlenecks.
• Data export: Exporting screening decisions and data for further analysis.

Covidence

Covidence is a web-based platform specifically designed for managing systematic reviews. It offers a range of features to support TAS, including:

• Studified References: Covidence automatically merges records that reference the same study, creating "studified references." This feature helps avoid duplication and ensures accurate reporting in the PRISMA flow diagram.
• Sorting Options: Covidence provides various sorting options for the list of references, including "Most relevant," "Author," "Title," and "Most recent". These options allow reviewers to prioritize studies based on different criteria and manage their workload effectively.
• Keyword Highlighting: Covidence allows reviewers to highlight keywords in titles and abstracts, which can aid in identifying relevant studies and applying eligibility criteria consistently.

Rayyan

Rayyan is another popular web-based tool for systematic reviews. It offers several features that can enhance the TAS process:

• AI-Powered Screening Assistance: Rayyan uses artificial intelligence to assist with title and abstract screening, potentially reducing screening time significantly.
• Collaborative Features: Rayyan allows multiple reviewers to collaborate on the same review, facilitating efficient teamwork and knowledge sharing.
• Filters, Bulk Actions, and PICO: Rayyan provides customizable filters to quickly find relevant studies, bulk actions to manage multiple items efficiently, and the ability to tag studies based on the PICO framework.
• Stages for Screening: Rayyan allows users to create different stages for title, abstract, and full-text screening, providing flexibility in managing the review workflow.
• Mobile App: Rayyan offers a mobile app that allows reviewers to screen studies from anywhere, increasing accessibility and convenience.

Recommendations Based on Features and User Survey Results

A study evaluating software tools for title and abstract screening in healthcare research found that Covidence and Rayyan were the most popular and user-friendly tools among researchers. These tools demonstrated good alignment with user requirements and were highly rated for their usability. Based on these findings, Covidence and Rayyan are recommended for researchers seeking efficient and easy-to-use tools to support TAS in systematic reviews.

6. Maintain Detailed Records

Document the entire screening process, including the number of studies screened, included, and excluded, as well as the reasons for exclusion. This documentation ensures transparency and facilitates reporting the review process accurately.

7. Regular Team Meetings

Schedule regular meetings with the screening team to discuss progress, address challenges, and ensure consistency in the application of eligibility criteria. These meetings also provide an opportunity for reviewers to share their experiences, learn from each other, and improve their understanding of the research topic.

8. Quality Control Measures

Implement quality control measures throughout the screening process to ensure accurate and consistent decisions. This may involve:

• Regularly reviewing screening decisions: Checking for consistency and identifying any potential biases.
• Calibrating reviewers: Ensuring reviewers are applying the eligibility criteria consistently.
• Monitoring inter-rater reliability: Assessing the level of agreement between reviewers.
• Minimizing changes to the screening tool: Frequent changes to the screening tool during the screening process can introduce inconsistencies and potentially bias the screening process.

While TAS is a crucial step in systematic reviews, it's not without its challenges. Some of the key challenges include:

• Potential for Errors and Inconsistencies: Applying eligibility criteria consistently across a large number of studies can be challenging. Even with well-defined criteria, there is potential for errors and inconsistencies in decision-making, especially when dealing with complex or ambiguous titles and abstracts.
• Need for Clear and Concise Criteria: Ambiguity in the eligibility criteria can lead to confusion and inconsistencies in screening decisions. It's crucial to ensure that the criteria are clear, concise, and easily understood by all reviewers.
• Time Commitment: Thoroughly screening titles and abstracts, especially with large datasets, can be time-consuming. This requires careful planning and allocation of resources to ensure that the screening process is completed efficiently.
• Reviewer Fatigue: Screening a large number of studies can lead to reviewer fatigue, which can affect concentration and decision-making. It's important to be aware of this and implement strategies to mitigate fatigue, such as taking breaks, dividing the workload, and using screening software to streamline the process.

Collaborative screening approaches involve multiple reviewers working together to screen studies. This can be done in different ways:

• Independent screening followed by consensus: Reviewers screen studies independently and then meet to discuss and resolve any disagreements. This approach allows for individual reviewers to apply their expertise and judgment while also benefiting from the collective wisdom of the team. For example, in a systematic review of interventions for chronic pain, two reviewers might independently screen titles and abstracts, and then meet to discuss any discrepancies in their decisions. This discussion can help clarify any uncertainties and ensure that all relevant studies are included for full-text review.
• Paired screening: Reviewers work in pairs to screen studies together, discussing each study and making decisions jointly. This approach can be particularly helpful for complex or ambiguous studies, as it allows reviewers to share their perspectives and reach a consensus in real-time. For instance, in a systematic review of educational interventions, two reviewers might work together to screen titles and abstracts, discussing each study's relevance to the research question and ensuring consistent application of the eligibility criteria.

Collaborative screening can improve the efficiency and accuracy of the screening process by:

• Reducing individual bias: Multiple perspectives help identify relevant studies that might be missed by a single reviewer.
• Increasing consistency: Discussion and consensus ensure consistent application of eligibility criteria.
• Facilitating knowledge sharing: Reviewers can learn from each other and improve their understanding of the research topic.

Collaborative reviews have also been shown to minimize the risk of bias and enhance the reliability of findings 20. By incorporating diverse perspectives and expertise, collaborative screening approaches contribute to a more objective and comprehensive assessment of the literature.

Disagreements between reviewers are common in TAS. Effective strategies for managing disagreements include:

• Open discussion: Encourage reviewers to discuss their rationale for inclusion or exclusion and try to reach a consensus. This allows reviewers to understand each other's perspectives and potentially identify any misinterpretations or inconsistencies in applying the eligibility criteria.
• Third reviewer: Involve a third reviewer to provide an independent assessment and break the tie. When selecting a third reviewer, it's important to choose someone who is knowledgeable in both the subject matter and the review protocol. This ensures that the third reviewer can make an informed decision based on a thorough understanding of the research question and the eligibility criteria.
• Pre-defined decision rules: Establish clear decision rules for resolving disagreements, such as majority rule or referral to a senior reviewer. This helps avoid prolonged discussions and ensures a consistent approach to conflict resolution.