Developing Effective Search Strategies for Systematic Reviews

Why It Matters

A well-defined search strategy is crucial for ensuring the thoroughness and reliability of systematic reviews. By crafting a precise strategy, researchers can:

• Capture all relevant citations, reducing the risk of missing critical studies.
• Enable reproducibility, allowing others to validate and replicate the review process.
• Optimize resource use by balancing sensitivity (retrieving all relevant studies) and specificity (minimizing irrelevant results).

Key Aspects of Sensitivity and Specificity

A highly sensitive search aims to include all relevant studies, even if it means finding some irrelevant ones. This approach helps ensure that researchers do not miss important studies.

On the other hand, specificity looks for the most relevant studies. This saves time during the screening process. Finding a balance between sensitivity and specificity is important for effective and trustworthy systematic reviews.

Studies vs. Reports of Studies

A single study can produce multiple reports, each presenting unique details such as methodologies, outcomes, or follow-up results. Effective searches must identify all reports linked to a study to provide a comprehensive view of its findings. Researchers should focus on collating these reports to avoid duplication and ensure accuracy.

Medical Subject Headings (MeSH)

MeSH terms are standardized keywords used to categorize articles in databases like MEDLINE. These controlled vocabularies help solve differences in terminology. They ensure that they find all relevant articles, no matter what words authors use. Leveraging MeSH terms enhances the consistency and depth of searches.

Boolean Operators and Filters

Boolean operators (AND, OR, NOT) enable researchers to combine search terms effectively, narrowing or broadening search results as needed. Filters help narrow down searches. They limit results to certain study types, languages, or publication dates. This makes the process more focused and easier to handle.

Step 1: Defining the Scope with a Clear Research Question

A well-defined research question is the foundation of a good search strategy. It offers a clear focus and helps you identify the key concepts that you need to address.

Frameworks like PICO (Population, Intervention, Comparison, Outcome) can help structure the research question and identify search concepts. Other frameworks, such as SPICE (Setting, Perspective, Intervention, Comparison, Evaluation), SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research Type), and PIPOH (Population, Intervention, Profession, Outcome, Health Care Setting), can be useful for different types of systematic reviews, particularly those with a qualitative or public health focus.

Step 2: Identify Key Concepts

Begin by breaking down your research question into its core components, such as Population, Intervention, Comparator, and Outcomes (PICO). These elements provide a structured framework for identifying relevant search terms and concepts.

Step 3: Developing Search Terms

For each key concept, brainstorm a list of keywords and synonyms. This ensures that the search captures a wide range of relevant studies. Consider variations in terminology, including:

• Plural vs. singular word forms (e.g., child/children)
• Hyphenated words (e.g., non-surgical/nonsurgical)
• Alternative spellings (e.g., estrogen/oestrogen)

It can be helpful to develop a "gold set" of relevant references to help identify search terms. This gold set might include key papers, papers by known authors in the field, or results of preliminary searches. Reviewing the references and "cited by" lists of these papers can help identify additional relevant keywords and subject headings.

Step 4: Utilizing Controlled Vocabulary

In addition to keywords, utilize controlled vocabulary terms specific to each database. Controlled vocabulary systems, like MeSH (Medical Subject Headings) in PubMed, help index articles in a standard way. This ensures consistency and improves search accuracy.

Tools like the Yale MeSH Analyzer help extract MeSH terms from relevant articles. This makes it easier to find the right headings.

Step 5: Combining Search Terms with Boolean Operators

Boolean operators (AND, OR, NOT) are essential for combining search terms effectively.

• OR: Broadens the search by including results with at least one of the keywords. (e.g., pediatric OR child*)
• AND: Narrows the search by requiring results to contain all keywords. (e.g., stress AND workplace)
• NOT: Excludes specific terms from the search results. (e.g., rabies NOT dog)

Step 6: Refining the Search Strategy

Test the search strategy in the chosen databases and refine it as needed. This may include adding or removing keywords. You might also adjust Boolean operators. Search filters can help limit results by date, language, or study type.

It's important to remember that systematic reviews aim for comprehensiveness (sensitivity). Therefore, researchers can capture some irrelevant results to ensure that they identify all relevant studies.

Step 7: Adapting the Search Strategy

When searching in different databases, it is important to change your search strategy. This helps you manage differences in how each database sets up and indexes information. This may involve changing search terms, using Boolean operators, or adjusting field codes. This helps to find the best studies in each database.

Step 8: Managing Search Results

After conducting the searches, the next step is to manage the results. This means organizing and removing duplicate search results.

We often use tools like EviSynth for this. This helps us screen and select relevant studies more efficiently. EviSynth automatically removes duplicate studies during import and records them for future reference in your report.

Step 1: Choose Relevant Databases

Select databases that align with your research topic, considering their scope, comprehensiveness, and indexing practices. Common databases include MEDLINE, Embase, Cochrane Library, and subject-specific repositories.

Step 2: Execute and Export Searches

Execute your primary search strategy in each database, exporting results to reference management software like EndNote or Zotero. EviSynth also includes a reference management feature, simplifying the organization and screening process.

Step 3: Set Up Alerts

Automate updates by setting up search alerts in platforms like MyNCBI or similar tools in other databases. Alerts notify you of newly published studies that match your search criteria, ensuring your review remains current.

Step 4: Additional Search Techniques

Add to your main search by using techniques like backward citation searching and forward citation searching. Backward citation searching means looking at the references of studies you have included. Forward citation searching involves finding studies that cite the articles you have included. Engage with experts or study authors to identify unpublished or ongoing research.

Thorough documentation of the search strategy is essential for transparency and reproducibility. This includes:

• Date of search: Record the date each database was searched.
• Database name and provider: Specify the database (e.g., PubMed) and the platform used to access it (e.g., Ovid).
• Search strategy: Provide the exact search string used, including keywords, Boolean operators, and any filters applied.
• Years searched: Indicate the time frame covered by the search.
• Number of studies identified: Note the number of records retrieved from each database.

It is a good practice to save search strategies within the databases themselves, as many databases like PubMed, Embase, and Cochrane Library offer this feature. This allows for easy retrieval and modification of the search strategy if needed. With EviSynth, you can effortlessly complete these steps as they are integrated into the platform.

When updating a systematic review search, there are two primary approaches to identifying new articles:

• Rerun the original search strategy without any changes and remove duplicates from the original set of results.
• Rerun the original search strategy with a date filter for when the article was added to the database.

This information should be reported clearly in the systematic review, often in an appendix, allowing others to replicate the search and assess its comprehensiveness.

Language Restrictions

While restricting searches to specific languages may simplify the process, it risks excluding important studies and introducing bias. Consider the implications of language limitations and strive for inclusivity where possible. I mean, can't we just use a large language model to do it for us? Just be sure you are transparent about the tools you use in your protocol and the final report.

Grey Literature

Explore sources of grey literature, such as government reports, dissertations, and conference proceedings. These often include valuable unpublished studies that may not appear in traditional databases.

Publication Bias

Be mindful of publication bias, which occurs when studies with positive results are more likely to be published. Incorporating strategies like searching trial registries and contacting study authors can help address this issue.