Weightage Identified Network of Keywords Technique: A Structured Approach in Identifying Keywords for Systematic Reviews

Article information

Healthc Inform Res. 2025;31(1):48-56

Publication date (electronic) : 2025 January 31

doi : https://doi.org/10.4258/hir.2025.31.1.48

Sasidharan Sivakumar¹, Gowardhan Sivakumar²

¹Indian Council of Medical Research, New Delhi, India

²Department of Maxillofacial Prosthodontics, Ragas Dental College and Hospital, Chennai, India

Corresponding Author: Sasidharan Sivakumar, Indian Council of Medical Research (ICMR), Ansari Nagar, New Delhi 110029, India. Tel: +91-8778309378, E-mail: sasidharan.icmr@gmail.com (https://orcid.org/0000-0003-1561-1755)

Received 2024 August 15; Revised 2024 December 17; Accepted 2024 December 17.

Abstract

I. Introduction

The impact of systematic reviews in biomedical research is profound and far-reaching, revolutionizing the landscape of evidence-based medicine. Systematic reviews employ a rigorous and methodical approach to synthesize the extensive body of biomedical literature, providing critical insights into the efficacy, safety, and effectiveness of healthcare interventions. Renowned for their methodological rigor, systematic reviews are widely regarded as a reliable source of evidence, playing a pivotal role in shaping clinical practice guidelines, informing healthcare policies, and guiding research priorities. By consolidating high-quality evidence, they aim to improve patient outcomes and advance scientific understanding in biomedicine [1]. This process begins with the meticulous identification of relevant articles using carefully selected, topic-specific keywords. The importance of precise keyword selection cannot be overstated, as it ensures the retrieval of highly relevant studies while minimizing the risk of overlooking critical evidence. Keywords are crucial in tasks such as text mining, information retrieval, identifying relevant documents, and natural language processing. Identifying keywords is a crucial step that forms the foundation for retrieving relevant articles from electronic databases, thereby laying the groundwork for the entire systematic review process [2,3]. The primary objective of a systematic review is to comprehensively identify all pertinent resources or, ideally, as many as feasible. Consequently, the reviewer must carefully select a search system that offers optimal coverage of the chosen research topic [4].

To increase the sensitivity and comprehensiveness of literature searches, it is essential to use precise and systematically selected keywords. PubMed, a widely utilized search engine, provides robust indexing through its standardized vocabulary, which facilitates efficient retrieval of relevant information. This vocabulary is known as Medical Subject Headings (MeSH) terms [5,6]. Traditionally, MeSH indexing has depended on manual annotation to ensure accuracy and consistency. However, recent advancements in machine learning are transforming this process. For example, MeSHProbeNet, an innovative end-to-end deep learning framework, has shown significant potential in automating MeSH indexing with high precision, thereby enhancing the scalability and efficiency of literature curation [7–9].

By employing precise and comprehensive keywords, researchers can increase the sensitivity of their literature searches, ensuring the inclusion of a sufficient number of relevant articles aligned with their research objectives [10]. Indeed, subject experts play a crucial role in selecting keywords for systematic reviews based on their domain knowledge, as their insights can significantly enrich the search process. However, relying solely on subject experts for keyword selection may introduce selection bias, potentially limiting the comprehensiveness of the review [11]. Therefore, integrating subject expert insights with systematic methodologies for MeSH term selection in the initial search strategy can enhance the efficiency and effectiveness of the review process, ensuring a comprehensive search while minimizing bias [12]. For instance, a study by Sampson et al. [13] evaluated the impact of different search strategies on the retrieval of studies for systematic reviews and found that systematic approaches to keyword selection, such as using controlled vocabulary terms and Boolean operators, significantly improved the sensitivity and specificity of literature searches compared to unstructured search strategies. This highlights the value of methodological rigor in keyword selection for maximizing the retrieval of relevant articles in systematic reviews.

The absence of standardized guidelines for describing and reporting information retrieval methods in systematic reviews poses a significant challenge in evidence synthesis. Researchers currently employ a variety of practices, with no universally accepted framework to ensure consistency or transparency in the search process. This variability not only increases the risk of bias but also undermines the reproducibility of systematic reviews [14]. Furthermore, studies have shown that using poorly selected or inadequate search keywords can impede the retrieval of relevant literature, thus compromising the comprehensiveness and validity of the review [15]. In contrast, a systematic and comprehensive approach to keyword selection results in more exhaustive and representative samples of pertinent literature [16]. To address this issue, our study developed a structured framework for keyword identification called the weightage identified network of keywords (WINK) technique, which is designed to enhance the relevance of articles in systematic reviews. This method utilizes VOSviewer [17], an open-access tool for scientific data visualization and trend analysis, which proves particularly valuable for extracting and organizing keywords from large datasets. Unlike general keyword extraction techniques, the WINK approach assigns weights to MeSH terms, providing a scientifically robust and efficient method for searching medical literature via PubMed. By employing these standardized terms, the WINK technique improves search precision and relevance, thereby enhancing the quality and comprehensiveness of systematic reviews.

II. Methods

In biomedical research, conducting a comprehensive literature search typically involves using various databases such as MEDLINE, Scopus, and Embase. With the exponential increase in the volume of text documents available online and in digital libraries, it has become essential to have an effective method for keyword extraction to facilitate article retrieval [7].

Our initial search strategy employed the open-access MEDLINE database through the PubMed search engine. We incorporated MeSH terms, which were identified with the help of subject experts. These terms were further refined using the “MeSH on Demand” tool available on PubMed [18], enhancing the precision of our keyword selection process. Although this strategy adheres to established practices for keyword selection, we adopted a structured approach named “WINK.” This approach was specifically developed to maximize both the rigor and the breadth of the literature base for our systematic review. The examples below demonstrate the application and effectiveness of this methodology.

Step-by-step approach of the WINK technique:

In these scenarios, the research questions were broadly framed, which complicated the retrieval of articles from databases when relying solely on expert insights. The initial search was conducted using keywords suggested by subject experts. These included either MeSH terms or terms that needed to appear in the title or abstract. Additionally, the “MeSH on Demand” platform was employed to identify MeSH terms pertinent to our research objective, as part of our standard search strategy. This method yielded 74 and 197 eligible articles for Q1 and Q2, respectively, by restricting the study type to “systematic reviews” and limiting the publication years from 2000 to 2024 (Table 1).

Table 1

Search strategy using the conventional and weightage identified network of keywords (WINK) techniques

Constructing a weighted network chart of identified MeSH terms: Initially, a comprehensive literature search was conducted using the conventional strategy with the PubMed engine. The articles retrieved were saved in PubMed format. Subsequently, MeSH terms from these articles were analyzed for their co-occurrence patterns using the VOSviewer platform [17], an open-source tool commonly employed in bibliometric research.

The process began by generating co-occurrence lists of MeSH terms from the downloaded PubMed format file and analyzing their relationships. Initiated through clicking the “Create” button on the platform (refer to Figure 1 for a step-by-step visualization in VOSviewer). Within the platform, researchers can set a minimum co-occurrence threshold to filter terms, enabling the selection of those most relevant to the research question (Figure 1F). Terms that met the threshold criteria were then examined in greater detail. Non-specific terms such as “study design,” “males,” “females,” and “adults,” though frequently occurring, were not central to the focus of our analysis and were excluded to refine the chart to domain-specific MeSH terms.

Figure 1

Step-by-step approach to the WINK strategy for Q1 in the VOSviewer platform: (A) create a map based on bibliographic data, (B) read data from bibliographic database files, (C) select the tab “PubMed” to upload the PubMed format files downloaded through conventional search, (D) select co-occurrence and fractional counting, (E) set the minimum number of keyword occurrences (fewer occurrences yield more keywords and vice versa), (F) determine the number of keywords, and (G) keyword selection by subject experts based on the weightage and relevancy to the study objectives.

Subsequently, a co-occurrence network chart was generated. In this chart, the software assigned weights to the selected MeSH terms based on their frequency of co-occurrences across the uploaded literature evidence. This method enabled us to quantify the strength of the associations between terms, offering valuable insights into thematic link-ages within the research domain. In the resulting chart, the size of each node corresponds to the occurrence frequency of each MeSH term, while the thickness of the edges reflects the strength of association between terms, as determined by their co-occurrence counts across the dataset (Figures 2, 3).

Figure 2

Comprehensive weighted network visualization of Medical Subject Headings (MeSH) terms for Q1.

Figure 3

Comprehensive weighted network visualization of Medical Subject Headings (MeSH) terms for Q2.

This refined list of MeSH terms (Table 1) and the corresponding weighted network chart provide a detailed view of term co-occurrences and their relative importance in the field. The complete network chart, together with the MeSH term list, can be accessed at:

In our analysis of the MeSH terms and generated networks, we divided the research question into two or three sections based on the contextual subheadings of the MeSH terms. For instance, in response to the question, “How do environmental pollutants affect endocrine function?” two distinct contexts emerged: (1) terms related to environmental pollutants and (2) terms related to endocrine abnormalities. The network analysis indicated that patterns of co-occurrence within a single context, such as connections exclusively among environmental pollutant terms without links to endocrine abnormalities, would not yield studies relevant to our research objectives. Although these intra-context relationships may occur, they fail to provide the association-type studies that are pertinent to the research question. The software used for the analysis does not align terms contextually with the research objectives but rather identifies repetitive co-occurrences within the articles.

The final search string, developed using the WINK technique, identified 106 systematic review articles for Q1 and 751 for Q2. In comparison, the conventional strategy yielded 69.81% and 26.23% fewer articles than the WINK strategy, respectively (Table 1).

III. Results

The differences between the WINK and conventional search strategies are summarized in Table 1. The WINK approach demonstrated significantly higher sensitivity, retrieving a broader range of eligible articles for both Q1 and Q2.

For Q1, the conventional search identified 74 articles, of which 16 were deemed ineligible, leaving 58 eligible articles. In contrast, the WINK strategy retrieved 106 articles, with 80 meeting the eligibility criteria. Notably, 46 of these eligible articles were also identified by the conventional search, while the WINK strategy uniquely contributed 34 additional relevant articles.

For Q2, 65.4% of articles (129) retrieved through the conventional search and 53.8% of articles (404) retrieved through the WINK strategy were deemed relevant to the study objectives. Although conventional searches yielded a more focused set of results, the WINK approach demonstrated its ability to access a broader array of studies by incorporating additional MeSH terms, thereby expanding the scope and comprehensiveness of the reviews. This method also led to the discovery of 19 new MeSH terms related to oral health and 22 new terms concerning systemic health, which were not included in the conventional search. This underscores the complementary nature of the two approaches and the potential to enhance systematic reviews by integrating both methodologies. By merging the results from both strategies, we increased the breadth and depth of the article pool, ensuring a more comprehensive and robust foundation for addressing the study’s research objectives. To enhance transparency and reproducibility, we have meticulously detailed the screening process for one of the key research questions (Q1). A curated list of eligible and ineligible articles, identified through both conventional methods and the WINK strategy, is available in the public data repository at https://osf.io/wmxda/.

IV. Discussion

This study introduces a novel methodology, the WINK technique, aimed at optimizing the selection of MeSH terms and keywords for systematic reviews. By leveraging bibliometric analysis via the VOSviewer platform, this method systematically prioritizes keywords based on their co-occurrence patterns within the literature. This prioritization improves the construction of search strategies, ensuring a more targeted and comprehensive retrieval of relevant resources. The literature underscores the critical importance of efficient keyword detection in developing effective search strings, highlighting the need for precision and inclusivity in systematic review methodologies [7,19].

Traditional methods for selecting keywords in scientific literature searches often depend heavily on expert input, which can be both time-consuming and error-prone [12,20]. The WINK strategy introduces a more systematic approach that significantly enhances the retrieval of relevant articles and improves the comprehensiveness of systematic reviews. This method not only streamlines the identification of key terms but also incorporates expert insights to maintain accuracy and relevance. By merging automation with expert input, the WINK strategy lays a robust foundation for evidence synthesis and analysis [13]. This data-driven approach improves the precision of keyword selection and ensures that the search strategy covers a broad spectrum of relevant topics and subtopics, thereby reducing the likelihood of missing important literature.

Siddaway et al. [2] highlighted the need to strike a balance between sensitivity and specificity when selecting articles. He recommended that during the initial phase of the search process, prioritizing sensitivity through the use of broad search terms can capture a wide array of potentially relevant articles, even if some may be irrelevant. Although this method may lead to a larger pool of studies for review, it ensures a thorough exploration of the literature and reduces the risk of overlooking important studies. Our method improves both sensitivity and specificity by choosing keywords that have strong network connections in the evidence and by incorporating insights from subject matter experts. This dual strategy reduces the chances of retrieving an excess of irrelevant studies, thereby optimizing the balance between inclusivity and precision in the systematic review process. An evaluation by Shaw et al. [21] confirmed that an effective strategy for retrieving qualitative research involves using a mix of specific free-text terms, broad terms, and thesaurus terms. Solely relying on one type of search term is insufficient for identifying relevant records. The findings further suggest that employing a variety of search terms is crucial for optimizing evidence synthesis. Thus, identifying additional search terms through literature co-occurrences can lead to the retrieval of more sensitive and specific articles that align with our search objectives.

With the integration of open-access network visualization tools like VOSviewer [17], researchers can identify clusters of related terms and explore the strength of associations between different contexts of the research objective. In our study, the use of network visualization tools resulted in the inclusion of 69.81% and 26.23% more articles for Q1 and Q2, respectively, compared to conventional approaches.

The methods for selecting keywords for conducting a systematic review have been extensively studied in the literature for a long time. However, a consensus on a more user-friendly and accurate technique remains elusive and debatable. Grames et al. [22] conducted one such study, introducing an R Studio package called “litsearchr” for keyword selection, which requires technical expertise to execute the process. In contrast, VOSviewer offers a user-friendly alternative for identifying keyword co-occurrences, featuring an intuitive graphical interface that does not require extensive programming knowledge. Its tools, such as network maps and density maps, allow researchers to effectively visualize and explore keyword relationships. While litsearchr in R Studio provides flexibility, it demands proficiency in R programming and manual scripting, which can be challenging for non-programmers. For those seeking a more accessible and efficient approach, VOSviewer is often the preferred option.

In our study, we focused on searching through the open-access, free-source search engine PubMed. A study by Bramer et al. [23] evaluated the performance of various databases in retrieving articles for systematic reviews. It revealed that Google Scholar achieved a high coverage rate of 97.2%, though it was slightly lower than the combined coverage of Embase and MEDLINE at 97.5%. MEDLINE alone demonstrated notable coverage of 92.3%. However, total recall was highest for the combined Embase/MEDLINE at 81.6%, compared to 72.8% for GS and 72.6% for MEDLINE alone. These findings underscore the efficiency and reliability of PubMed, powered by MEDLINE, as an open-access platform. It serves as a primary tool for systematic reviews, offering robust coverage, ease of use, and access to high-quality, indexed literature.

Radhakrishnan et al. [24] conducted a study on the use of a keyword co-occurrence network (KCN) based presystematic review method for keyword selection. However, relying solely on KCN may introduce bias, as it could lead to the overrepresentation or underrepresentation of certain keywords in the literature, resulting in skewed outcomes. In our methodology, we included MeSH terms extracted from articles obtained during the initial search phase. Subject experts then assessed the relevance of these co-occurring patterns, taking into account their importance to the research question. Their insights were crucial and were carefully integrated into the weighted network analysis process [20].

By combining expert insights with systematic methodologies such as the WINK technique, the review process can be enhanced to develop a more comprehensive and effective search strategy, while also minimizing potential biases. The WINK technique proves particularly valuable in situations where formulating a search strategy or pinpointing relevant terms for effective evidence retrieval is challenging. In these cases, it serves as an efficient tool for identifying articles that align with our research objectives, providing a structured approach that simplifies the evidence retrieval process and improves the efficacy of literature searches.

The WINK technique demonstrated a significant advantage in identifying additional relevant articles by employing an expanded search strategy; however, it also presents certain limitations. The increased number of articles necessitates a rigorous eligibility screening to maintain qualitative relevance to the research objectives. This screening process is essential for both newly identified articles and all articles retrieved from the databases. Therefore, the WINK technique may yield irrelevant articles along with relevant ones. This underscores the importance of comprehensive screening protocols and highlights the need to balance enhanced article retrieval with the potential for an increased workload in screening and verification. Despite this limitation, the additional relevant articles retrieved through the WINK strategy underscore its utility in systematic review methodology.

In conclusion, the WINK technique presents a robust, innovative, and user-friendly approach to developing search strategies for systematic reviews. By combining bibliometric analysis with expert input, WINK improves the precision and comprehensiveness of literature retrieval, significantly outperforming conventional strategies in retrieving relevant articles. This methodology optimally balances sensitivity and specificity and enables researchers to effectively visualize and explore keyword relationships, thus enriching the evidence base for systematic reviews. As the field of systematic reviews progresses, WINK stands out as a valuable tool for enhancing the rigor and efficiency of evidence synthesis.

Article information Continued