research question spider

Qualitative Research

• What is Qualitative Research
• Developing a Qualitative Question: PEO
• Tips for Finding Qualitative Studies
• Critical Appraisal of Qualitative Research Articles
• Qualitative Synthesis
• Developing a Mixed-Methods Question: SPIDER

SPIDER: Mixed Methods Qualitative Research Questions

SPIDER  is a search strategy for finding research to answer a mixed-method qualitative research question.

• Sample:  Sample is similar to Patient/Population of PICO. This is the group of people you are interesting in studying qualitatively. For example, in the qualitative research question, "What are the barriers felt by clinicians that lead to the reluctance to use EBP in practice?", S = clinicians  
• Phenomenon of Interest:  The Phenomenon of Interest can be similar to Intervention of PICO. This can be viewed as the topic of the research. For example, in the qualitative research question, "What are the barriers felt by clinicians that lead to the reluctance to use EBP in practice?", PI = Use of EBP  
• Focus Groups
• Observations  
• Evaluation:  Evaluation is similar to Outcomes of PICO. For example, in the qualitative research question, "What are the barriers felt by clinicians that lead to the reluctance to use EBP in practice?", E = Barriers to use of EBP  
• Phenomenology
• Ethnography
• Grounded theory

Using SPIDER for Search Terms

Now that you have your qualitative research questions broken into SPiDER, you can now think about your search strategy and search terms.

For example, in the qualitative research question, " What are the barriers felt by clinicians that lead to the  reluctance to use EBP in practice?"

NOTE: You might not use all these terms in your search.

FINAL BOOLEAN SEARCH:

(clinician* OR health care professional OR health care provider) AND (evidence-based practice OR EBP) AND  ( interview* OR focus group*) AND (barrier* OR hinder* OR resist*) AND phenomenology

Cooke, A., Smith, D., & Booth, A. (2012). Beyond PICO: The SPIDER tool for qualitative evidence  synthesis.  Quality Health Research, 22 (10), 1435-1443. doi: 10.1177/1049732312452938

• << Previous: Mixed Methods Research
• Last Updated: Oct 17, 2024 9:11 AM
• URL: https://researchguides.gonzaga.edu/qualitative

Frameworks for creating answerable (re)search questions

• Definition of research question frameworks
• Further help

SPIDER Framework

-useful for finding qualitative and, in particular, mixed-method research:

S - Sample (similar to Patient/Population of PICO )

PI - Phenomenon of interest (similar to PICO Intervention = topic of the research)

D - Design (methods used by the qualitative researchers to gather data, e.g. Focus Groups, Interviews,        Observations)

E - Evaluation (similar to PICO Outcomes )

R - Research type (e.g. Phenomenology, Ethnography, Grounded theory, Case study)

"What are the barriers felt by clinicians that lead to the reluctance to use EBP in practice?"

Possible keywords for the corresponding database search:

clinician* OR health care professional OR health care provider AND evidence-based practice OR EBP AND interview* OR focus group* AND barrier* OR hinder* OR resist* AND phenomenology

Cooke, A., Smith, D. and Booth, A. (2012) @Beyond PICO: The SPIDER tool for qualitative evidence synthesis',  Quality Health Research, 22 (10), pp. 1435-1443. DOI: 10.1177/1049732312452938.

• << Previous: SPICE
• Next: ECLIPSE >>
• Last Updated: Sep 4, 2024 11:44 AM
• URL: https://library.lsbu.ac.uk/questionframeworks

Evidence-Based Practice: PICO and SPIDER

• What is EBP?
• PICO and SPIDER
• More Resources
• Types of Research
• Levels of Evidence
• How to Search for Evidence
• Where to Search for Evidence
• Literature Reviews / Systematic Reviews
• Clinical Colleagues as a Source of Evidence
• What is Being Appraised?
• Critical Appraisal Tools
• Step 4: Apply
• Step 5: Assess/Audit
• EBP Resources by Discipline

The PICO concept is commonly used to formulate the clinical question.

It is especially useful for questions that are about a therapy or intervention. PICO is commonly used when one intervention is being compared with another, or with no intervention at all.

Each of the 4 letters identifies a key concept that will help you formulate your research question and/or your search strategy:

Another acronym that is sometimes used is PICOTT. The extra letters are for:

PICO Examples

Health example.

"I work in an aged care facility where urinary tract infections are a common problem. I've heard that cranberry juice can help prevent UTIs. I wonder if there's any evidence for that and whether it might help our patients?"

P      Patients in aged care homes

I       Cranberry juice

C      No intervention (status quo)

O      Prevention of UTIs.

NON-HEALTH EXAMPLE

"I conduct Library induction classes for new students in an academic library. I wonder if playing music during the class would improve their retention of information and their experience of using the library."

P      First year students at an academic library attending face-to-face induction classes

I       Playing background music in the classroom

C      No intervention (status quo)

O      Increased retention of information and improved student comfort with the library.

The SPIDER tool can be used when dealing with qualitative research questions - that is, when the research is about attitudes and experiences rather than scientifically measurable data. It focuses less on the intervention and more on the design of the study, and deals with "samples" rather than a "patient" or "populations".

Research questions framed using the SPIDER tool tend to begin with "What are the experiences of ...?"

SPIDER Example

"What are the experiences of first year university students in using their library?"

S      First year university students

PI     Library use

D      Survey

E      Experiences (of using the Library)

R      Qualitative

• << Previous: Step 1: Ask
• Next: More Resources >>
• Last Updated: Aug 19, 2024 3:38 PM
• URL: https://libguides.csu.edu.au/ebp

Charles Sturt University is an Australian University, TEQSA Provider Identification: PRV12018. CRICOS Provider: 00005F.

Beyond PICO: the SPIDER tool for qualitative evidence synthesis

Affiliation.

• 1 Central Manchester NHS Foundation Trust, Manchester, United Kingdom. [email protected]
• PMID: 22829486
• DOI: 10.1177/1049732312452938

Standardized systematic search strategies facilitate rigor in research. Current search tools focus on retrieval of quantitative research. In this article we address issues relating to using existing search strategy tools, most typically the PICO (Population, Intervention, Comparison, Outcome) formulation for defining key elements of a review question, when searching for qualitative and mixed methods research studies. An alternative search strategy tool for qualitative/mixed methods research is outlined: SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type). We used both the SPIDER and PICO search strategy tools with a qualitative research question. We have used the SPIDER tool to advance thinking beyond PICO in its suitable application to qualitative and mixed methods research. However, we have highlighted once more the need for improved indexing of qualitative articles in databases. To constitute a viable alternative to PICO, SPIDER needs to be refined and tested on a wider range of topics.

• Information Storage and Retrieval / methods*
• Meta-Analysis as Topic
• Qualitative Research*
• Review Literature as Topic*
• Sensitivity and Specificity

• Reserve a study room
• Library Account
• Undergraduate Students
• Graduate Students
• Faculty & Staff

How to Conduct a Literature Review (Health Sciences and Beyond)

• What is a Literature Review?

The Research Questions

• Selection Criteria
• Database Search
• Documenting Your Search
• Organize Key Findings
• Reference Management

Background vs. Foreground Questions

You may need to find answers to background questions (i.e. about general knowledge) before seeking answers to foreground questions (i.e. about specific knowledge, such as information that might inform a clinical decision).

The research questions on this page are for foreground questions.

A well-formulated research question:

• starts your entire search process
• provides focus for your searches
• guides the selection of literature sources

Question formats are helpful tools researchers can use to structure a question that will facilitate a focused search. Such formats include: PICO , PEO , SPIDER , and  COSMIN . Other formats can be found here .  

The  PICO  format is commonly used in evidence-based clinical practice.  This format creates a "well-built" question that identifies four concepts: (1) the P atient problem or P opulation, (2) the I ntervention, (3) the C omparison (if there is one), and (4) the O utcome(s) .

Example : In adults with recurrent furunculosis (skin boils), do prophylactic antibiotics, compared to no treatment, reduce the recurrence rate?  ( Cochrane Library Tutorial, 2005 )

The  PEO  question format is useful for qualitative research questions. Questions based on this format identify three concepts: (1) P opulation, (2) E xposure, and (3) O utcome(s) .

Example:  In infants, is there an association between exposure to soy milk and the subsequent development of peanut allergy ( Levine, Ioannidis, Haines, & Guyatt, 2014 )?

The  SPIDER  question format was adapted from the PICO tool to search for qualitative and mixed-methods research.  Questions based on this format identify the following concepts: (1) S ample, (2) P henomenon of I nterest, (3) D esign, (4) E valuation, and (5) R esearch type .

Example:  What are young parents’ experiences of attending antenatal education? 

Search for ( S  AND  P of I   AND ( D  OR  E ) AND  R ) ( Cooke, Smith, & Booth, 2012 ).

The COSMIN  ( CO nsensus-based  S tandards for the selection of health status M easurement IN struments ) format is used for systematic review of measurement properties.  Questions based on this format identify (1) the construct or the name(s) of the outcome measurement instrument(s) of interest,  (2) the target population, (3) the type of measurement instrument of interest, and (4) the measurement properties on which the review focuses.

Visit the COSMIN website to view the COSMIN manual and checklist.

• << Previous: What is a Literature Review?
• Next: Selection Criteria >>
• Last Updated: Mar 15, 2024 12:22 PM
• URL: https://guides.library.vcu.edu/health-sciences-lit-review

• UNC Libraries
• HSL Subject Research
• Forming Focused Questions with PICO
• Other Question Frameworks

Forming Focused Questions with PICO: Other Question Frameworks

Created by health science librarians.

• PICO's Limitations
• PICO Examples

Other question development frameworks

PICO is a helpful framework for clinical research questions, but may not be the best for other types of research questions. Did you know there are at least 25 other question frameworks besides variations of PICO?  Frameworks like PEO, SPIDER, SPICE, ECLIPSE, and others  can help you formulate a focused research question. The table and example below were created by the  Medical University of South Carolina (MUSC) Libraries .

The PEO question framework is useful for qualitative research topics. PEO questions identify three concepts: population, exposure, and outcome.

Research question : What are the daily living experiences of mothers with postnatal depression?

The SPIDER question framework is useful for qualitative or mixed methods research topics focused on "samples" rather than populations.

SPIDER questions identify five concepts: sample, phenomenon of interest, design, evaluation,and research type.

Research question : What are the experiences of young parents in attendance at antenatal education classes?

The SPICE question framework is useful for qualitative research topics evaluating the outcomes of a service, project, or intervention. SPICE questions identify five concepts: setting, perspective, intervention/exposure/interest, comparison, and evaluation.

Research question : For teenagers in South Carolina, what is the effect of provision of Quit Kits to support smoking cessation on number of successful attempts to give up smoking compared to no support ("cold turkey")?

The ECLIPSE framework is useful for qualitative research topics investigating the outcomes of a policy or service. ECLIPSE questions identify six concepts: expectation, client group, location, impact, professionals, and service.

Research question : How can I increase access to wireless internet for hospital patients?

• Booth A, Noyes J, Flemming K, Moore G, Tunçalp Ö, Shakibazadeh E. Formulating questions to explore complex interventions within qualitative evidence synthesis. BMJ Glob Health. 2019;4(Suppl 1):e001107-e001107.
• UMD Libraries' Guide to Question Frameworks
• << Previous: PICO Examples
• Next: Resources >>
• Last Updated: May 14, 2024 12:49 PM
• URL: https://guides.lib.unc.edu/pico

• Research article
• Open access
• Published: 21 November 2014

PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews

• Abigail M Methley 1 ,
• Stephen Campbell 1 , 4 ,
• Carolyn Chew-Graham 2 ,
• Rosalind McNally 3 &
• Sudeh Cheraghi-Sohi 1 , 4  

BMC Health Services Research volume  14 , Article number:  579 ( 2014 ) Cite this article

242k Accesses

1265 Citations

56 Altmetric

Metrics details

Qualitative systematic reviews are increasing in popularity in evidence based health care. Difficulties have been reported in conducting literature searches of qualitative research using the PICO search tool. An alternative search tool, entitled SPIDER, was recently developed for more effective searching of qualitative research, but remained untested beyond its development team.

In this article we tested the ‘SPIDER’ search tool in a systematic narrative review of qualitative literature investigating the health care experiences of people with Multiple Sclerosis. Identical search terms were combined into the PICO or SPIDER search tool and compared across Ovid MEDLINE, Ovid EMBASE and EBSCO CINAHL Plus databases. In addition, we added to this method by comparing initial SPIDER and PICO tools to a modified version of PICO with added qualitative search terms (PICOS).

Results showed a greater number of hits from the PICO searches, in comparison to the SPIDER searches, with greater sensitivity. SPIDER searches showed greatest specificity for every database. The modified PICO demonstrated equal or higher sensitivity than SPIDER searches, and equal or lower specificity than SPIDER searches. The modified PICO demonstrated lower sensitivity and greater specificity than PICO searches.

Conclusions

The recommendations for practice are therefore to use the PICO tool for a fully comprehensive search but the PICOS tool where time and resources are limited. Based on these limited findings the SPIDER tool would not be recommended due to the risk of not identifying relevant papers, but has potential due to its greater specificity.

Peer Review reports

Systematic reviews are a crucial method, underpinning evidence based practice and informing health care decisions [ 1 ],[ 2 ]. Traditionally systematic reviews are completed using an objective and primarily quantitative approach [ 3 ] whereby a comprehensive search is conducted, attempting to identify all relevant articles which are then integrated and assimilated through statistical analysis. The comprehensiveness of the search process has been viewed as a key factor in preventing bias and providing a true representation of available research [ 4 ]. Current research investigating the process of quantitative systematic reviews therefore focuses on methods for ensuring the most comprehensive and bias free searches possible [ 5 ]. Because of the time and resources required to complete a systematic and comprehensive search, efforts have been made to investigate the sensitivity of searches, and thus lessen the amount of time spent reviewing irrelevant articles with no benefit [ 6 ].

However, conducting comprehensive searches also forms the bedrock of qualitative or narrative reviews, now commonly referred to as qualitative evidence syntheses [ 7 ]. Qualitative evidence syntheses are now acknowledged as a necessary and valuable type of information to answer health services research questions [ 8 ]. However, difficulties in completing a sensitive yet comprehensive search of qualitative literature have been previously noted [ 9 ]-[ 11 ] including: poor indexing and use of key words of qualitative studies, the common use of titles that lack the keywords describing the article, and unstructured abstracts.

When devising a search strategy, a search tool is used as an organising framework to list terms by the main concepts in the search question, especially in teams where it is not possible to have an experienced information specialist as a member of the review team. The PICO tool focuses on the Population, Intervention, Comparison and Outcomes of a (usually quantitative) article. It is commonly used to identify components of clinical evidence for systematic reviews in evidence based medicine and is endorsed by the Cochrane Collaboration [ 2 ]. Due to its target literature base several of these search terms such as “control group” and “intervention” are not relevant to qualitative research which traditionally does not utilise control groups or interventions, and therefore may not appropriately locate qualitative research. However, these terms may become more relevant in the future as more trials and interventions incorporate qualitative research [ 12 ].

As the PICO tool does not currently accommodate terms relating to qualitative research or specific qualitative designs, it has often been modified in practice to “PICOS” where the “S” refers to the Study design [ 4 ], thus limiting the number of irrelevant articles.

Cooke et al. also addressed this issue of relevance by developing a new search tool entitled “SPIDER” (sample, phenomenon of interest, design, evaluation, research type), designed specifically to identify relevant qualitative and mixed-method studies [ 9 ]. The key features and differences of the SPIDER and PICO search tools are shown in Table  1 . The addition of the “design” and “research type” categories to the SPIDER tool was intended to further increase the ability of this tool to identify qualitative articles, whilst removing irrelevant PICO categories such as the “comparison” group [ 9 ].

Cooke et al. recommended that the SPIDER tool was tested further in qualitative literature searches [ 9 ]. Although it has been used previously in a scoping review to investigate gaps in an evidence base on community participation in rural health care [ 13 ], SPIDER has not yet been tested and evaluated in a qualitative systematic narrative review context. The authors of this article recently completed a systematic review of the qualitative research investigating experiences of health care services for people with Multiple Sclerosis [ 14 ]. On embarking on this review topic we faced many of the difficulties commonly discussed in identifying qualitative literature on a given topic, and identified SPIDER as a potential way of overcoming some of these difficulties. Therefore, the aim of this article was to test SPIDER by broadly replicating the work of Cooke et al. [ 9 ], specifically by comparing the two approaches: 1) the traditional PICO method of searching electronic databases with 2) the newly devised SPIDER tool, developed for qualitative and mixed-method research. In addition we wished to build and expand on the work of Cooke et al. [ 9 ] and so our third aim was to compare PICO and SPIDER to a modified PICO with qualitative study designs (PICOS, see Table  1 by investigating specificity and sensitivity across 3 major databases.

Inclusion and exclusion criteria

Studies eligible for inclusion were those that qualitatively investigated patients’ experiences, views, attitudes to and perceptions of health care services for Multiple Sclerosis. No date restriction was imposed on searches as this was an original review. Qualitative research, for this purpose, was defined by the Cochrane qualitative methods group [ 7 ] as using both a qualitative data collection method and qualitative analysis. Quantitative and mixed method studies were therefore excluded.

We define experience as “ Patients’ reports of how care was organised and delivered to meet their needs p.301” [ 15 ]. Patients’ reports could refer to either experience of health care services delivery and organisation overall or their experiences of care by specific health care personnel. We included studies that investigated adults (aged 18 years old and older) with a diagnosis of Multiple Sclerosis, who had experience of utilising health care services at any time point. There were no restrictions on subtype of Multiple Sclerosis, gender, ethnicity or frequency of use of health care. Health care in this sense referred to routine clinical care (either state funded or privately funded) not trial protocols or interventions. Excluded studies included studies that focussed on self-management and studies that investigated quality of life.

Because of the focus on Multiple Sclerosis, studies were excluded if they used a mixed sample of various conditions (e.g. studies reported a mixed sample of people with neurological conditions) or if they used a sample of mixed respondents (i.e. people with Multiple Sclerosis and their carers) where results of patients with Multiple Sclerosis could not be clearly separated. If an article had a section or subtheme on health care services but this was not the main research area of the article, then that article was included; however only data from the relevant subtheme were extracted and included in the findings. Additional exclusion criteria were articles that only described carer or health care professional experiences not patient experiences. Conference abstracts, editorials and commentaries were not included.

Search strategy

For this systematic search we developed a detailed search strategy in collaboration with a specialist librarian and information specialist. This search strategy was tailored to the three largest medical and nursing databases (Ovid MEDLINE, Ovid EMBASE, and EBSCO CINAHL Plus) as in Cooke et al.’s study [ 9 ] and search terms used a mixture of medical subject headings and keywords. To investigate the benefit of the SPIDER,PICO and PICOS tools we used identical search terms but combined them in different ways as shown in Tables  2 , 3 and 4 below.

One reviewer judged titles and abstracts against the inclusion criteria. If a title and abstract met the inclusion criteria then full text copies of all articles were retrieved for further investigation. Two authors reviewed these full text articles independently for relevance to the search aim (i.e. patients/service users with multiple sclerosis, experiences of health care services and qualitative research). Any disagreements were resolved via discussion. Data from included studies were extracted by both reviewers independently to ensure accuracy and then stored on a Microsoft Excel spread sheet. No ethical approval was required for this study.

All searches spanned from database inception until 12th October 2013. As in Cooke et al. [ 9 ], we reviewed our findings based on two metrics; the number of hits generated and of these, the number relevant to the search aim (see Table  5 ).

Number of articles generated

As found in Cooke et al. [ 9 ], PICO created a much greater number of hits compared to SPIDER. A total of 23758 hits were generated using PICO, 448 hits were generated using PICOS and 239 hits were generated using SPIDER. Overall, the average reduction of hits (% across all three databases) was 98.58% for SPIDER vs. PICO, 97.94% for PICO vs. PICOS and 68.64% for PICOS vs. SPIDER. The time spent screening hits for relevant articles equated to weeks for the PICO hits and hours for the PICOS and SPIDER hits.

Proportion of relevant articles

Articles which met the inclusion criteria after full text review are displayed in Table  6 [ 16 ]-[ 33 ]. Examination of the titles and abstracts of the identified articles resulted in the obtainment of 18 full text articles relevant at full text, across all databases and search tools.

For the PICO tool in CINAHL Plus, 5.78% of hits were deemed relevant after the title and abstract stage (78 articles/1350 articles), and 14/78 articles (17.95%) were confirmed to meet the inclusion criteria after full text review. For the PICO tool in MEDLINE, 0.42% of hits were deemed relevant after the title and abstract stage (34 articles/8158 articles) and 12/34 (35.29%) articles were confirmed to meet the inclusion criteria after full text review. For the PICO tool in EMBASE, 0.25% hits were deemed relevant after the title and abstract stage (35 articles/ 14250 articles) and 14/35(40%) articles were confirmed to meet the inclusion criteria after full text review.

For the PICOS tool in CINAHL Plus, 38.36% of articles were relevant after the title and abstract stage (56 articles/146 articles) and 12/56 (21.43%) were confirmed to meet the inclusion criteria after full text review. For the PICOS tool in MEDLINE 14.16% of articles were relevant after the title and abstract stage (16 articles/ 113 articles) and 6/16 (37.5%) were confirmed to meet the inclusion criteria after full text review. For the PICOS tool in EMBASE 7.94% of articles were deemed relevant after the title and abstract stage (15 articles/189 articles) and 7/15 (46.67%) were confirmed to meet the inclusion criteria after full text review.

SPIDER tool

For the SPIDER tool in CINAHL Plus 38.36% of articles were relevant after the title and abstract stage (56 articles/146 articles) and 12/56 (21.43%) were confirmed to meet the inclusion criteria after full text review. For the SPIDER tool in MEDLINE, 36.81% hits were deemed relevant at the title stage (14 articles/38 articles) and 5/14 articles (35.71%) were confirmed to meet the inclusion criteria after full text review. For the SPIDER tool in EMBASE, 16.36% were relevant at the title stage (9 articles/55 articles) and 3/9 (33.33%) were confirmed to meet the inclusion criteria after full text review.

Sensitivity and specificity

The SPIDER tool identified 13 relevant articles out of 239 articles across all three databases (5.43%) compared to PICOS which identified 13 articles out of 448 articles (2.90%) and PICO which identified 18 articles out of 23758 articles (0.076%). Of the 18 relevant articles identified by the PICO tool, 66.66% came from both MEDLINE and CINAHL Plus (12 articles each), and 72.22% came from EMBASE (13 articles). Of the 13 relevant articles identified by the PICOS tool 46.15% came from MEDLINE (6 articles), 53.84% came from EMBASE (7 articles) and 92.31% came from CINAHL Plus (12 articles). Of the 13 relevant articles identified by SPIDER, 38.46% came from MEDLINE (5 articles) and 23.07% came from EMBASE (3 articles) and 92.30% came from CINAHL Plus (12 articles) Table  7 .

Different articles were found across different tools and databases (as shown in Table  6 ). All three databases were checked for all articles. One article was available in CINAHL Plus but not identified by any of the tools [ 17 ]. Two papers were identified in all databases through all search tools. Five papers were identified in MEDLINE through all search tools, three identified in EMBASE through all search tools and 12 identified in CINAHL through all search tools. Five papers were identified solely in CINAHL Plus, with one of these papers only identified using the PICO search method. One paper was identified by all search tools in EMBASE but not identified by any in MEDLINE. No new studies were identified using the SPIDER or PICOS tools alone in any database.

In this article we addressed the aim of replicating a comparison between the SPIDER, PICOS and PICO search tools. As previously described in Cooke et al. [ 9 ], the SPIDER tool produced a greatly reduced number of initial hits to sift through, however in this study it missed five studies that were identified through the PICO method. This may be partly be explained by the nature of the research question prompting the search. As this study included subthemes of studies whose focus differed from the initial research question (i.e. only a smaller section of the paper related to health care) then it’s possible that these studies were picked up by a broader search but not the highly specific SPIDER search. Other authors researching the process of qualitative literature reviews have previously commented that there appears to be a decision to be made about the benefits of comprehensiveness of findings versus the accuracy of the studies identified [ 11 ]. Given the common nature of using sub-sections of papers for systematic reviews then our findings suggest that comprehensiveness needs to be the key for this type of search.

The PICOS tool was more specific than the PICO tool, but did not identify any additional relevant hits to the SPIDER tool, suggesting it is of approximately equal sensitivity. PICOS identified the same number of papers as the SPIDER tool and both demonstrated a substantially lower number of hits generated than a regular PICO search. The SPIDER tool showed the greatest specificity due the small number of hits generated. This may mean that review teams with very limited resources or time, and who are not aiming for a totally comprehensive search (i.e. in the case of scoping studies), would benefit from using the SPIDER tool. This might be applicable particularly to studies such as qualitative syntheses, where the research aim is theoretical saturation, not a comprehensive search [ 34 ]. In addition, articles written to influence policy often require swift publication, providing another area in which either SPIDER or PICOS might improve current practice.

The issue of time was also related to the number of relevant articles identified per database. Whilst EMBASE generated nearly twice as many hits as MEDLINE, only one additional paper was found. The PICO tool identified all articles, suggesting that where time is not a factor, it might be of more benefit to use this tool, as SPIDER demonstrated lower sensitivity, did not identify any new articles and identified fewer relevant articles than PICO.

Our findings indicate that it is worthwhile testing a chosen search tool across various databases as they produce different results; i.e. CINHAL Plus identified papers not identified in MEDLINE or EMBASE databases. It is therefore important for future research to investigate the potential of the SPIDER vs. PICOS and PICO tools as a base for the recommended comprehensive searching process, by investigating the contribution of the SPIDER and PICOS tools at every stage from the initial search hits, to the final included relevant articles.

As CINAHL is a database dedicated to nursing and allied health research, it was expected that it would produce a greater number of relevant articles than more medically focussed databases [ 10 ], as nursing and allied areas have traditionally been at the forefront of qualitative investigations into Multiple Sclerosis.

SPIDER proved to be a tool designed to formulate search terms easily, as it naturally fits the crucial elements of the search question. However, even though some qualitative keywords are necessary to identify qualitative studies, including the words “ qualitative research ” AND the name of the type of research e.g. “ grounded theory ” might be too restrictive, particularly given the poor use of the qualitative index term, and might partially explain the fewer studies identified by SPIDER in comparison to PICO. Studies not identified by the SPIDER model in MEDLINE and EMBASE databases did not use keywords such as “ qualitative ”, but some described qualitative methods, such as “ phenomenological-hermeneutic ” [ 16 ] or “ interview(s) ” [ 20 ],[ 23 ].

In all PICO searches for MEDLINE and EMBASE the word “ qualitative ” combined with the phrase “ multiple sclerosis ” identified many quantitative studies reporting brain scan assessments that were wholly unrelated to the search aim. This was because the word “ qualitative ” in this context referred to using a qualitative method to provide information about the quality of the scan and any potential flaws [ 35 ]. This caused a problem with specificity, resulting in thousands of inappropriate hits as there was no way to exclude studies with the word “ qualitative ” unless all articles clearly utilised and indexed qualitative research methods in the title, abstract and keywords.

Many studies were excluded at the full text stage on the basis that the samples were mixed: being comprised of either various neurological conditions or mixed groups of people i.e. patients and carers/patients and health care professionals and so forth. Without clearer titles and abstracts, and potentially an indexing phrase that indicates mixed samples, there is no way of avoiding this issue. Excluding the phrases “ caregivers ” or “ health care professionals ” would have excluded any studies that used these phrases (for example in the introduction or implication for future research sections) and therefore it is difficult to see how this could be prevented. A strength and limitation of our study is that whilst it details a real world example of evidence searching, it only addresses one topic. Further research should test these search tools against a wider variety of narrative review and meta-synthesis topics.

SPIDER greatly reduced the initial number of articles identified on a given search due to increased specificity, however because of lower sensitivity omitted many relevant papers. The PICOS tool resulted in an overall more sensitive search, but still demonstrated poor specificity on this topic. Further investigations of the specificity and sensitivity of SPIDER and PICOS on varied topics will be of benefit to research teams with limited time and resources or articles necessary to impact on policy or change current practice. However, where comprehensiveness is a key factor we suggest that the PICO tool should be used preferentially. Part of the lower identification rate for SPIDER (in comparison to PICO) was poor labelling and use of qualitative keywords in indexing studies. As both individual research submissions and journal/database indexers improve, or standardise, the indexing of qualitative studies, it is likely that the relevance of the SPIDER tool will increase. The recommendation for current practice therefore is to use the PICO tool across a variety of databases. In this article we have shown that SPIDER is relevant for those researchers completing systematic narrative reviews of qualitative literature but not as effective as PICO. Future research should investigate the use of SPIDER and PICOS across varied databases.

Authors’ information

Caroly Chew-Graham is part-funded by the National Institute for Health Research (NIHR) Collaborations for Leadership in Applied Health Research and Care West Midlands.

Stephens KR: Systematic reviews: the heart of evidence-based practice. Am Assoc Crit Care Nurs Clin Issu. 2001, 12 (4): 529-538.

Google Scholar  

Higgins JPT, Green S: Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. The Cochrane Collaboration. 2013

Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, Hsu R, Katbamna S, Olson R, Smith L, Riley R, Sutton AJ: Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol. 2006, 6: 3-10.1186/1471-2288-6-35.

Article   Google Scholar  

Centre for reviews and dissemination: Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care. 2006, University of York, York

Helmer D, Savoie I, Green C, Kazanjian A: Evidence-based practice: extending the search to find material for the systematic review. Bull Med Libr Assoc. 2001, 89 (4): 346-352.

CAS   PubMed   PubMed Central   Google Scholar  

Stevinson C, Lawlor DA: Searching multiple databases for systematic reviews: added value or diminishing returns?. Complement Ther Med. 2004, 12: 228-232. 10.1016/j.ctim.2004.09.003.

Article   CAS   PubMed   Google Scholar  

Noyes J: Never mind the qualitative feel the depth! The evolving role of qualitative research in Cochrane intervention reviews. J Res Nurs. 2010, 15: 525-534. 10.1177/1744987110381696.

Noyes J, Popay J, Pearson A, Hannes K, Booth A: Chapter 20: qualitative research and cochrane reviews. Cochrane Handbook for Systematic Reviews of Interventions. Edited by: Higgins JPT, Green S. 2008, 20.1-20.18

Cooke A, Smith D, Booth A: Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012, 22: 1435-1443. 10.1177/1049732312452938.

Article   PubMed   Google Scholar  

Evans D: Database searches for qualitative research. J Med Library Assoc. 2002, 9: 290-293.

Shaw RL, Booth A, Sutton AJ, Miller T, Smith JA, Young B, Jones DR, Dixon-Woods M: Finding qualitative research: an evaluation of search strategies. BMC Med Res Methodol. 2004, 4: 5-10.1186/1471-2288-4-5.

Article   PubMed   PubMed Central   Google Scholar  

Lewin S, Glenton C, Oxman AD: Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ 2009, 339(b3496). http://www.bmj.com/content/339/bmj.b3496 ..

Kenny A, Hyett N, Sawtell J, Dickson-Swift V, Farmer J, O’Meara P: Community participation in rural health: a scoping review. BMC Health Serv Res. 2013, 13: 64-10.1186/1472-6963-13-64.

Methley AM, Chew-Graham C, Campbell S, Cheraghi-Sohi S: Experiences of UK health care services for people with Multiple Sclerosis: a systematic narrative review. Health Expect 2014, epub ahead of print..

Sinfield P, Baker R, Camosso- Stefinovic J, Colman AM, Tarrant C, Mellon JK, Steward W, Kockelbergh R, Agarwal S: Men’s and carer’s experiences of care for prostate cancer: a narrative literature review. Health Expect. 2009, 12: 301-312. 10.1111/j.1369-7625.2009.00546.x.

Lohne V, Aasgaard T, Caspari S, Slettebo A, Naden D: The lonely battle for dignity: individuals struggling with multiple sclerosis. Nurs Ethics. 2010, 17 (3): 301-311. 10.1177/0969733010361439.

Mackereth PA, Booth K, Hillier VF, Caress A: Reflexology and progressive muscle relaxation training for people with multiple sclerosis: a crossover trial. Complement Ther Clin Pract. 2008, 15: 14-21. 10.1016/j.ctcp.2008.07.002.

Isaksson AK, Ahlström G: Managing chronic sorrow: experiences of patients with multiple sclerosis. J Neurosci Nurs. 2008, 40 (3): 180-191. 10.1097/01376517-200806000-00009.

Edwards RG, Barlow JH, Turner AP: Experiences of diagnosis and treatment among people with multiple sclerosis. J Eval Clin Pract. 2008, 14 (3): 460-464. 10.1111/j.1365-2753.2007.00902.x.

Barker-Collo S, Cartwright C, Read J: Into the unknown: The experiences of individuals living with multiple sclerosis. J Neurosci Nurs. 2006, 38 (6): 435-441. 10.1097/01376517-200612000-00008.

Isaksson AK, Ahlström G: From symptom to diagnosis: illness experiences of multiple sclerosis patients. J Neurosci Nurs. 2006, 38 (4): 229-237. 10.1097/01376517-200608000-00005.

Miller CE, Jezewski MA: Relapsing MS patients’ experiences with glatiramer acetate treatment: a phenomenologic study. J Neurosci Nurs. 2006, 38 (1): 37-41. 10.1097/01376517-200602000-00008.

Johnson J: On receiving the diagnosis of multiple sclerosis: managing the transition. Mult Scler. 2003, 9 (1): 82-88. 10.1191/1352458503ms856oa.

Miller CE, Jezewski MA: A phenomenologic assessment of relapsing MS patients’ experiences during treatment with Interferon Beta-1(*). J Neurosci Nurs. 2001, 33 (5): 240-244. 10.1097/01376517-200110000-00004.

Miller CM: The lived experience of relapsing multiple sclerosis: a phenomenological study. J Neurosci Nurs. 1997, 29 (5): 294-304. 10.1097/01376517-199710000-00003.

Aars H, Bruusgaard D: Chronic disease and sexuality: an interview study on sexual dysfunction in patients with multiple sclerosis. Tidsskrift Den Norske Laegeforening. 1989, 109 (32): 3352-3354.

CAS   Google Scholar  

Rintell DJ, Frankel D, Minden SL, Glanz BI: Patients’ perspectives on quality of mental health care for people with MS. Gen Hosp Psychiatry. 2012, 34 (6): 604-610. 10.1016/j.genhosppsych.2012.04.001.

Laidlaw A, Henwood S: Patients with multiple sclerosis: their experiences and perceptions of the MRI investigation. J Diagn Radiogr Imaging. 2003, 5 (1): 19-25. 10.1017/S146047280300004X.

Koopman W, Schweitzer A: The journey to multiple sclerosis: a qualitative study. J Neurosci Nurs. 1999, 31 (1): 17-26. 10.1097/01376517-199902000-00003.

Hansen AK, Krogh H, Bangsgaard L, Aabling S: Facing the diagnosis. Sygeplejersken. 2008, 4: 52-56.

Loveland CA: The experiences of African americans and euro-americans with multiple sclerosis. Sex Disabil. 1999, 17 (1): 19-35. 10.1023/A:1021499628918.

Moriya R, Suzuki S: A qualitative study relating to the experiences of people with MS: differences by disease severity. Br J Neurosci Nurs. 2011, 7 (4): 593-600. 10.12968/bjnn.2011.7.4.593.

Classen S, Lou JQ: Exploring rehabilitation and wellness needs of people with MS living in South Florida: a pilot study. Intern J MS Care. 2004, 1: 26-31. 10.7224/1537-2073-6.1.26.

Booth A: Cochrane or cock-eyed? How should we conduct systematic reviews of qualitative research? In Qualitative Evidence-Based Practice Conference. Coventry; 2001. [ http://www.leeds.ac.uk/educol/documents/00001724.htm ]

Chalavi S, Simmons A, Dijkstra H, Barker GJ, Reindeers AATS: Quantitative and qualitative assessment of structural magnetic resonance imaging data in a two-center study. BMC Med Imaging. 2012, 12: 27-10.1186/1471-2342-12-27.

Download references

Acknowledgements

This study was funded by a School for Primary Care Research PhD studentship from the National Institute of Health Research. Support in selecting search terms is acknowledged from Olivia Walsby, Academic Engagement Librarian at the University of Manchester. We are grateful to Professor Peter Bower for his comments on the protocol.

Author information

Authors and affiliations.

University of Manchester, Centre for Primary Care, Williamson Building, Oxford Road, Manchester, M13 9PL, UK

Abigail M Methley, Stephen Campbell & Sudeh Cheraghi-Sohi

Institute of Primary Care and Health Sciences, Keele University, Keele, UK

Carolyn Chew-Graham

Central Manchester Hospitals Site, Manchester Mental Health and Social Care Trust, Research and Innovation 3rd Floor, Rawnsley Building, Hathersage Road, Manchester, M13 9WL, UK

Rosalind McNally

NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre, Institute of Population Health, The University of Manchester, Manchester, M13 9WL, UK

Stephen Campbell & Sudeh Cheraghi-Sohi

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Abigail M Methley .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors’ contributions

AM designed the study, conducted all searches, appraised all potential studies and wrote and revised the draft manuscript and subsequent manuscripts. SC made significant contributions to the conception and design of the study, assisted with the presentation of findings and assisted with drafting and revising the manuscript. CCG and RM made significant contributions to the conception and design of the study, assisted with the presentation of findings and assisted with drafting and revising the manuscript. SCS conceived and designed the study, assisted with searches, appraised relevant studies and assisted with drafting and revising the manuscript. All authors read and approved the final manuscript.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/4.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Methley, A.M., Campbell, S., Chew-Graham, C. et al. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res 14 , 579 (2014). https://doi.org/10.1186/s12913-014-0579-0

Download citation

Received : 03 February 2014

Accepted : 03 November 2014

Published : 21 November 2014

DOI : https://doi.org/10.1186/s12913-014-0579-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health care
• Users’ experiences
• Multiple sclerosis (MS)
• Research evaluation
• Qualitative
• Systematic reviews

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]