Warning: fopen(/home/virtual/epih/journal/upload/ip_log/ip_log_2024-12.txt): failed to open stream: Permission denied in /home/virtual/lib/view_data.php on line 95 Warning: fwrite() expects parameter 1 to be resource, boolean given in /home/virtual/lib/view_data.php on line 96 Narrative reviews


Your browser does not support the NLM PubReader view.

Go to this page to see a list of supporting browsers.

Narrative reviews

Article information

Epidemiol Health. 2014;36.e2014018
Publication date (electronic) : 2014 September 11
doi : https://doi.org/10.4178/epih/e2014018
Jong-Myon Bae
Department of Preventive Medicine, Jeju National University School of Medicine, Jeju, Korea
Correspondence: Jong-Myon Bae  Department of Preventive Medicine, Jeju National University School of Medicine, 102 Jejudaehak-ro, Jeju 690-756, Korea  Tel: +82-64-755-5567, Fax: +82-64-725-2593, E-mail: jmbae@jejunu.ac.kr
Received 2014 September 3; Accepted 2014 September 11.

Abstract

Although qualitative researches (QR) are invaluable in understanding complex healthcare situations, the quantitative systematic reviews could not treat them. To improve quality of healthcare services, results of QR should be considered in healthcare decision-making processes. Several methods and theories for synthesizing evidences of QR have been developed. In order to activate the narrative reviews and mixed methods reviews in Korean healthcare academies, I arranged the related nomenclatures and suggested some issues to conduct them.

Keywords: Review literature as topic; Qualitative research; Qualitative evaluation; Meta-analysis as topic; Evidence-based practice

INTRODUCTION

In providing healthcare services, the evidence-based medicine (EBM) paradigm [1-3]—the idea that healthcare-related decisions should be made on the basis of the best evidence—is expanding to healthcare-related policy-making in nursing and health science as well as medicine [4-6]. Amidst this trend, quantitative systematic reviews is a new research approach. It is a term used to designate the approach to examining the effects of a treatment or an intervention adopting a statistical method called meta-analysis [7]. However, the results of qualitative research which cannot obtain quantitative outcomes, such as relative risk, odds ratio, and number needed to treat (NNT), tend to be ignored or excluded in the application of the systematic reviews process [8].

Not only the definition of qualitative research, but also the characteristic comparisons between qualitative and quantitative research are well documented in the papers by Murphy et al. [9] and Draper [10]. In short, qualitative research can be a useful guide for understanding complicated situations in the real world and can serve as groundwork for new hypotheses [11-15]. Consequently, efforts to use the synthesis of qualitative research results as the grounds for decision-making were already underway in 1990s when EBM emerged [16-23]. However, recently, in line with the trend toward improving medical treatment quality through patient-centered and evidence-based diagnostic and therapeutic services [24,25], efforts for complementary utilization of qualitative research results are experiencing a revival [13, 15,26-34]. In particular, the claim that both quantitative and qualitative research should be utilized for proper understanding of overall healthcare problems is gaining influence. [18,35,36].

Meanwhile, because the synthesis of qualitative research results is inevitably different from a quantitative systematic reviews, there have been many attempts to overcome this gap [8, 37-40]. For this reason, I would like to systemize the attempts to synthesize qualitative research results undertaken so far. This work is expected to show the framework of the discussions related to qualitative research and lay a cornerstone for the vitalization of systematic reviews on qualitative research in South Korea.

MAIN BODY

Terms related to the synthesis of qualitative research

Among the theories about qualitative research presented so far, works of Draper [10], Barnett-Page & Thomas [41], Dixon-Woods et al. [42], and Thorne et al. [43] may be representative. I arranged various terms related to qualitative research, grouping them according to the emphasis intended.Appendix 1 is the overview of the healthcare research applying this nomenclature. At a glance, we can see many terms are suggested with the emphasis placed on the meaning of the term “synthesis” of qualitative research results in contrast to the application of the meta-analytic statistical method of quantitative research results. In particular, we can see the term “mixed methods research” has often been applied recently in the attempt to include both quantitative and qualitative research [44].

Process of synthesis of qualitative research

As can be seen in Appendix 1, the high number of related terms implies that the establishment of the relevant research methodology is a difficult task [45]. Sinuff et al. [46] shows the difference between quantitative and qualitative research processes in a diagram. However, a look at the suggested processes in relation to qualitative research synthesis [34,45,47-54] reveals that they stick to the big frame of “Ask - Acquire - Appraise - Apply - Assess,” 5A of the evidence cycle, although they show a certain diversity [55].

Searches of qualitative research to collect a body of related literature are more difficult than quantitative research [56-58]. This is because of different database services and the need to search for gray literature that has been issued but not officially published and made available in market, such as reports published by institutions or academic narrative reports [8]. It is also attributable to the need to resort, in addition to securing lists through search formulae, to hand searching which involves bibliography browsing in search of related papers and snowballing searching which traces one paper after another in chronological order [14,59].

Because of the diversity of research methods and fields of application, qualitative research does not easily lend itself to standardizing the items of qualitative evaluation in the literature of interest [8,59-64]. Nevertheless, the following achievements harvested so far deserve to be listed: (1) Thomas et al. [65] suggested evaluation items which matched quantitative-qualitative research. (2) Clark [66] developed ‘RATS’ evaluation tool, which is an acronym for Relevance, Appropriateness, Transparency, and Soundness. (3) Daly et al. [67] suggested a stratified structure by the contents of qualitative research. (4) Rodgers et al. [34] applied the EPPI approach that evaluates the persuasive power of evidence. And (5) Dixon-Woods et al. [68] suggested a tool called CASP (Critical Appraisal Skills Programme tool) which is composed of 10 items.

For the synthesis of evaluation results, a best fit frameworks is established [65,69-73] or a simulations model is selected [74]. The commercial program called NVivo has been developed [52, 75,76]. Feasibility research examining the applicability of this program in Korean society needs to be accumulated.

Also, a reporting guideline named RAMESES (Realist And MEta-narrative Evidence Synthesis: Evolving Standards) has been developed for application when the research results of systematic reviews of qualitative research are to be reported in papers [77,78]. Making a flow chart is also suggested in cases where the mixed methods reviews approach is adopted which involves both quantitative and qualitative research [79,80].

CONCLUSION AND PROPOSAL

There are few systematic reviews on qualitative research in healthcare-related scholarship in South Korea. For the qualitative improvement of healthcare in this scarce situation, research in various healthcare fields should be conducted. Therefore, I suggest three things as below.

First, establishing the nomenclature of systematic reviews on qualitative research is an urgent task, because systematic and coherent use of well-established terms is important when a multitude of suggested terms are in use, as shown in Appendix 1. The synthesis of quantitative research through the meta-analytic statistical method can be termed ‘quantitative systematic reviews’ and that of qualitative research, ‘qualitative systematic reviews’ [81]. However, given the current situation that systematic reviews have been established as the major research methodology of the synthesis of the evidences of quantitative research and the meta-analysis applied to this is recognized as statistical methodology [81], systematic reviews in the narrow sense mean quantitative systematic reviews [82]. On the contrary, qualitative systematic reviews is also called narrative systematic reviews and recently in a more abbreviated form called “narrative reviews” [29,83-85]. However, the term ‘meta-narrative reviews’ [77,78] does not fit and its use should be avoided because the meta-analysis corresponding to this is a statistical method, not a research method [81]. Therefore, I suggest a terminological differentiation between (quantitative) systematic reviews and (qualitative) narrative reviews, whereby the term “mixed methods reviews” may be used when both quantitative and qualitative research are involved. It was in consideration of this point that I titled this study “Narrative Reviews.”

Second, the practical way to revitalize narrative reviews research when Korean researchers do not have much experience conducting narrative reviews is the critical reading of good research cases from various academic fields and their application to practical use. To facilitate this process, I organized useful research cases by academic field and presented them in Appendix 2.

Third, there is a need to establish a research-supporting organization and expand research human resources for qualitative research in the process of planning and conducting clinical studies [15]. The basic prior condition of proper narrative reviews is good results from qualitative research. First and foremost, given the fact that multidisciplinary cooperation is of vital importance for qualitative research, an organizational reshuffling appears necessary to facilitate efficient cooperation.

Acknowledgements

This study was conducted by a 2014 academic promotion research funding project grant of Jeju National University.

Notes

The author has no conflicts of interest to declare for this study.

SUPPLEMENTARY MATERIAL

Supplementary material is available at http://www.e-epih.org/.

epih-36-e2014018-supplementary.pdf

References

1. Miles A, Bentley P, Polychronis A, Grey J. Evidence-based medicine: why all the fuss? This is why. J Eval Clin Pract 1997;3:83–86.
2. Miles A, Bentley P, Polychronis A, Grey J, Price N. Recent progress in health services research: on the need for evidence-based debate. J Eval Clin Pract 1998;4:257–265.
3. Miles A, Bentley P, Polychronis A, Grey J, Price N. Advancing the evidence-based healthcare debate. J Eval Clin Pract 1999;5:97–101.
4. King S, Forbes K, Hanks GW, Ferro CJ, Chambers EJ. A systematic review of the use of opioid medication for those with moderate to severe cancer pain and renal impairment: a European Palliative Care Research Collaborative opioid guidelines project. Palliat Med 2011;25:525–552.
5. Fox DM. Evidence of evidence-based health policy: the politics of systematic reviews in coverage decisions. Health Aff (Millwood) 2005;24:114–122.
6. Greenhalgh T, Wong G, Westhorp G, Pawson R. Protocol--realist and meta-narrative evidence synthesis: evolving standards (RAMESES). BMC Med Res Methodol 2011;11:115.
7. Bae JM. Methodological issues for determining intervals of subsequent cancer screening. Epidemiol Health 2014;36e2014010.
8. Jones K. Mission drift in qualitative research, or moving toward a systematic review of qualitative studies, moving back to a more systematic narrative review. Qual Rep 2004;9:95–112.
9. Murphy E, Dingwall R, Greatbatch D, Parker S, Watson P. Qualitative research methods in health technology assessment: a review of the literature. Health Technol Assess 1998;2:iii-ix:1–274.
10. Draper AK. The principles and application of qualitative research. Proc Nutr Soc 2004;63:641–646.
11. Jones R. Why do qualitative research? BMJ 1995;311:2.
12. Murthy GV, Gupta SK. Qualitative research in ophthalmic sciences. Indian J Ophthalmol 1999;47:257–264.
13. Sofaer S. Qualitative methods: what are they and why use them? Health Serv Res 1999;34:1101–1118.
14. Greenhalgh T, Heath I. Measuring quality in the therapeutic relationship--part 1: objective approaches. Qual Saf Health Care 2010;19:475–478.
15. Rapport F, Storey M, Porter A, Snooks H, Jones K, Peconi J, et al. Qualitative research within trials: developing a standard operating procedure for a clinical trials unit. Trials 2013;14:54.
16. Popay J, Rogers A, Williams G. Rationale and standards for the systematic review of qualitative literature in health services research. Qual Health Res 1998;8:341–351.
17. Barbour RS. Mixing qualitative methods: quality assurance or qualitative quagmire? Qual Health Res 1998;8:352–361.
18. McDowell I, MacLean L. Blending qualitative and quantitative study methods in health services research. Health Inform J 1998;4:15–22.
19. Green J, Britten N. Qualitative research and evidence based medicine. BMJ 1998;316:1230–1232.
20. Harding G, Gantley M. Qualitative methods: beyond the cookbook. Fam Pract 1998;15:76–79.
21. Popay J, Williams G. Qualitative research and evidence-based healthcare. J R Soc Med 1998;91 Suppl 35:32–37.
22. Dixon-Woods M, Fitzpatrick R. Qualitative research in systematic reviews. Has established a place for itself. BMJ 2001;323:765–766.
23. Donovan J, Mills N, Smith M, Brindle L, Jacoby A, Peters T, et al. Quality improvement report: Improving design and conduct of randomised trials by embedding them in qualitative research: ProtecT (prostate testing for cancer and treatment) study. Commentary: presenting unbiased information to patients can be difficult. BMJ 2002;325:766–770.
24. Bae JM. Global trends in the use of nationwide big data for solving healthcare problems. J Korean Med Assoc 2014;57:386–390. (Korean).
25. Mohammed K, Nolan MB, Rajjo T, Shah ND, Prokop LJ, Varkey P, et al. Creating a patient-centered health care delivery system: a systematic review of health care quality from the patient perspective. Am J Med Qual 2014;(in press).
26. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy 2005;10 Suppl 1:21–34.
27. Barbour RS. The case for combining qualitative and quantitative approaches in health services research. J Health Serv Res Policy 1999;4:39–43.
28. Morgan DL. Practical strategies for combining qualitative and quantitative methods: applications to health research. Qual Health Res 1998;8:362–376.
29. Collins JA, Fauser BC. Balancing the strengths of systematic and narrative reviews. Hum Reprod Update 2005;11:103–104.
30. Moffatt S, White M, Mackintosh J, Howel D. Using quantitative and qualitative data in health services research - what happens when mixed method findings conflict? [ISRCTN61522618]. BMC Health Serv Res 2006;6:28.
31. Martin M. The need for an overall strategy. J Health Serv Res Policy 2005;10 Suppl:49–50.
32. Flemming K. The synthesis of qualitative research and evidence-based nursing. Evid Based Nurs 2007;10:68–71.
33. Jack SM. Utility of qualitative research findings in evidence-based public health practice. Public Health Nurs 2006;23:277–283.
34. Rodgers M, Sowden A, Petticrew M, Arai L, Roberts H, Britten N, et al. Testing methodological guidance on the conduct of narrative synthesis in systematic reviews: effectiveness of interventions to promote smoke alarm ownership and function. Evaluation 2009;15:47–71.
35. Black N. Why we need qualitative research. J Epidemiol Community Health 1994;48:425–426.
36. Berwick DM. The science of improvement. JAMA 2008;299:1182–1184.
37. Hoddinott P, Pill R. A review of recently published qualitative research in general practice More methodological questions than answers? Fam Pract 1997;14:313–319.
38. Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ 2001;322:1115–1117.
39. Dixon-Woods M, Fitzpatrick R, Roberts K. Including qualitative research in systematic reviews: opportunities and problems. J Eval Clin Pract 2001;7:125–133.
40. Lewin S, Glenton C, Oxman AD. Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ 2009;339:b3496.
41. Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol 2009;9:59.
42. Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 2005;10:45–53.
43. Thorne S, Jensen L, Kearney MH, Noblit G, Sandelowski M. Qualitative metasynthesis: reflections on methodological orientation and ideological agenda. Qual Health Res 2004;14:1342–1365.
44. O’Cathain A, Murphy E, Nicholl J. Why, and how, mixed methods research is undertaken in health services research in England: a mixed methods study. BMC Health Serv Res 2007;7:85.
45. Forbes A, Griffiths P. Methodological strategies for the identification and synthesis of ‘evidence’ to support decision-making in relation to complex healthcare systems and practices. Nurs Inq 2002;9:141–155.
46. Sinuff T, Cook DJ, Giacomini M. How qualitative research can contribute to research in the intensive care unit. J Crit Care 2007;22:104–111.
47. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004;82:581–629.
48. Abu Dabrh AM, Firwana B, Cowl CT, Steinkraus LW, Prokop LJ, Murad MH. Health assessment of commercial drivers: a meta-narrative systematic review. BMJ Open 2014;4e003434.
49. Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 2005;10 Suppl 1:6–20.
50. Flemming K. Synthesis of quantitative and qualitative research: an example using Critical Interpretive Synthesis. J Adv Nurs 2010;66:201–217.
51. Walsh D, Downe S. Meta-synthesis method for qualitative research: a literature review. J Adv Nurs 2005;50:204–211.
52. Leamy M, Bird V, Le Boutillier C, Williams J, Slade M. Conceptual framework for personal recovery in mental health: systematic review and narrative synthesis. Br J Psychiatry 2011;199:445–452.
53. Harden A, Garcia J, Oliver S, Rees R, Shepherd J, Brunton G, et al. Applying systematic review methods to studies of people’s views: an example from public health research. J Epidemiol Community Health 2004;58:794–800.
54. Tricco AC, Tetzlaff J, Moher D. The art and science of knowledge synthesis. J Clin Epidemiol 2011;64:11–20.
55. Bae JM, Park BJ, Ahn YO. Perspectives of clinical epidemiology in Korea. J Korean Med Assoc 2013;56:718–723.
56. Evans D. Database searches for qualitative research. J Med Libr Assoc 2002;90:290–293.
57. Flemming K, Briggs M. Electronic searching to locate qualitative research: evaluation of three strategies. J Adv Nurs 2007;57:95–100.
58. Shaw RL, Booth A, Sutton AJ, Miller T, Smith JA, Young B, et al. Finding qualitative research: an evaluation of search strategies. BMC Med Res Methodol 2004;4:5.
59. Methley AM, Chew-Graham C, Campbell S, Cheraghi-Sohi S. Experiences of UK health-care services for people with Multiple Sclerosis: a systematic narrative review. Health Expect 2014;doi: 10.1111/hex.12228 .
60. Mays N, Pope C. Qualitative research in health care. Assessing quality in qualitative research. BMJ 2000;320:50–52.
61. Dixon-Woods M, Shaw RL, Agarwal S, Smith JA. The problem of appraising qualitative research. Qual Saf Health Care 2004;13:223–225.
62. Reynolds J, Kizito J, Ezumah N, Mangesho P, Allen E, Chandler C. Quality assurance of qualitative research: a review of the discourse. Health Res Policy Syst 2011;9:43.
63. Boeije HR, van Wesel F, Alisic E. Making a difference: towards a method for weighing the evidence in a qualitative synthesis. J Eval Clin Pract 2011;17:657–663.
64. Crossland L, Janamian T, Jackson CL. Key elements of high-quality practice organisation in primary health care: a systematic review. Med J Aust 2014;201:S47–S51.
65. Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, et al. Integrating qualitative research with trials in systematic reviews. BMJ 2004;328:1010–1012.
66. Clark JP. How to peer review a qualitative manuscript. In : Godlee F, Jefferson Tm, eds. Peer review in health sciences 2nd edth ed. London: BMJ Books; 2003. p. 219–235.
67. Daly J, Willis K, Small R, Green J, Welch N, Kealy M, et al. A hierarchy of evidence for assessing qualitative health research. J Clin Epidemiol 2007;60:43–49.
68. Dixon-Woods M, Sutton A, Shaw R, Miller T, Smith J, Young B, et al. Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods. J Health Serv Res Policy 2007;12:42–47.
69. Carroll C, Booth A, Leaviss J, Rick J. “Best fit” framework synthesis: refining the method. BMC Med Res Methodol 2013;13:37.
70. Lemire M, Demers-Payette O, Jefferson-Falardeau J. Dissemination of performance information and continuous improvement: a narrative systematic review. J Health Organ Manag 2013;27:449–478.
71. Jacobson N, Butterill D, Goering P. Development of a framework for knowledge translation: understanding user context. J Health Serv Res Policy 2003;8:94–99.
72. Oliver SR, Rees RW, Clarke-Jones L, Milne R, Oakley AR, Gabbay J, et al. A multidimensional conceptual framework for analysing public involvement in health services research. Health Expect 2008;11:72–84.
73. Eakin JM, Mykhalovskiy E. Reframing the evaluation of qualitative health research: reflections on a review of appraisal guidelines in the health sciences. J Eval Clin Pract 2003;9:187–194.
74. Fone D, Hollinghurst S, Temple M, Round A, Lester N, Weightman A, et al. Systematic review of the use and value of computer simulation modelling in population health and health care delivery. J Public Health Med 2003;25:325–335.
75. QSR International. NVivo 10 for windows. [cited 2014 Sep 3]. Available from: http://www.qsrinternational.com/products_nvivo.aspx.
76. Noble H, Smith J. Qualitative data analysis: a practical example. Evid Based Nurs 2014;17:2–3.
77. Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: meta-narrative reviews. BMC Med 2013;11:20.
78. Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: meta-narrative reviews. J Adv Nurs 2013;69:987–1004.
79. Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Syst Rev 2012;1:28.
80. Sandelowski M, Voils CI, Leeman J, Crandell JL. Mapping the Mixed Methods-Mixed Research Synthesis Terrain. J Mix Methods Res 2012;6:317–331.
81. Crombie IK, Davies HT. What is meta-analysis? [cited 2014 Sep 3] Available from: http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/meta-an.pdf.
82. Ahn HS, Kim HJ. An introduction to systematic review. J Korean Med Assoc 2014;57:49–59. (Korean).
83. Del Pozzo-Magaña BR, Rieder MJ, Lazo-Langner A. Quality of life in children with adverse drug reactions: a narrative and systematic review. Br J Clin Pharmacol 2014;doi: 10.1111/bcp.12423 .
84. Cipriani A, Geddes J. Comparison of systematic and narrative reviews: the example of the atypical antipsychotics. Epidemiol Psichiatr Soc 2003;12:146–153.
85. Icahn School of Medicine at Mount Sinai. EBM skills on the wards. [cited 2014 Sep 3]. Available from: http://libguides.mssm.edu/content.php?pid=417116&sid=3882519.

Appendices

Appendix 1. Summary tables of nomenclatures about methodologies for synthesis of qualitative researches

A01. Abbott A. Conceptions of time and events in social science methods: causal and narrative approaches. Hist Methods 1990;23:140–150. A02. Fairbank L, O'Meara S, Renfrew MJ, Woolridge M, Sowden AJ, Lister-Sharp D. A systematic review to evaluate the effectiveness of interventions to promote the initiation of breastfeeding. Health Technol Assess 2000;4:1–171. A03. Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 2008;8:45. A04. Lucas PJ, Baird J, Arai L, Law C, Roberts HM. Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews. BMC Med Res Methodol 2007;7:4. A05. Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, et al. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol 2006;6:35. A06. Flemming K. Synthesis of quantitative and qualitative research: an example using Critical Interpretive Synthesis. J Adv Nurs 2010;66:201–217. A07. Oliver SR, Rees RW, Clarke-Jones L, Milne R, Oakley AR, Gabbay J, et al. A multidimensional conceptual framework for analysing public involvement in health services research. Health Expect 2008;11:72–84. A08. Pawson R. Evidence-based policy: the promise of ‘Realist Synthesis’. Evaluation 2002;8:340–358. A09. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy 2005;10 Suppl 1:21–34. A10. Flemming K. Synthesis of qualitative research and evidence-based nursing. Br J Nurs 2007;16:616–620. A11. Timulak L. Meta-analysis of qualitative studies: a tool for reviewing qualitative research findings in psychotherapy. Psychother Res 2009;19:591–600. A12. Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol 2009;9:59. A13. Roundtree AK, Kallen MA, Lopez-Olivo MA, Kimmel B, Skidmore B, Ortiz Z, et al. Poor reporting of search strategy and conflict of interest in over 250 narrative and systematic reviews of two biologic agents in arthritis: a systematic review. J Clin Epidemiol 2009;62:128–137. A14. Haggerty TR. Unravelling patterns of multiple conjunctural causation in comparative research: Ragin’s qualitative comparative method. J Comp Phys Educ Sports 1992;14:19–27. A15. Cress DM, Snow DA. The Outcomes of homeless mobilization: the influence of organization, disruption, political mediation, and framing. Am J Sociol 2000;105:1063–1104. A16. Thorne S, Jensen L, Kearney MH, Noblit G, Sandelowski M. Qualitative metasynthesis: reflections on methodological orientation and ideological agenda. Qual Health Res 2004;14:1342–1365. A17. Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol 2009;9:59. A18. Sandelowski M, Docherty S, Emden C. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol 2009;9:59. A19. Walsh D, Downe S. Meta-synthesis method for qualitative research: a literature review. J Adv Nurs 2005;50:204–211. A20. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004;82:581–629. A21. Abu Dabrh AM, Firwana B, Cowl CT, Steinkraus LW, Prokop LJ, Murad MH. Health assessment of commercial drivers: a meta-narrative systematic review. BMJ Open 2014;4e003434. A22. Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M, et al. Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med 2003;56:671–684. A23. Campbell R, Pound P, Morgan M, Daker-White G, Britten N, Pill R, et al. Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research. Health Technol Assess 2011;15:1–164. A24. Thorne S, Paterson B, Acorn S, Canam C, Joachim G, Jillings C. Chronic illness experience: insights from a metastudy. Qual Health Res 2002;12:437–452. A25. Finfgeld DL. Courage as a process of pushing beyond the struggle. Qual Health Res 1999;9:803–814. A26. Sandelowski M, Voils CI, Barroso J. Defining and designing mixed research synthesis studies. Res Sch 2006;13(1):29. A27. Flemming K. The knowledge base for evidence-based nursing: a role for mixed methods research? ANS Adv Nurs Sci 2007;30:41–51. A28. Sandelowski M, Voils CI, Leeman J, Crandell JL. Mapping the mixed methods-mixed research synthesis terrain. J Mix Methods Res 2012;6:317–331. A29. Seymour J. Combined qualitative and quantitative research designs. Curr Opin Support Palliat Care 2012;6:514–524.

Appendix 2. Some articles related to qualitative researches about health care services

B01. Pound P, Britten N, Morgan M, Yardley L, Pope C, Daker-White G, et al. Resisting medicines: a synthesis of qualitative studies of medicine taking. Soc Sci Med 2005;61:133–155. B02. Teasdale E, Santer M, Geraghty AW, Little P, Yardley L. Public perceptions of non-pharmaceutical interventions for reducing transmission of respiratory infection: systematic review and synthesis of qualitative studies. BMC Public Health 2014;14:589. B03. Ruggiero JS, Redeker NS. Effects of napping on sleepiness and sleep-related performance deficits in night-shift workers: a systematic review. Biol Res Nurs 2014;16:134–142. B04. Mousazadeh Y, Jannati A, Jabbari Beiramy H, Asgharijafarabadi M, Ebadi A. Advantages and disadvantages of different methods of hospitals’ downsizing: a narrative systematic review. Health Promot Perspect 2013;3:276–287. B05. Boath E, Bradley E, Henshaw C. The prevention of postnatal depression: a narrative systematic review. J Psychosom Obstet Gynaecol 2005;26:185–192. B06. Flemming K. The use of morphine to treat cancer-related pain: a synthesis of quantitative and qualitative research. J Pain Symptom Manage 2010;39:139–154. B07. Santer M, Ring N, Yardley L, Geraghty AW, Wyke S. Treatment non-adherence in pediatric long-term medical conditions: systematic review and synthesis of qualitative studies of caregivers' views. BMC Pediatr 2014;14:63. B08. Davidson MB. The effectiveness of nurse- and pharmacist-directed care in diabetes disease management: a narrative review. Curr Diabetes Rev 2007;3:280–286. B09. Davidson MB. Detailed treatment algorithms for effective nurse- and pharmacist-directed diabetes care: a personal approach. Diabetes Educ 2009;35:61–71. B10. Jayde V, White K, Blomfield P. Symptoms and diagnostic delay in ovarian cancer: a summary of the literature. Contemp Nurse 2009;34:55–65. B11. Leamy M, Bird V, Le Boutillier C, Williams J, Slade M. Conceptual framework for personal recovery in mental health: systematic review and narrative synthesis. Br J Psychiatry 2011;199:445–452. B12. Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, et al. Integrating qualitative research with trials in systematic reviews. BMJ 2004;328:1010–1012. B13. Harden A, Brunton G, Fletcher A, Oakley A. Teenage pregnancy and social disadvantage: systematic review integrating controlled trials and qualitative studies. BMJ 2009;339:b4254. B14. Bonell C, Harden A, Wells H, Jamal F, Fletcher A, Petticrew M, et al. Protocol for a systematic review of the effects of schools and school-environment interventions on health: evidence mapping and syntheses. BMC Public Health 2011;11:453. B15. Jamal F, Fletcher A, Harden A, Wells H, Thomas J, Bonell C. The school environment and student health: a systematic review and meta-ethnography of qualitative research. BMC Public Health 2013;13:798. B16. Liu J, Davidson E, Bhopal R, White M, Johnson M, Netto G, et al. Adapting health promotion interventions to meet the needs of ethnic minority groups: mixed-methods evidence synthesis. Health Technol Assess 2012;16:1–469. B17. Neville LM, O'Hara B, Milat A. Computer-tailored physical activity behavior change interventions targeting adults: a systematic review. Int J Behav Nutr Phys Act 2009;6:30. B18. Neville LM, Milat AJ, O'Hara B. Computer-tailored weight reduction interventions targeting adults: a narrative systematic review. Health Promot J Austr 2009;20:48–57. B19. Neville LM, O'Hara B, Milat AJ. Computer-tailored dietary behaviour change interventions: a systematic review. Health Educ Res 2009;24:699–720. B20. Thomas R, Perera R. School-based programmes for preventing smoking. Cochrane Database Syst Rev 2006;(3)CD001293. B21. Mills E, Jadad AR, Ross C, Wilson K. Systematic review of qualitative studies exploring parental beliefs and attitudes toward childhood vaccination identifies common barriers to vaccination. J Clin Epidemiol 2005;58:1081–1088. B22. Contandriopoulos D, Lemire M, Denis JL, Tremblay E. Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature. Milbank Q 2010;88:444–483. B23. Noyes J, Popay J. Directly observed therapy and tuberculosis: how can a systematic review of qualitative research contribute to improving services? A qualitative meta-synthesis. J Adv Nurs 2007;57:227–243.

Article information Continued

©2014, Korean Society of Epidemiology

This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

To Nomenclatures [References]
Emphasize qualitative synthesis Narrative summary [A01, A02]
Thematic synthesis [A03]
Textual narrative synthesis [A04]
Critical interpretive synthesis [A05, A06]
Framework synthesis [A07]
Realist synthesis [A08, A09]
Contrast quantitative systematic review Qualitative systematic review [A10,A11]
Synthesis of qualitative research [A12]
Narrative systematic review [A13]
Qualitative comparative analysis [A14, A15]
Qualitative meta-synthesis [A16]
Qualitative meta-summary [A17]
Contrast meta-analysis Meta-synthesis [A18, A19]
Meta-narrative review [A20, A21]
Meta-ethnography [A22, A23]
Meta-study [A24]
Meta-interpretation [A25]
Include quantitative & qualitative researches Mixed methods research [A26-A29]
Ecological triangulation [A01]
Domain [References]
Preventive medicine Pharmacoepidemiology [B01, B02]
Occupation [B03]
Hospital management [B04]
Clinical medicine Obstetrics [B05]
Hospice [B06]
Patient-doctor relationship [B07]
Nurse Diabetic care [B08, B09]
Insight into cancer [B10]
Health promotion Mental health [B11]
School health [B12-B15]
Health program [B16-B20]
Vaccine program [B21]
Health policy Policy decision [B22]
Control plan of tuberculosis [B23]