Warning: fopen(/home/virtual/epih/journal/upload/ip_log/ip_log_2023-03.txt): failed to open stream: Permission denied in /home/virtual/lib/view_data.php on line 83 Warning: fwrite() expects parameter 1 to be resource, boolean given in /home/virtual/lib/view_data.php on line 84 A suggestion for quality assessment in systematic reviews of observational studies in nutritional epidemiology
Skip Navigation
Skip to contents

Epidemiol Health : Epidemiology and Health

OPEN ACCESS
SEARCH
Search
Advanced Search
mobile menu button

Articles

Page Path
HOME > Epidemiol Health > Volume 38; 2016 > Article
Methods
 A suggestion for quality assessment in systematic reviews of observational studies in nutritional epidemiology
Jong-Myon Baeorcid
Epidemiol Health 2016;38:e2016014.
DOI: https://doi.org/10.4178/epih.e2016014
Published online: April 26, 2016
  • 16,841 Views
  • 233 Download
  • 48 Crossref
  • 52 Scopus

Department of Preventive Medicine, Jeju National University School of Medicine, Jeju, Korea

Correspondence: Jong-Myon Bae  Department of Preventive Medicine, Jeju National University School of Medicine, 102 Jejudaehak-ro, Jeju 63243, Korea  Tel: +82-64-755-5567, Fax: +82-64-725-2593, E-mail: jmbae@cheju.ac.kr
• Received: March 27, 2016   • Accepted: April 25, 2016

©2016, Korean Society of Epidemiology

This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Full Article

Download PDF Download PDF

  • OBJECTIVES:
    It is important to control the quality level of the observational studies in conducting meta-analyses. The Newcastle-Ottawa Scale (NOS) is a representative tool used for this purpose. We investigated the relationship between high-quality (HQ) defined using NOS and the results of subgroup analysis according to study design.
  • METHODS:
    We selected systematic review studies with meta-analysis which performed a quality evaluation on observational studies of diet and cancer by NOS. HQ determinations and the distribution of study designs were examined. Subgroup analyses according to quality level as defined by the NOS were also extracted. Equivalence was evaluated based on the summary effect size (sES) and 95% confidence intervals computed in the subgroup analysis.
  • RESULTS:
    The meta-analysis results of the HQ and cohort groups were identical. The overall sES, which was obtained by combining the sES when equivalence was observed between the cohort and case-control groups, also showed equivalence.
  • CONCLUSIONS:
    The results of this study suggest that it is more reasonable to control for quality level by performing subgroup analysis according to study design rather than by using HQ based on the NOS quality assessment tool.
  • Keywords: Meta-analysis; Quality evaluation; Research design; Quality control
Systematic reviews (SRs) with meta-analyses are a useful methodology to assess inconsistent epidemiological study findings [1]. However, controversy arose in the early 1990s regarding meta-analysis of the results of observational studies rather than randomized-controlled trials (RCT) [2-5]. Researchers have concluded that meta-analyses are not beneficial in cases where there are differences in the quality of the included studies [6,7]; in addition, the necessity for a scoring system to assess the quality levels of selected publications has also been emphasized [8,9].
The Newcastle-Ottawa Scale (NOS) is a representative tool developed for meta-analysis of observational studies [10]. The NOS is suitable for SR due to its easy application [11], and has been widely used due to recommendations from the Cochrane Collaboration [12]. Recently, however, the validity and reliability of the NOS has been questioned [13-15]. It was claimed to be additionally supplemented because the guidelines to be applied to each evaluation item were unclear [13,14]. Furthermore, Stang [13] suggested serious errors in the SRs that assessed the quality of observational studies using the NOS.
Since it is impossible to apply RCTs to nutritional epidemiology studies that investigate the relationships between daily food intake and the incidence of various cancers, only SRs for observational studies are available. Thus, Yang et al. [16] included ‘data analysis that used an energy-adjusted residual or nutrient-density model’ to the existing nine items of the NOS as an adjustment item to evaluate the quality of articles included in meta-analyses, which was then reflected to subgroup analyses. However, in their quality assessment [16], all three cohort studies were determined to be of high-quality (HQ), whereas only two of eight case-control studies were determined to be HQ. The reason for the difference in HQ determination between study designs appears to be attributable to the characteristics of the NOS, which was designed to give a higher score to cohort studies that are more scientifically persuasive [15].
Based on these observations, then, is it possible to replace the NOS tool with evaluating the quality level of observational studies in meta-analysis according to study design? It would be more reasonable and efficient if quality levels could be controlled according to study design rather than spending labor and time in applying the NOS. In other words, if there were consistency between HQ classification and study design, subgroup analysis by study design would be sufficient, instead of quality assessment using the NOS. Thus, the present study investigated the equivalence of meta-analysis results between HQ classification by NOS and cohort studies in same SR of observational studies on nutritional epidemiology.
Subject article searches and selection criteria
The articles selected in the present study were those included in the analytical epidemiology SRs that investigated the relationships between daily food intake and the incidence of various cancers. In addition, they should use the NOS for quality evaluation and show the results of subgroup analysis by study design. The PubMed literature database (www.ncbi.nlm.nih.gov/pubmed) was used to search for articles using search terms corresponding to foods – diet, food, fruit, vegetable, or meat – and SR or meta-analysis for cancer incidence in the article title, abstract and keyword among lists published between January 2000 and October 2015.
After obtaining a list of articles, the following exclusion criteria were applied: (1) study hypotheses that did not assess the association between diet and cancer, (2) RCT rather than observational study, (3) SR without conducting meta-analysis, (4) SR without quality assessment, (5) SR having quality assessment by a tool other than the NOS, and (6) despite assessing quality using the NOS during the SR, subgroup analysis results were not included.
Collection of related information and statistical analysis
Inclusion of articles in the analysis of quality assessment was based on statements in the methods section of each article, from which the type of tool used for evaluation and HQ decision criteria were identified. In order to determine the distribution of HQ subjects in each SR article according to study design, the articles were divided into cohort and case-control studies, and the statistical differences between the selected fractions (%) were examined using chi-squared tests.
In addition, the summary effect size (sES) and 95% confidence intervals (CI) estimated from the HQ group and cohort study group in subgroup analysis were extracted. The equivalence between the results of the HQ group and cohort study group was determined based on consistent direction of sES values to the null (=1) as well as consistent statistically significance based on the 95% CI.
Figure 1 shows the process of selecting final articles included in the analysis. After remove duplicate publications from the list obtained by searching formula, resulting in 371 articles identified for further review. Among them, (1) 256 articles were excluded because their study hypotheses did not assess the relationship between diet and cancer, (2) 14 articles were RCT, and (3) six articles performed SR without meta-analysis. Of the remaining 95 papers, 14 SR conducted quality assessments using the NOS as stated in their methods sections and also performed quality assessment in subgroup analysis [16-29]. Of those 14 articles, 4 [16-19] used modified NOS that included energy intake. Except for the study by Liu et al. [21], all others used seven points or more as the criterion for HQ. Table 1 presents results of HQ assessment as evaluated by the NOS according to cohort or case-control studies. Of the papers selected as subjects of meta-analysis in the 14 papers included in the current study, 81 (91%) of 89 cohort and 72 (34%) of 209 case-control studies were considered HQ, a statistically significant difference (p<0.01).
Table 2 summarizes the sES and 95% CI of the HQ and cohorts group by food item. The paper by Wang et al. [22] was excluded because it considered only cohort studies. Results of 19 datasets by food item from the 13 papers were summarized by food item; of these, 15 datasets showed the same magnitude and same statistical significance. The remaining four datasets lost statistical significance as CI became wide, although their magnitudes were consistent.
For the 15 datasets that showed equivalence between the HQ and cohort groups in Table 2, Table 3 was constructed to compare the results of subgroup analysis in the cohort and case-control study groups with overall sES by combining both results. Eight datasets showed equivalence in sES of the cohort and case-control study groups, and their overall sES also showed equivalence. In the seven datasets with non-equivalence, the results of case-control studies with higher article numbers greatly influenced the overall sES.
In summary, the HQ and cohort groups had similar meta-analysis results because most cohort studies were classified as HQ based on quality assessment using the NOS. In other words, for SR of observational studies in nutritional epidemiology, quality assessment by NOS is decisively dependent on study methodology. Thus, subgroup analysis by study design may be more valid than the conducting quality assessment method based on the NOS, until a new quality assessment tool is developed in consideration of the characteristics of nutritional epidemiology.
As shown in Table 2, four datasets had no equivalence between the HQ and cohort groups. However, the width of CI changed while the magnitude of sES remained consistent, which resulted in their non-equivalence. Since the width of CI changes according to the number of subject articles included in the meta-analysis, the non-equivalence results was attributed to the difference in the number of subject articles, rather than differences between the HQ and cohort groups.
The current study also assessed whether non-equivalence could be controlled based on assessing results according to study design rather than the NOS. The results showed that when sES between the cohort and case-control groups showed equivalence, the overall sES for the combination of both group also showed equivalence. In addition, since the CI of the sES between the cohort and case-control groups in the paper by Hu et al. [29] was wide, there was no statistical significance; however, the CI of the overall sES of both groups combined was narrower, resulting in statistical significance. Thus, these findings suggest that it is reasonable to combine both groups when the sES calculated for each of the cohort and case-control groups shows equivalence. However, non-equivalence in the sES between these groups indicates that the overall sES should be interpreted carefully, as the value is dependent on the number of articles.
Colditz et al. [30] proposed that study design, quality of implementation, exposure, and covariates contribute to heterogeneity in SR. Since meta-analysis in nutritional epidemiology uses both cohort and case-control studies, there is heterogeneity by study design [31,32]. Therefore, while NOS evaluation within a group with the same study design would be meaningful, subgroup analysis only with the NOS while ignoring differences in study design may compromise the results.
In the present study, only 16.8% (=16/95) of papers (Figure 1) applied quality assessment results to subgroup analysis in their SR of nutritional epidemiology studies, even after including two papers that used their own quality assessment criteria instead of the NOS [33,34]. In addition, the first article to apply the NOS was published in 2006 [34], although the search included publication dates from January 2000. The primary reason for the lack of quality assessment in SR of nutritional epidemiology studies was the lack of valid assessment tools [33,34]. Although the NOS has been used since its development, the present study found that HQ decision in the NOS is fully dependent on study design. Thus, subgroup analysis should be performed separately for each study design until a quality assessment tool specific for nutritional epidemiology is developed.
The current study has several limitations. First, only the PubMed literature database was searched to investigate the level of quality assessment and a narrow set of terms related to diet items were applied to the search formula. In particular, items such as dietary fat, fiber, and vitamins indirectly assessed through diet measurement were excluded. Thus, the proportion as 16.8% (=16/95) (Figure 1) of papers that had subgroup analysis after quality assessment seems to be over-estimated. The results of the present study emphasize the importance of quality level control through subgroup analysis according to study design even without application of the NOS. Secondly, the equivalence of sES between the cohort and case-control study groups was comparatively analyzed based on study results that applied NOS. Thus, further investigations are necessary to determine if quality level can be controlled for in meta-analysis of nutritional epidemiology based on subgroup analysis results according to study design and application methods.
In conclusion, it is advisable to conduct subgroup analysis by study design for quality assessment of meta-analysis in nutritional epidemiology rather than applying the NOS assessment tool, and interpretation of overall sES should rely on the equivalence of sES by study design. These suggestions are consistent with the statement from Greenland [35] in 1994:
“Just as a diet and health study needs to examine the effects of each major dietary factor, quality scoring should be replaced by direct regression or stratification on objective quality-related study characteristics, such as study design (cohort, case-control, etc.), sources of data (direct interviews, mailed questionnaire, medical records, etc.), and sources of subjects (registry, hospital, etc.).”
I thank Eun Hee Kim at the Jeju National University School of Medicine for searching and producing the list of publications for review.

The author has no conflicts of interest to declare for this study.

Supplementary material (Korean version) is available at http://www.e-epih.org/.
epih-38-e2016014-supplementary.pdf
Figure 1.
Flow chart of article selection.
epih-38-e2016014f1.gif
Table 1.
Quality ratings of selected papers by study design
Author YP RN Criterion of high quality Cohort studies
 Case-control studies
Selected [A] High quality [B] [B]/[A] (%) Selected [C] High quality [D] [D]/[C] (%)
Yang et al. 2011 [16] 7+/10 3 3 100 8 2 25
Wu et al. 2013 [17] 7+/10 11 10 91 24 11 46
Wu et al. 2013 [18] 7+/10 6 6 100 16 5 31
Zhu et al. 2013 [19] 7+/10 12 10 83 30 13 43
Choi et al. 2013 [20] 7+/9 4 4 100 23 5 22
Liu et al. 2014 [21] 8+/9 2 2 100 31 4 13
Wang et al. 2014 [22] 7+/9 17 14 82 0 0 -
Yang et al. 2014 [23] 7+/9 10 9 90 9 2 22
Song et al. 2014 [24] 7+/9 4 4 100 14 8 57
Xin et al. 2015 [25] 7+/9 5 4 80 11 3 27
Wang et al. 2015 [26] 7+/9 6 6 100 13 7 54
Wu et al. 2015 [27] 7+/9 1 1 100 21 10 48
Li et al. 2015 [28] 7+/9 4 4 100 5 1 20
Hu et al. 2015 [29] 7+/9 4 4 100 4 1 25
Total 89 81 (91) 209 72 (34)

YP, year of publication; RN, reference number.

Table 2.
Summary effect size (sES) and 95% confidence intervals (CI) of high quality groups on the basis of Newcastle-Ottawa Scale for quality assessment of cohort studies
Author [RN] Food items High quality
 Cohort studies
 Eq
sES (95% CI) NP sES (95% CI) NP
Yang et al. [16] Soy 0.70 (0.45, 0.99) 5 0.92 (0.85, 0.98) 3 Yes
Wu et al. [17] Cruciferous vegetable 0.88 (0.80, 0.97) 21 0.93 (0.87, 1.00) 11 Yes
Wu et al. [18] Cruciferous vegetable 0.84 (0.76, 0.93) 11 0.89 (0.77, 1.02) 6 No
Zhu et al. [19] Red meat 1.30 (1.05, 1.61) 9 1.02 (0.90, 1.17) 4 No
Processed meat 1.26 (1.10, 1.46) 17 1.18 (1.00, 1.38) 9 Yes
Choi et al. [20] Red meat 1.60 (1.20, 2.13) 8 1.26 (1.00, 1.59) 4 Yes
Processed meat 1.20 (0.88, 1.62) 6 1.25 (0.83, 1.86) 3 Yes
Liu et al. [21] Vegetable 0.97 (0.78, 1.21) 3 0.91 (0.68, 1.21) 2 Yes
Fruit 0.96 (0.69, 1.33) 2 0.81 (0.58, 1.12) 1 Yes
Soy 1.12 (0.68, 1.84) 3 1.46 (1.07, 1.98) 2 No
Yang et al. [23] Vegetable 0.68 (0.59, 0.78) 9 0.66 (0.51,0.86) 9 Yes
Fruit 1.03 (0.87, 1.20) 7 1.04 (0.91, 1.20) 6 Yes
Song et al. [24] Red meat 1.27 (1.09, 1.48) 17 1.00 (0.83, 1.20) 8 No
Xin et al. [25] Vegetable oil 0.99 (0.87, 1.13) 7 0.94 (0.84, 1.06) 5 Yes
Wang et al. [26] Cruciferous vegetable 0.61 (0.44, 0.86) 6 0.76 (0.62, 0.93) 6 Yes
Wu et al. [27] Vegetable 0.78 (0.54, 1.14) 7 1.00 (0.52, 1.92) 1 Yes
Soy 0.87 (0.60, 1.26) 5 1.09 (0.60, 1.98) 1 Yes
Li et al. [28] Cruciferous vegetable 0.78 (0.55, 1.01) 5 0.87 (0.67, 1.05) 4 Yes
Hu et al. [29] Cruciferous vegetable 0.89 (0.77, 1.02) 5 0.92 (0.80, 1.07) 4 Yes

RN, reference number; Eq, equivalent direction and statistical significance of sES between high quality group and cohort studies (yes or no); NP, number of papers.

Table 3.
Summary effect size (sES) and 95% confidence intervals (CI) on the basis of overall and case-control studies about papers showing equivalence of direction and statistical significance between cohort and high quality group in Table 2
Author [RN] Food items Cohort studies
 Case-control studies
 Eq Overall
sES (95% CI) NP sES (95% CI) NP sES (95% CI) NP
Yang et al. [16] Soy 0.92 (0.85, 0.98) 3 0.72 (0.56, 0.92) 8 Yes 0.77 (0.65, 0.92) 11
Wu et al. [17] Cruciferous vegetable 0.93 (0.87, 1.00) 11 0.74 (0.65, 0.84) 23 Yes 0.82 (0.75, 0.90) 35
Zhu et al. [19] Processed meat 1.18 (1.00, 1.38) 9 1.64 (1.47, 1.83) 17 Yes 1.45 (1.26, 1.65) 26
Choi et al. [20] Red meat 1.26 (1.00, 1.59) 4 1.44 (1.16, 1.80) 18 Yes 1.38 (1.17, 1.64) 22
Processed meat 1.25 (0.83, 1.86) 3 1.36 (1.07, 1.74) 15 No 1.32 (1.08, 1.62) 18
Liu et al. [21] Vegetable 0.91 (0.68, 1.21) 2 0.67 (0.42, 1.06) 7 Yes 0.72 (0.51,1.02) 9
Fruit 0.81 (0.58, 1.12) 1 0.63 (0.42, 0.94) 6 No 0.66 (0.47, 0.91) 7
Yang et al. [23] Vegetable 0.66 (0.51,0.86) 9 0.76 (0.48, 1.20) 8 No 0.70 (0.56, 0.87) 17
Fruit 1.04 (0.91, 1.23) 6 0.78 (0.61,0.98) 6 No 0.93 (0.80, 1.08) 12
Xin et al. [25] Vegetable oil 0.94 (0.84, 1.06) 5 0.86 (0.72, 1.02) 11 Yes 0.87 (0.77, 0.99) 16
Wang et al. [26] Citrus fruit 0.76 (0.62, 0.93) 6 0.54 (0.41,0.72) 13 Yes 0.63 (0.52, 0.75) 19
Wu et al. [27] Vegetable 1.00 (0.52, 1.92) 1 0.76 (0.60, 0.96) 12 No 0.77 (0.62, 0.96) 13
Soy 1.09 (0.60, 1.98) 1 0.66 (0.48, 0.92) 14 No 0.68 (0.50, 0.93) 15
Li et al. [28] Cruciferous vegetable 0.87 (0.67, 1.06) 4 0.72 (0.55, 0.89) 5 No 0.78 (0.64, 0.91) 9
Hu et al. [29] Cruciferous vegetable 0.92 (0.80, 1.07) 4 0.87 (0.73, 1.03) 4 Yes 0.89 (0.81,0.99) 8

RN, reference number; Eq, equivalent direction and statistical significance as sES between cohort and case-control studies (yes or no); NP, number of papers.

  • 1. Blettner M, Sauerbrei W, Schlehofer B, Scheuchenpflug T, Friedenreich C. Traditional reviews, meta-analyses and pooled analyses in epidemiology. Int J Epidemiol 1999;28:1-9.ArticlePubMed
  • 2. Shapiro S. Meta-analysis/Shmeta-analysis. Am J Epidemiol 1994;140:771-778.ArticlePubMed
  • 3. Greenland S. Can meta-analysis be salvaged? Am J Epidemiol 1994;140:783-787.ArticlePubMed
  • 4. Petitti DB. Of babies and bathwater. Am J Epidemiol 1994;140:779-782.ArticlePubMed
  • 5. Shapiro S. Is there is or is there ain’t no baby? Dr. Shapiro replies to Drs. Petitti and Greenland. Am J Epidemiol 1994;140:788-791.Article
  • 6. Egger M, Schneider M, Davey Smith G. Spurious precision? Metaanalysis of observational studies. BMJ 1998;316:140-144.ArticlePubMedPMC
  • 7. Chalmers TC. Problems induced by meta-analyses. Stat Med 1991;10:971-979.ArticlePubMed
  • 8. Khoury MJ. Commentary: epidemiology and the continuum from genetic research to genetic testing. Am J Epidemiol 2002;156:297-299.ArticlePubMed
  • 9. Margetts BM, Thompson RL, Key T, Duffy S, Nelson M, Bingham S, et al. Development of a scoring system to judge the scientific quality of information from case-control and cohort studies of nutrition and disease. Nutr Cancer 1995;24:231-239.ArticlePubMed
  • 10. Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. [cited 2016 Apr 25]. Available from: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.
  • 11. Deeks JJ, Dinnes J, D’Amico R, Sowden AJ, Sakarovitch C, Song F, et al. Evaluating non-randomised intervention studies. Health Technol Assess 2003;7:iii-x 1-173.Article
  • 12. Higgins JP, Green S; Cochrane Collaboration. Cochrane handbook for systematic reviews of interventions. Chichester: Wiley-Blackwell; 2008. p 418.
  • 13. Stang A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol 2010;25:603-605.ArticlePubMed
  • 14. Hartling L, Milne A, Hamm MP, Vandermeer B, Ansari M, Tsertsvadze A, et al. Testing the Newcastle Ottawa Scale showed low reliability between individual reviewers. J Clin Epidemiol 2013;66:982-993.ArticlePubMed
  • 15. Lo CK, Mertz D, Loeb M. Newcastle-Ottawa Scale: comparing reviewers’to authors’ assessments. BMC Med Res Methodol 2014;14:45.ArticlePubMedPMCPDF
  • 16. Yang WS, Va P, Wong MY, Zhang HL, Xiang YB. Soy intake is associated with lower lung cancer risk: results from a meta-analysis of epidemiologic studies. Am J Clin Nutr 2011;94:1575-1583.ArticlePubMedPMC
  • 17. Wu QJ, Yang Y, Vogtmann E, Wang J, Han LH, Li HL, et al. Cruciferous vegetablesintake and the risk of colorectal cancer: a meta-analysis of observational studies. Ann Oncol 2013;24:1079-1087.ArticlePubMedPMC
  • 18. Wu QJ, Yang Y, Wang J, Han LH, Xiang YB. Cruciferous vegetable consumption and gastric cancer risk: a meta-analysis of epidemiological studies. Cancer Sci 2013;104:1067-1073.ArticlePubMedPMC
  • 19. Zhu H, Yang X, Zhang C, Zhu C, Tao G, Zhao L, et al. Red and processed meat intake is associated with higher gastric cancer risk: a meta-analysis of epidemiological observational studies. PLoS One 2013;8:e70955.ArticlePubMedPMC
  • 20. Choi Y, Song S, Song Y, Lee JE. Consumption of red and processed meat and esophageal cancer risk: meta-analysis. World J Gastroenterol 2013;19:1020-1029.ArticlePubMedPMC
  • 21. Liu XO, Huang YB, Gao Y, Chen C, Yan Y, Dai HJ, et al. Association between dietary factors and breast cancer risk among Chinese females: systematic review and meta-analysis. Asian Pac J Cancer Prev 2014;15:1291-1298.ArticlePubMed
  • 22. Wang Q, Chen Y, Wang X, Gong G, Li G, Li C. Consumption of fruit, but not vegetables, may reduce risk of gastric cancer: results from a meta-analysis of cohort studies. Eur J Cancer 2014;50:1498-1509.ArticlePubMed
  • 23. Yang Y, Zhang D, Feng N, Chen G, Liu J, Chen G, et al. Increased intake of vegetables, but not fruit, reduces risk for hepatocellular carcinoma: a meta-analysis. Gastroenterology 2014;147:1031-1042.ArticlePubMed
  • 24. Song P, Lu M, Yin Q, Wu L, Zhang D, Fu B, et al. Red meat consumption and stomach cancer risk: a meta-analysis. J Cancer Res Clin Oncol 2014;140:979-992.ArticlePubMed
  • 25. Xin Y, Li XY, Sun SR, Wang LX, Huang T. Vegetable oil intake and breast cancer risk: a meta-analysis. Asian Pac J Cancer Prev 2015;16:5125-5135.ArticlePubMed
  • 26. Wang A, Zhu C, Fu L, Wan X, Yang X, Zhang H, et al. Citrus fruit intake substantially reduces the risk of esophageal cancer: a metaanalysis of epidemiologic studies. Medicine (Baltimore) 2015;94:e1390.ArticlePubMedPMC
  • 27. Wu YC, Zheng D, Sun JJ, Zou ZK, Ma ZL. Meta-analysis of studies on breast cancer risk and diet in Chinese women. Int J Clin Exp Med 2015;8:73-85.PubMedPMC
  • 28. Li LY, Luo Y, Lu MD, Xu XW, Lin HD, Zheng ZQ. Cruciferous vegetable consumption and the risk of pancreatic cancer: a meta-analysis. World J Surg Oncol 2015;13:44.ArticlePubMedPMC
  • 29. Hu J, Hu Y, Hu Y, Zheng S. Intake of cruciferous vegetables is associated with reduced risk of ovarian cancer: a meta-analysis. Asia Pac J Clin Nutr 2015;24:101-109.PubMed
  • 30. Colditz GA, Burdick E, Mosteller F. Heterogeneity in meta-analysis of data from epidemiologic studies: a commentary. Am J Epidemiol 1995;142:371-382.ArticlePubMed
  • 31. Gandini S, Merzenich H, Robertson C, Boyle P. Meta-analysis of studies on breast cancer risk and diet: the role of fruit and vegetable consumption and the intake of associated micronutrients. Eur J Cancer 2000;36:636-646.ArticlePubMed
  • 32. Xu X, Yu E, Liu L, Zhang W, Wei X, Gao X, et al. Dietary intake of vitamins A, C, and E and the risk of colorectal adenoma: a meta-analysis of observational studies. Eur J Cancer Prev 2013;22:529-539.ArticlePubMed
  • 33. Bandera EV, Kushi LH, Moore DF, Gifkins DM, McCullough ML. Fruits and vegetables and endometrial cancer risk: a systematic literature review and meta-analysis. Nutr Cancer 2007;58:6-21.ArticlePubMed
  • 34. Pavia M, Pileggi C, Nobile CG, Angelillo IF. Association between fruit and vegetable consumption and oral cancer: a meta-analysis of observational studies. Am J Clin Nutr 2006;83:1126-1134.ArticlePubMed
  • 35. Greenland S. Invited commentary: a critical look at some popular meta-analytic methods. Am J Epidemiol 1994;140:290-296.ArticlePubMed

Figure & Data

References

    Citations

    Citations to this article as recorded by  
    • The association between vision impairment and cognitive outcomes in older adults: a systematic review and meta-analysis
      Gui-Ying Cao, Zi-Shuo Chen, Shan-Shan Yao, Kaipeng Wang, Zi-Ting Huang, He-Xuan Su, Yan Luo, Carson M. De Fries, Yong-Hua Hu, Beibei Xu
      Aging & Mental Health.2023; 27(2): 350.     CrossRef
    • Dental age estimation using cone-beam computed tomography: A systematic review and meta-analysis
      Faezeh Yousefi, Younes Mohammadi, Mehrnaz Ahmadvand, Parnian Razaghi
      Imaging Science in Dentistry.2023;[Epub]     CrossRef
    • Anger and substance abuse: a systematic review and meta-analysis
      Helen V. Laitano, Amanda Ely, Anne O. Sordi, Felipe B. Schuch, Flavio Pechansky, Thiago Hartmann, Juliana B. Hilgert, Eliana M. Wendland, Lisia Von Dimen, Juliana N. Scherer, Alessandra Mendes Calixto, Joana C.M. Narvaez, Felipe Ornell, Félix H.P. Kessler
      Brazilian Journal of Psychiatry.2022; 44(1): 103.     CrossRef
    • The Prognostic Value of Human Papilloma Virus Infection in Oral Cavity Squamous Cell Carcinoma: A Meta‐Analysis
      Stephanus Christianto, Kar Yan Li, Ting Hsiang Huang, Yu‐Xiong Su
      The Laryngoscope.2022; 132(9): 1760.     CrossRef
    • The impact of diabetes mellitus on the pharmacokinetics of rifampicin among tuberculosis patients: A systematic review and meta-analysis study
      Amera Sh Metwally, Sawsan M.A. El-Sheikh, Azza A.A. Galal
      Diabetes & Metabolic Syndrome: Clinical Research & Reviews.2022; 16(2): 102410.     CrossRef
    • Effect of perinatal depression on birth and infant health outcomes: a systematic review and meta-analysis of observational studies from Africa
      Abel Fekadu Dadi, Temesgen Yihunie Akalu, Haileab Fekadu Wolde, Adhanom Gebreegziabher Baraki
      Archives of Public Health.2022;[Epub]     CrossRef
    • Selenium Status in Patients with Chronic Liver Disease: A Systematic Review and Meta-Analysis
      Yaduan Lin, Fanchen He, Shaoyan Lian, Binbin Xie, Ting Liu, Jiang He, Chaoqun Liu
      Nutrients.2022; 14(5): 952.     CrossRef
    • Impact of diabetes mellitus on rifampicin's plasma concentration and bioavailability in patients with tuberculosis: A systematic review and meta-analysis study
      Sawsan M.A. El-Sheikh, Amera Sh. Metwally, Azza A.A. Galal
      Therapies.2022;[Epub]     CrossRef
    • Review of Ecological Approach Factors Affecting Physical Activity among Older People
      Wonjung Noh, Ka Young Kim
      Western Journal of Nursing Research.2022; 44(8): 799.     CrossRef
    • Meta-analysis of echinocandins combined with trimethoprim-sulfamethoxazole for treatment of Pneumocystis pneumonia
      Jiayu Guo, Zhongbao Chen, Chenyang Kong, Bo Yu, Tianyu Wang, Yalong Zhang, Yiting Liu, Jiangqiao Zhou, Tao Qiu
      Journal of Chemotherapy.2022; : 1.     CrossRef
    • Selenium status in the body and cardiovascular disease: a systematic review and meta-analysis
      Angelica Kuria, Hongdou Tian, Mei Li, Yinhe Wang, Jan Olav Aaseth, Jiajie Zang, Yang Cao
      Critical Reviews in Food Science and Nutrition.2021; 61(21): 3616.     CrossRef
    • Improving the trustworthiness of findings from nutrition evidence syntheses: assessing risk of bias and rating the certainty of evidence
      Lukas Schwingshackl, Holger J. Schünemann, Joerg J. Meerpohl
      European Journal of Nutrition.2021; 60(6): 2893.     CrossRef
    • The Benefit of Atrioventricular Junction Ablation for Permanent Atrial Fibrillation and Heart Failure Patients Receiving Cardiac Resynchronization Therapy: An Updated Systematic Review and Meta-analysis
      Yoga Waranugraha, Ardian Rizal, Dion Setiawan, Indra Jabbar Aziz
      Indian Pacing and Electrophysiology Journal.2021; 21(2): 101.     CrossRef
    • Multivitamins and risk of fragility hip fracture: a systematic review and meta-analysis
      Indeevar Beeram, Sharri J. Mortensen, Diana Yeritsyan, Kaveh Momenzadeh, Arvind von Keudell, Ara Nazarian
      Archives of Osteoporosis.2021;[Epub]     CrossRef
    • Effectiveness and safety of immunotherapy in NSCLC patients with ECOG PS score ≥2 – Systematic review and meta-analysis
      Bartłomiej Tomasik, Michał Bieńkowski, Marcin Braun, Sanjay Popat, Rafał Dziadziuszko
      Lung Cancer.2021; 158: 97.     CrossRef
    • Intake of Unprocessed and Processed Meat and the Association with Cardiovascular Disease: An Overview of Systematic Reviews
      Marianne Uhre Jakobsen, Anette Bysted, Heddie Mejborn, Anders Stockmarr, Ellen Trolle
      Nutrients.2021; 13(10): 3303.     CrossRef
    • Paediatric obesity: a systematic review and pathway mapping of metabolic alterations underlying early disease processes
      Margot De Spiegeleer, Ellen De Paepe, Lieven Van Meulebroek, Inge Gies, Jean De Schepper, Lynn Vanhaecke
      Molecular Medicine.2021;[Epub]     CrossRef
    • Assessments of risk of bias in systematic reviews of observational nutritional epidemiologic studies are often not appropriate or comprehensive: a methodological study
      Dena Zeraatkar, Alana Kohut, Arrti Bhasin, Rita E Morassut, Isabella Churchill, Arnav Gupta, Daeria Lawson, Anna Miroshnychenko, Emily Sirotich, Komal Aryal, Maria Azab, Joseph Beyene, Russell J de Souza
      BMJ Nutrition, Prevention & Health.2021; 4(2): e000248.     CrossRef
    • Endometriosis and Ectopic Pregnancy: A Meta-analysis
      Paul J. Yong, Samaa Matwani, Chantalle Brace, Andrea Quaiattini, Mohamed A. Bedaiwy, Arianne Albert, Catherine Allaire
      Journal of Minimally Invasive Gynecology.2020; 27(2): 352.     CrossRef
    • Risk factors of tuberculosis infection among health care workers: A meta-analysis
      Septyani Prihatiningsih, Jonny Karunia Fajar, Fredo Tamara, Aditya Indra Mahendra, Chrisandi Yusuf Rizqiansyah, Oktavia Rahayu Adianingsih, Tjipto Suwandi
      Indian Journal of Tuberculosis.2020; 67(1): 121.     CrossRef
    • The association between vitamin D receptor gene polymorphisms and the risk of nephrolithiasis: A meta-analysis
      Besut Daryanto, Basuki Bambang Purnomo, Atma Gunawan, Ema Dianita Mayasari, Arum Gladys Kusumaningrum, Fredo Tamara, Saga Aditya Hutama, Jonny Karunia Fajar
      Meta Gene.2020; 23: 100628.     CrossRef
    • Association between calcium-sensing receptor (CaSR) R990G, CaSR A986S, and CaSR Q1011E gene polymorphisms and the risk of urolithiasis: a meta-analysis
      Besut Daryanto, Basuki Bambang Purnomo, Atma Gunawan, Fredo Tamara, Saga Aditya Hutama, Ema Dianita Mayasari, Arum Gladys Kusumaningrum, Jonny Karunia Fajar
      Egyptian Journal of Medical Human Genetics.2020;[Epub]     CrossRef
    • Risk factors for developing acute compartment syndrome in the pediatric population: a systematic review and meta-analysis
      Sharri J. Mortensen, Sebastian Orman, Edward J. Testa, Amin Mohamadi, Ara Nazarian, Arvind G. von Keudell
      European Journal of Orthopaedic Surgery & Traumatology.2020; 30(5): 839.     CrossRef
    • Epidemiology of postnatal depression and its associated factors in Africa: A systematic review and meta-analysis
      Abel Fekadu Dadi, Temesgen Yihunie Akalu, Adhanom Gebreegziabher Baraki, Haileab Fekadu Wolde, Thach Duc Tran
      PLOS ONE.2020; 15(4): e0231940.     CrossRef
    • Association of Intravenous Tranexamic Acid and the Risk of Allogenic Blood Transfusion in Revision Hip Arthroplasty: a Meta-Analysis of 4517 Patients
      Liang Sun, Qian Gao, Yi Feng
      SN Comprehensive Clinical Medicine.2020; 2(11): 2368.     CrossRef
    • Ergebnisse der systematischen Literatursuche als Grundlage für die „evidenzbasierten Therapieempfehlungen für FMF-Patienten mit unzureichendem Ansprechen bzw. Unverträglichkeit auf Kolchizin“ der Gesellschaft für Kinder- und Jugendrheumatologie und der De
      T. Sahr, U. Kiltz, C. Weseloh, T. Kallinich, J. Braun
      Zeitschrift für Rheumatologie.2020; 79(9): 943.     CrossRef
    • Treatment Strategies for the Optimal Management of Locally Advanced Pancreatic Adenocarcinoma With Curative Intent
      Joseph A. Attard, Alexia Farrugia, Adithya Pathanki, Keith J. Roberts, Bobby Dasari, John Isaac, Yuk Ting Ma, Nikolaos A. Chatzizacharias
      Pancreas.2020; 49(10): 1264.     CrossRef
    • Global prevalence of congenital heart disease in school-age children: a meta-analysis and systematic review
      Yingjuan Liu, Sen Chen, Liesl Zühlke, Sonya V. Babu-Narayan, Graeme C. Black, Mun-kit Choy, Ningxiu Li, Bernard D. Keavney
      BMC Cardiovascular Disorders.2020;[Epub]     CrossRef
    • The role of hepatitis B infection in anti-tuberculosis drug-induced liver injury: a meta-analysis of cohort studies
      Jing Zheng, Mei-Hong Guo, He-Wei Peng, Xiao-Ling Cai, Yun-Li Wu, Xian-E Peng
      Epidemiology and Infection.2020;[Epub]     CrossRef
    • Neurological outcomes by mode of delivery for fetuses with open neural tube defects: a systematic review and meta-analysis
      MC Tolcher, SA Shazly, AA Shamshirsaz, WE Whitehead, J Espinoza, AC Vidaeff, MA Belfort, AA Nassr
      BJOG: An International Journal of Obstetrics & Gynaecology.2019; 126(3): 322.     CrossRef
    • Disordered eating behaviours and autistic traits—Are there any associations in nonclinical populations? A systematic review
      Stephanie Stensbjerg Christensen, Mette Bentz, Lars Clemmensen, Katrine Strandberg‐Larsen, Else Marie Olsen
      European Eating Disorders Review.2019; 27(1): 8.     CrossRef
    • Physical Activity Level and Risk of Falling in Community-Dwelling Older Adults: Systematic Review and Meta-Analysis
      Wuber J.S. Soares, Alexandre D. Lopes, Eduardo Nogueira, Victor Candido, Suzana A. de Moraes, Monica R. Perracini
      Journal of Aging and Physical Activity.2019; 27(1): 34.     CrossRef
    • Oral hormone pregnancy tests and the risks of congenital malformations: a systematic review and meta-analysis
      Carl J. Heneghan, Jeffrey K. Aronson, Elizabeth Spencer, Bennett Holman, Kamal R. Mahtani, Rafael Perera, Igho Onakpoya
      F1000Research.2019; 7: 1725.     CrossRef
    • Global birth prevalence of congenital heart defects 1970–2017: updated systematic review and meta-analysis of 260 studies
      Yingjuan Liu, Sen Chen, Liesl Zühlke, Graeme C Black, Mun-kit Choy, Ningxiu Li, Bernard D Keavney
      International Journal of Epidemiology.2019; 48(2): 455.     CrossRef
    • Impact of acculturation on oral health among immigrants and ethnic minorities: A systematic review
      Rana Dahlan, Parvaneh Badri, Humam Saltaji, Maryam Amin, Frédéric Denis
      PLOS ONE.2019; 14(2): e0212891.     CrossRef
    • Roux-en-Y gastric bypass or sleeve gastrectomy for obstructive sleep apnea: A systematic review and meta-analysis
      Hussein Al-Rubaye, Emma Rose McGlone, Borna Farzaneh, Livyar Mustafa, Mae Johnson, Ajit Kayal, Caroline-Louise English, Vasha Kaur, Myutan Kalendran, Marcus Reddy, Omar A. Khan
      Laparoscopic, Endoscopic and Robotic Surgery.2019; 2(3): 53.     CrossRef
    • Impact of social support on oral health among immigrants and ethnic minorities: A systematic review
      Rana Dahlan, Ebtehal Ghazal, Humam Saltaji, Bukola Salami, Maryam Amin, Astrid M. Kamperman
      PLOS ONE.2019; 14(6): e0218678.     CrossRef
    • Left Ventricular Systolic Dysfunction and Cardiovascular Outcomes in Tetralogy of Fallot: Systematic Review and Meta-analysis
      Alexander C. Egbe, Rosalyn Adigun, Vidhu Anand, Collin P. West, Victor M. Montori, Hassan M. Murad, Emmanuel Akintoye, Karim Osman, Heidi M. Connolly
      Canadian Journal of Cardiology.2019; 35(12): 1784.     CrossRef
    • A systematic review and meta-analysis of the genetic characterization of human echinococcosis in Iran, an endemic country
      Abolghasem Siyadatpanah, Davood Anvari, Amir Emami Zeydi, Seyed Abdollah Hosseini, Ahmad Daryani, Shahabeddin Sarvi, Christine M. Budke, Reza Esmaeelzadeh Dizaji, Mohammad Ali Mohaghegh, Mohammad Hasan Kohansal, Samira Dodangeh, Reza Saberi, Shirzad Ghola
      Epidemiology and Health.2019; 41: e2019024.     CrossRef
    • The Association between Autism Spectrum Disorder and Pre- and Postnatal Antibiotic Exposure in Childhood—A Systematic Review with Meta-Analysis
      Eunmi Lee, Jeonghyun Cho, Ka Young Kim
      International Journal of Environmental Research and Public Health.2019; 16(20): 4042.     CrossRef
    • Association between blood circulating vitamin D and colorectal cancer risk in Asian countries: a systematic review and dose-response meta-analysis
      Lin Zhang, Huachun Zou, Yang Zhao, Chunlei Hu, Adejare Atanda, Xuzhen Qin, Peng Jia, Yu Jiang, Zhihong Qi
      BMJ Open.2019; 9(12): e030513.     CrossRef
    • Antibiotics exposure and risk of inflammatory bowel disease: a systematic review
      Nikoletta A. Theochari, Anastasios Stefanopoulos, Konstantinos S. Mylonas, Konstantinos P. Economopoulos
      Scandinavian Journal of Gastroenterology.2018; 53(1): 1.     CrossRef
    • Prescribed Dose of Opioids and Overdose: A Systematic Review and Meta-Analysis of Unintentional Prescription Opioid Overdose
      Adeleke D. Adewumi, Samantha A. Hollingworth, Joemer C. Maravilla, Jason P. Connor, Rosa Alati
      CNS Drugs.2018; 32(2): 101.     CrossRef
    • Systematic review with meta-analysis: association between acetaminophen and nonsteroidal anti-inflammatory drugs (NSAIDs) and risk of Crohn's disease and ulcerative colitis exacerbation
      O. O. Moninuola, W. Milligan, P. Lochhead, H. Khalili
      Alimentary Pharmacology & Therapeutics.2018; 47(11): 1428.     CrossRef
    • Predictive value of coronary computed tomography angiography in asymptomatic individuals with diabetes mellitus: Systematic review and meta-analysis
      Ebba Beller, Felix G. Meinel, Franziska Schoeppe, Wolfgang G. Kunz, Kolja M. Thierfelder, Jörg Hausleiter, Fabian Bamberg, U. Joseph Schoepf, Verena S. Hoffmann
      Journal of Cardiovascular Computed Tomography.2018; 12(4): 320.     CrossRef
    • Assessing the impact of TRAP laws on abortion and women’s health in the USA: a systematic review
      Nichole Austin, Sam Harper
      BMJ Sexual & Reproductive Health.2018; 44(2): 128.     CrossRef
    • Association between plasminogen activator inhibitor-1 and cardiovascular events: a systematic review and meta-analysis
      Richard G. Jung, Pouya Motazedian, F. Daniel Ramirez, Trevor Simard, Pietro Di Santo, Sarah Visintini, Mohammad Ali Faraz, Alisha Labinaz, Young Jung, Benjamin Hibbert
      Thrombosis Journal.2018;[Epub]     CrossRef
    • Oral hormone pregnancy tests and the risks of congenital malformations: a systematic review and meta-analysis
      Carl J. Heneghan, Jeffrey K. Aronson, Elizabeth Spencer, Bennett Holman, Kamal R. Mahtani, Rafael Perera, Igho Onakpoya
      F1000Research.2018; 7: 1725.     CrossRef

    Figure
    Epidemiol Health : Epidemiology and Health
    © Korean Society of Epidemiology.