Table 3

Mixed-methods studies. Quality assessed using the Mixed Methods Appraisal Tool (MMAT)

Criteria for mixed-methods characteristics of mixed-methods studies
Authors S1. Are there clear research questions? S2. Do the collected data allow to address the research questions? 5.1. Is there an adequate rationale for using a mixed-methods design to address the research question? 5.2. Are the different components of the study effectively integrated to answer the research question? 5.3. Are the outputs of the integration of qualitative and quantitative components adequately interpreted? 5.4. Are divergences and inconsistencies between quantitative and qualitative results adequately addressed? 5.5. Do the different components of the study adhere to the quality criteria of each tradition of the methods involved?
Kim et al 57 YesYesNoYesYesYesPartial
Nolan et al 58 YesYesPartialYesYesYesPartial
Puts et al 59 YesYesYesPartialPartialYesPartial
Criteria for qualitative component of mixed-methods studies
1.1. Is the qualitative approach appropriate to answer the research question? 1.2. Are the qualitative data collection methods adequate to address the research question? 1.3. Are the findings adequately derived from the data? 1.4. Is the interpretation of results sufficiently substantiated by data? 1.5. Is there coherence between qualitative data sources, collection, analysis and interpretation?
Kim et al 57 PartialYesYesYesNo
Nolan et al 58 PartialYesYesYesYes
Puts et al 59 YesYesYesYesYes
Criteria for quantitative component of mixed-methods studies
4.1. Is the sampling strategy relevant to address the research question? 4.2. Is the sample representative of the target population? 4.3. Are the measurements appropriate? 4.4. Is the risk of non-response bias low? 4.5. Is the statistical analysis appropriate to answer the research question?
Kim et al 57 YesPartialYesNoYes
Nolan et al 58 YesPartialYesPartialYes
Puts et al 59 YesPartialCan't tellNoPartial
  • Rating mixed-methods studies using MMAT involves scoring criteria 1.1–1.5 (qualitative dimension) plus (criteria 2.1–2.5 (quantitative—RCTs) or criteria 3.1–3.5 (quantitative—non-randomised trials) or criteria 4.1–4.5 (quantitative descriptive)) plus criteria 5.1–5.5 (mixed methods).

  • Criteria 2.1–2.5 or criteria 3.1–3.5 of the MMAT are not listed here because none of these studies are intervention studies.

  • RCT, randomised controlled trial.