Table 3

Mixed-methods studies. Quality assessed using the Mixed Methods Appraisal Tool (MMAT)

Criteria for mixed-methods characteristics of mixed-methods studies
AuthorsS1. Are there clear research questions?S2. Do the collected data allow to address the research questions?5.1. Is there an adequate rationale for using a mixed-methods design to address the research question?5.2. Are the different components of the study effectively integrated to answer the research question?5.3. Are the outputs of the integration of qualitative and quantitative components adequately interpreted?5.4. Are divergences and inconsistencies between quantitative and qualitative results adequately addressed?5.5. Do the different components of the study adhere to the quality criteria of each tradition of the methods involved?
Kim et al57YesYesNoYesYesYesPartial
Nolan et al58YesYesPartialYesYesYesPartial
Puts et al59YesYesYesPartialPartialYesPartial
Criteria for qualitative component of mixed-methods studies
1.1. Is the qualitative approach appropriate to answer the research question?1.2. Are the qualitative data collection methods adequate to address the research question?1.3. Are the findings adequately derived from the data?1.4. Is the interpretation of results sufficiently substantiated by data?1.5. Is there coherence between qualitative data sources, collection, analysis and interpretation?
Kim et al57PartialYesYesYesNo
Nolan et al58PartialYesYesYesYes
Puts et al59YesYesYesYesYes
Criteria for quantitative component of mixed-methods studies
4.1. Is the sampling strategy relevant to address the research question?4.2. Is the sample representative of the target population?4.3. Are the measurements appropriate?4.4. Is the risk of non-response bias low?4.5. Is the statistical analysis appropriate to answer the research question?
Kim et al57YesPartialYesNoYes
Nolan et al58YesPartialYesPartialYes
Puts et al59YesPartialCan't tellNoPartial
  • Rating mixed-methods studies using MMAT involves scoring criteria 1.1–1.5 (qualitative dimension) plus (criteria 2.1–2.5 (quantitative—RCTs) or criteria 3.1–3.5 (quantitative—non-randomised trials) or criteria 4.1–4.5 (quantitative descriptive)) plus criteria 5.1–5.5 (mixed methods).

  • Criteria 2.1–2.5 or criteria 3.1–3.5 of the MMAT are not listed here because none of these studies are intervention studies.

  • RCT, randomised controlled trial.