Misuse of analysis of variance in African biomedical journals: a call for more vigilance

Abstract

Background

Misuse of analysis of variance alongside other statistical methods has been an important topic of discussion in many scientific gatherings. Although misuse of analysis of variance is of global concern, its prevalence in African-based biomedical journals has raised concerns among colleagues.

Methods

We sampled the current issues/last published issues of all African biomedical journals aggregated in an African journal aggregator. We scanned through all the journals’ sample articles in the journal aggregator and retrieved articles that either applied analysis of variance or were supposed to use the test.

Results

Our results show that 126 articles of the total 1789 sampled articles employed analysis of variance test for analysis. Of these, 99 (78.6%) articles used the analysis of variance test correctly, while 27 (21.4%) misused the test. Not stating the type of analysis of variance test employed for analysis was the most prevalent misuse of analysis of variance test.

Conclusions

In overall, the study demonstrated that misuse of analysis of variance test is prevalent in African biomedical journals. Therefore, all stakeholders (medical schools, reviewers, journal editors) need to do more to nip it in the bud.

Background

Statistics is the science of collection, organization, analysis of data and deduction of conclusions from samples to the whole population (Adeneye et al. 2019). Data collected from a sample population are analysed using appropriate statistical tests to provide a useful information about the population. However, improper use of these statistical tests may result in erroneous conclusions, misleading interpretations and inferences to the main population (Adeyemi et al. 2020). One such widely used statistical tests is analysis of variance (ANOVA).

ANOVA was developed by Ronald Fisher in the year 1918 (Ahmad et al. 2018). Since then, the test has become widespread and increasingly used statistical methods in biomedical, educational and psychological research. ANOVA maybe prone to misuse, potentially due to its common application in research, inadequate knowledge about biostatistical methods, lack of consultation of statistics experts, or conspicuously to achieve a desired result.

On a general note, ANOVA belongs to a category of statistical test known as parametric tests, and by implication, data to be analysed using ANOVA must fulfil certain assumptions (Akindele et al. 2018). These assumptions include the independence of samples, normality and homogeneity of the data variance (Ali et al. 2018). By and large, ANOVA types include one-way ANOVA, two-way ANOVA and multivariate ANOVA (MANOVA) (Assaad et al. 2018).

Biomedical-related disciplines contribute the largest chunk of the articles published in the academic world, especially in Africa (Björk and Solomon 2019; Chiarotti 2004). A good number of experiments and tests performed in biomedical-related disciplines involve three or more test groups. Therefore, they need a statistical test that can analyse investigations involving more than two test groups. Here, some researchers without adequate statistic knowledge assume that ANOVA is the gold standard statistical test when investigating differences among more than two test groups. Misuse of ANOVA has been a major topic of discussion in many scientific gatherings. Also, on several encounters, many colleagues have raised concerns regarding the misuse of ANOVA in some biomedical-related journals based in Africa. However, to our knowledge, there are no empirical data in the literature evaluating the misuse of ANOVA in peer-reviewed journals from African journals. We, therefore, undertook a well-detailed investigation into the issue, by assessing the articles in current issues of journals aggregated by the African Journals Online (www.ajol.info).

Methods

We retrieved scientific articles from current issues or last published issues, as the case may be, of health-related journals aggregated by the African Journals Online (AJOL-https://www.ajol.info/index.php/ajol). Articles were clicked on and downloaded from each journal in the AJOL website. We selected health-related journals from the list of “categories of journals” located on the AJOL web page. We selected articles that reported any ANOVA statistical test from the health-related journals. Articles that employed ANOVA were detected by scrutinizing the methods and results of the articles for mentions of ANOVA or data that depict the use of ANOVA. These articles were evaluated to verify whether they applied the ANOVA statistical test correctly. Articles that did not use the ANOVA statistical test and were presumably supposed to use it were also documented. Based on the scope of this research, an article is assumed to have misused ANOVA if it fails to meet all of the following criteria. Not performing a follow-up (pairwise, or similar) comparisons after a significant ANOVA in specific cases, not performing an ANOVA test while stating otherwise and vice versa, using ANOVA to compare only 2 groups, not ensuring that the samples are independent of one another, not ensuring that the dependent variable is continuous, not performing a post hoc test after ANOVA in some situations where it is necessary, stating that ANOVA test was used while in reality another statistical test was what was used, not stating the type (one-way, two-way or multivariate) of ANOVA used for the study, being silent about the type of statistical test employed for the analysis when ANOVA was clearly used and comparing raw scores instead of means of categorical independent variables using the ANOVA statistical test. Drioiche et al. is a typical example of an article in the list that applied ANOVA statistical test correctly (David et al. 2018). The authors stated the type of ANOVA they used, they also used means of the scores for the ANOVA test and not raw data from the variables. They also performed a post hoc test, and the variables (groups) analysed by the authors are more than two. Their dependent variables are continuous, and the samples are independent.

We designed a data collection form based on guidance from the literature (Drioichea et al. 2019; Ehwarieme and Anarado 2017; Sherbiny et al. 2018). We collected information related to the name of the journal, type of data analysed, the number of groups in the study, sample size per group, independence of sample, type of ANOVA statistical test, post hoc test and citations. All authors collected the data. To ensure accuracy, the collected information was reviewed by two of the authors (BA and AIJ). The authors resolved discrepancies through consensus discussion. Articles without any misuse of ANOVA test were only counted and documented.

Results

As of 31 December 2020, the information on the web page of AJOL database showed a total number of 873 African journals, of which 168 (19.2%) were biomedical-related journals. Articles in 42 journals (25%) of the 168 biomedical-related journals employed one or more ANOVA statistical tests at least in one of the articles of their current issues (December 2020). Of the 42 journals that used the ANOVA test, 15 (41.7%) journals published papers that misused the test, while 36 journals reported studies that used the test correctly. Nine journals published articles that comprise both correct use and misuse of ANOVA test (Fig. 1a).

Fig. 1

Organogram of journals (1a) and articles (1b) that misused ANOVA tests in the current issues of journals aggregated by African Journal Online (AJOL)

The 168 AJOL journals published 1789 articles in total as their respective current issues. Of these articles, 126 articles used the ANOVA test in analysing the outcome of their study at least once. Ninety-nine (78.6%) articles used the ANOVA statistical test correctly, while 27 (21.4%) misused the test. Except for one article that involves the two-way ANOVA, all other misuses of ANOVA involved the one-way ANOVA. There was no case of MANOVA misuse (Fig. 1b). The list of articles that misused ANOVA test is given in Table 1.

Table 1 Articles that misused ANOVA from the current issues of health-related journals published by the African Journals Online

Discussion

Except for five articles in Table 1, all the 27 articles that misused ANOVA test fulfilled the three verifiable statistical assumptions. The unfulfilled assumption of the four articles in question relates to the independent variable. Igbigbi et al. (2018) in article number 12 stated that “the comparison between ethnic groups was by means of one-way analysis of variance (ANOVA)”, whereas the authors studied only two ethnic groups (Delta Igbos and Isokos) as categorical independent variables (Emara et al. 2018). A similar scenario also played out in articles number 27 (Ephraim et al. 2018), number 13 (Field 2013) and number 5 (Gore et al. 1977) where all groups of authors stated that they employed one-way ANOVA test. In contrast, only two groups were analysed as categorical independent variables. Muhammad et al. (2018) in article number 18 has a more obvious scenario as the authors claimed they employed one-way ANOVA in analysing one variable at a time (Han et al. 2020). In reality, all the aforementioned group of authors in the four articles did not perform the ANOVA test but decided to state otherwise. Such misinformation could misguide young academics into the correct use of ANOVA test in data analysis.

Adeyemi et al. (2020) in article number 1 compared the difference among mean temperatures of six categorical independent variables without using the ANOVA test (Idenya and Gichangi 2020). Although no statistical test was mentioned to have been employed in the analysis, ANOVA test is conspicuously missing in the whole article. Furthermore, a post hoc test is usually necessary after an ANOVA test to pinpoint the independent variable(s) where the difference in means lies for the analysed means. The authors of articles number 1 did not perform or state that they performed a post hoc analysis. Similar to article 1, authors of articles number 3 (Igbigbi et al. 2018), 4 (Iikasha et al. 2019) and 6 (Iyomon et al. 2016) also compared differences among means of several categorical independent variables without mentioning the use the ANOVA test. The act of not mentioning the statistical test employed for each analysis embarked upon in a study could be misleading to readers.

Authors of article number 7 used the ANOVA test and post hoc analysis correctly, but they did not state the type of ANOVA test (whether MANOVA, one-way or two-way ANOVA) they employed (Khowailed et al. 2018). Another form of this error is to be silent about the statistical test employed for analysis, as seen in article number 20 (Korany et al. 2018). This error is usually made by some authors employing one-way ANOVA in their analysis. They assume that stating that ANOVA is employed in a study is similar to saying that one-way ANOVA is employed. This error is rarely found with other than one-way ANOVA statistical test. We also found a similar error in articles number 8 (Kunle-Alabi et al. 2019), 9 (Lee et al. 2020), 10 (Milojević 2020), 15 (Muhammad et al. 2018), 21 (Ogunro et al. 2019), 23 (Ogunro et al. 2019) and 25 (Osman et al. 2019).

Lee et al. (2020) in article number 2 compared raw score, instead of means, of more than three categorical independent variables using the ANOVA test (Park et al. 2009). This blatantly does not seem right because the ANOVA test compares means and not a raw score of independent variables. Deductions made as a result of this type of one-way ANOVA test application would be misleading. We found similar errors in articles number 10 (Milojević 2020), 19 (Sanni et al. 2020) and 26 (Skaik 2015).

El Sherbiny et al. (2018) in article number 11 failed to perform a post hoc analysis after arriving at a significant one-way ANOVA test result (Sprent 2003). The authors discussed the comparison of the means of the ANOVA test results without running a post hoc analysis. This is impossible in statistical terms as only a post hoc analysis will pinpoint where a significant difference in means exists.

David et al. (2018) in article number 14 stated that they performed an ANOVA test, while in reality, the authors analysed their data with chi-square test (Tabachnick and Fidell 2007). The authors employed the right statistical test as they were interested in relationships between the categorical variables. This type of misinformation could mislead inexperienced academics trying to learn from such articles in applying the right statistical test for the right data. The scientific method is unique for its reproducibility. Not stating the proper statistical test used during a study could defeat this cardinal objective of reproducibility in science. Authors of articles number 16 (Uduagbamen et al. 2020), 22 (Winters et al. 2010) and 24 (Yuan et al. 2020) also made similar errors where the authors stated under their methodology section that they used the ANOVA test which was not the case. Also, authors of article number 17 did not state that they performed a post hoc analysis as is evident in the article after a one-way ANOVA test (Zhou and Skidmore 2018).

Correct application of ANOVA test involves fulfilling well-established statistical assumptions. These include that the dependent variable should be continuous, the independent variable should consist of three or more categorical independent groups, there should be independence of observations, the dependent variable should be normally distributed for each category of the independent variable, and there should be homogeneity of variance. Of the five stated assumptions for ANOVA statistical test, only the first 3 are verifiable from an already published journal article.

By and large, the misuse of ANOVA test in African biomedical journals followed six patterns. The first is employing the ANOVA test when less than three categorical independent variables are involved. The second is not mentioning that an ANOVA test and/or a post hoc analysis was employed while the tests were used for analysis. The third is not stating the type of ANOVA test one employs for an analysis. This includes statements like “ANOVA was used to analyse the data”. This is the most prevalent misuse recorded in this study. The fourth is using ANOVA tests to compare raw scores instead of means of categorical independent variables. The fifth is failing to perform a post hoc analysis after obtaining a significant p value in an ANOVA test while one is interested in pinpointing the mean(s) of the categorical independent variable(s) that is different from others. The sixth is mentioning that a type of ANOVA test was employed for the analysis while another type of ANOVA test or a different statistical test like chi-square was used.

Recommendations

Understanding the proper use of ANOVA statistical tests is an important aspect of reporting biomedical research. This translates into lesser errors while reporting results from studies. Alongside other statistical tests, ANOVA tests should be taught by experienced bio-statisticians to biomedical researchers and students. Also, journal reviewers and editors should ensure a thorough job is done before certifying a manuscript fit for publication in their respective journals. Again, researchers should always seek the opinions of biostatistician and their peers on the use of ANOVA tests and other statistical tests.

Limitations

We obtained information from the full text of articles of free access journals. However, given the lack of funding, we were unable to access five journals that required subscription. Also, misuse of ANOVA involving data that do not pass the normality test cannot be assessed because we do not have access to the raw data of the articles.

Conclusions

ANOVA statistical tests misuse is relatively common in African-based biomedical-related health journal with misuse related to not stating the type of ANOVA test used for analysis being the most common.