Introduction {#sec1-1}
============

Statistical analysis is a part of the process of writing a scientific article. It is an essential technique that enables a medical researcher to draw meaningful conclusions from their data analysis.\[[@ref1]\] Statisticians and methodological experts should be consulted during the study design, analysis, and manuscript writing phases to improve the quality of research and to ensure clear and appropriate application of quantitative methods.\[[@ref2]\] On the other hand, many researchers have difficulty or delay in getting a statistical advice or the statistician\'s involvement in their research from early stages of study design.\[[@ref3]\]

The statistical software programs over the past years expanded analytic capabilities and broadened the spectrum of appropriate statistical options.\[[@ref4]\] Researchers have to be adequately trained in the application of statistics for biomedical research.\[[@ref5]\] It is of great importance to implement statistics accurately and carefully so that the results will be more credible and meaningful.\[[@ref6]\] With limited statistical knowledge of most physicians, it is not uncommon to find statistical errors. Statisticians have documented that statistical errors are common, and at least one error could be found in about 50% of the published articles.\[[@ref7]\]

Many journals adopt guidelines to improve reporting manuscripts as the Consolidated Standards of Reporting Trials,\[[@ref8]\] the Transparent Reporting of Evaluations with Nonrandomized Designs\[[@ref9]\] Statement, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE),\[[@ref10]\] Guideline specific for reporting statistical analysis, the Statistical Analyses and Methods in the Published Literature (SAMPL Guidelines).\[[@ref11]\] These guidelines and others for reporting scientific research were also available at EQUATOR network.\[[@ref12]\]

"Because society depends on sound statistical practice, all practitioners of statistics, whatever their training and occupation, have social obligations to perform their work in a professional, competent, and ethical manner." (Ethical Guidelines for Statistical Practice, American Statistical Association, 1999.)\[[@ref13]\] These principles should also guide the statistical work of professionals in all other disciplines that use statistical methods.\[[@ref13]\]

All assistant lecturers in Suez Canal University have to attend a statistical course and evaluation in the educational curriculum of doctorate degree. Publishing research articles are a mandatory process in postdoctorate promotion. Revising the statistical reporting in the past articles aimed to improve the future manuscripts for publication. This study had two objectives. Objective 1: To determine the types and frequencies of statistical methods in family medicine (FM) research articles. Objective 2: To assess the quantity and character of statistical errors and deficiencies.

Methods {#sec1-2}
=======

This was a cross-sectional study, the data were collected retrospectively. It was conducted by the researcher between May and August 2015. FM research article selection: Included all original FM articles that were published by FM authors with affiliation to Suez Canal University; all published articles in different National and International Medical Journals between 2010 and 2014. All articles were downloaded in full text as a portable document format. Commentaries, letters to the editor, review articles, and articles with themes that were not related to the scope of FM were excluded.

FM research article search: (1) All published articles by FM authors with affiliation to Suez Canal University were available in FM Department database from 1992 to 2013 in 39 medical journals as previously collected in a previous study.\[[@ref14]\] The researcher updated the search to include all articles that were published in 2014 on(2) National Journal Websites (The Egyptian journal of Community Medicine, The Medical Journal of Cairo University, and Suez Canal University) and (3) Google and PubMed search for other publications.

The searched articles were published in 24 medical journal (African Safety Promotion, Annals of Burns and Fire Disasters, Eastern Mediterranean Health Journal, Egyptian Journal of Neurology and Psychiatry, Elective Medical Journal, FM and Medical Science Research, International Journal of Health Sciences, International Journal of Medicine and Public Health, Journal of American Science, Journal of FM and Primary Care, Journal of Family and Community Medicine, Journal of the Egyptian Public Health, Journal of Tibah University Medical Sciences, Medical Journal of Cairo University, Middle East Journal of FM, Medical Teacher, Open Access Scientific Reports, Pan African Medical Journal, Peer Journal, Saudi Medical Journal, Suez Canal University Medical Journal, The Arab Journal of Psychiatry, The Egyptian Journal of Community Medicine, and The Egyptian Rheumatologist).

Main outcomes were the types and frequencies of statistical methods in all screened articles; the statistical errors and deficiencies related to study designs, application, and documentation of statistical analyses, data presentation and interpretation in articles with identified inferential statistics.

Statistical methods {#sec2-1}
-------------------

Types and frequencies of applied statistical methods were recorded for all the 66 articles and classified into 15 out of 21 categories, earlier used by Emerson and Colditz in 1983 \[[Table 1](#T1){ref-type="table"}\].\[[@ref15]\] If the same statistical method was repeatedly used in the same article, the method was documented once; however, if more than one statistical technique were used in one article, each of them was considered separately.

###### 

Categories of statistical procedures used to assess the statistical contents of articles

![](JFMPC-5-24-g001)

Articles containing identified inferential statistical methods beyond descriptive statistics were further classified into basic or advanced analyses according to the sophistication of applied statistical techniques as previously used by Strasak *et al*., 2007.\[[@ref16][@ref17]\] Basic analyses included *t*-test, simple contingency table analysis, nonparametric methods, one-way analysis of variance (ANOVA), correlation, and simple linear regression. Advanced analyses included any method of statistical modeling, multivariate analysis (e.g., multivariate ANOVA, multivariate analysis of covariance MANCOVA), advanced contingency table analysis, epidemiologic statistics, or survival analysis.

Statistical errors {#sec2-2}
------------------

All articles that included identified basic or advanced inferential methods beyond descriptive statistics were included. The articles were screened using a comprehensive 58-item checklist: \[[Appendix 1](#App1){ref-type="app"}\]; 46 items were the checklist developed and used in two studies by Strasak *et al*., 2007\[[@ref16][@ref17]\] and the researcher added 10 items specific to regression analysis and one item in the presentation of results regarding the error of not reporting the test statistics, these additional items originated from SAMPL guidelines\[[@ref11]\] and previously used in the study by Hassan *et al*., 2015.\[[@ref18]\] Another item was added in the documentation related to reporting the name of the statistical software package used in statistical analysis.\[[@ref11]\] In the application of the checklist, the error committed was restricted to obvious ones that could clearly be identified. Unable to assess/not clear was recorded if an article contained insufficient information to assess a specific item of the checklist. Application correct was given to perfect issues. The researcher was adherent to the guidelines by Strasak *et al*., 2007,\[[@ref19]\] and SAMPL\[[@ref11]\] which provided a detailed clarification to the items of the checklist. Some items were more detailed by others.\[[@ref20][@ref21][@ref22][@ref23][@ref24][@ref25]\]

Categorization of study designs within FM research articles was previously sorted in the study by Abdulmajeed *et al*., 2013:\[[@ref14]\] quantitative study designs, observational studies (cross-sectional, case--control, and cohort), and intervention studies (with randomization or without randomization). None of the published FM articles were with a cohort design.

Incompatibility of the applied tests with data type was checked based on Chi-square tests are suitable for categorical data presented in frequencies and percentages. Parametric tests (e.g., *t*-test, ANOVA) are suitable with normally distributed continuous data presented in mean (standard deviation \[SD\]). Nonparametric tests (e.g., Mann--Whitney U/Wilcoxn rank sum, Kruskal--Wallis H, Wilcoxon signed rank, and Friedman\'s tests) are suitable in comparison of continuous data not normally distributed expressed in medians and (interquartile range) or mean ranks.\[[@ref20][@ref21]\]

Independence: Student\'s *t*-test and Wilcoxon test were checked for reporting the used variant paired/dependent in comparison of pre- and post-experimental studies and matched controlled studies or unpaired/independent in comparison of two independent samples.\[[@ref20][@ref22]\] Furthermore, paired and matched comparisons were checked for the use of (paired *t*-test, Wilcoxon signed rank-test, and Mcnemar test).\[[@ref22]\]

Checking the distribution of continuous data (normal or nonnormal) is a prerequisite to the presentation of descriptive statistics and the selection of parametric or nonparametric tests.\[[@ref20]\] The assumption of normality is that the normal distribution of variables in case of *t*-test or ANOVA and the distribution of residuals in case of regression. The assumption of homogeneity of variance requires equal population variances per group in case of *t*-test and ANOVA.\[[@ref11][@ref23]\]

Skewness of data was checked based on the two tricks by Altman and Bland 1996\[[@ref24]\] as the data were likely to be skewed if the mean was smaller than twice the SD and highly skewed if the mean was smaller than SD. The second trick in case of several groups stated that if SD increased as the mean increased was a good indication of positive skewed data.

Adequate cell size was checked in Chi-square test no more than 20% of the cells should have expected frequencies \<5. For example, within 2 × 2 tables, no cell should have an expected frequency \<5.\[[@ref25]\] Fisher exact test is used when this assumption is not met. The expected frequency of a contingency table cell was calculated as expected cell frequency = (row total × column total)/grand total.\[[@ref25]\]

In presentation of data, confidence interval (CI) as a measure of precision was checked in reporting effect size measures such as risks (e.g., absolute risks; relative risk differences); rates (e.g., incidence rates; survival rates); ratios (e.g., odds ratios, hazards ratios); and in reporting coefficients in association, correlation, and regression.\[[@ref11]\]

Data analysis {#sec2-3}
-------------

The data were extracted from the published articles then entered and analyzed using a Statistical Package for Social Sciences program (SPSS, version 20 IBM, Chicago, IL, USA). Data were presented using descriptive statistics in the form of frequencies and percentages for the qualitative variables.

Results {#sec1-3}
=======

Statistical methods {#sec2-4}
-------------------

The majority of the reviewed articles contained inferential statistical tests (93.9%). More than half of the screened articles contained contingency tables 38/66 (57.6%). Regression analyses (logistic and multiple linear) were recorded in more than one-third of the searched articles 26/66 (39.4%) and a quarter of articles 17/66 (25.8%) mentioned *t*-test. The least used inferential tests were Wilcoxon signed rank test, Kruskal--Wallis H, and McNemar 1/66 (1.5%) for each test. Furthermore, normality test and log transformation were mentioned in only (1.5%). More than one-third of articles contained advanced analyses 29/66 (43.9%) \[[Table 2](#T2){ref-type="table"}\].

###### 

Types and frequencies of statistical methods

![](JFMPC-5-24-g002)

Deficiencies in study design {#sec2-5}
----------------------------

No mentioned sample size calculation was found in approximately one-third of the articles 19/60 (31.7%). Methods of randomization/allocation to intervention were not clearly stated in 2/60 (3.3%) which represented 2/2 (100.0%) of randomized controlled trial (RCT) articles \[[Table 3](#T3){ref-type="table"}\].

###### 

Statistical deficiencies and errors in study design

![](JFMPC-5-24-g003)

Errors in statistical analysis {#sec2-6}
------------------------------

Wrong analyses were recorded in more than a quarter of articles as 17/60 (28.3%). Failure to proof/report that Student\'s *t*-test assumptions is not violated in a quarter all articles 15/60 (25.0%), in most of articles with *t*-test 15/17 (88.2%). The assumptions of multiple regression were not reported in 6/60 (10.0%) which represented most of articles with multiple linear regression 6/8 (75.0%) that mentioned the use of multiple regression. Use of Chi-square test instead of Fisher\'s exact was mentioned in 5/60 (5.8%). Failure to include alpha correction in multiple comparisons was in 4/60 (6.7%) of all articles and these were all articles 4/4 (100.0%) that mentioned the use of multiple comparisons \[[Table 4](#T4){ref-type="table"}\].

###### 

Statistical deficiencies and errors in data analysis

![](JFMPC-5-24-g004)

Errors in documentation {#sec2-7}
-----------------------

Fifty-nine articles (98.3%) showed failure to define details of a test performed. Failure to state number of tails of significance tests was at 59/60 (98.3%). One-fifth of the articles, i.e., 12/60 (20.0%) showed failure to specify which test was performed on a given set of data when multiple tests were used. In a quarter of articles, there was failure to state if *t*-test was paired or unpaired 15/60 (25.0%) \[[Table 5](#T5){ref-type="table"}\].

###### 

Statistical errors and deficiencies in documentation, data presentation, and interpretation

![](JFMPC-5-24-g005)

Errors in data presentation {#sec2-8}
---------------------------

More than half of the articles, i.e., 32/60 (53.3%) showed no value of test statistics (at least one table in the article contains this error). One-fifth of the articles., i.e., 12/60 (20.0%) presented only *P* value without CIs for main effect size measures. Use of mean (SD) to describes ordinal/nonnormal data 8/60 (13.3%). Numerical imprecision was found in 6/60 (10.0%) \[[Table 5](#T5){ref-type="table"}\].

Errors in data interpretation {#sec2-9}
-----------------------------

Errors related to conclusions without support by the study data 5/60 (8.3%), reporting significance without data analysis and missing the discussion of the problem of multiple significance testing were shown in only 2/60 (3.3%) of articles \[[Table 5](#T5){ref-type="table"}\].

Discussion {#sec1-4}
==========

The use of inferential statistics was found in the vast majority of the screened articles giving the advantage and evidence of their analytic character. Although the more frequently recorded deficiencies were related to inadequate documentation of the used statistical methods, the use of wrong statistical test in more than a quarter of the articles was a major finding.

Contingency table analysis was used twice more frequently than *t*-test among the simple tests. These results were relatively in agreement with the British study in Family Practice Articles over 1 year by Rigby *et al*., 2004\[[@ref26]\] and Emerson and Colditz 1983\[[@ref15]\] in articles with cross-sectional studies. Contingency tables were less used in prospective and retrospective study designs in other studies.\[[@ref16][@ref17][@ref27]\] The use of survival analysis and Chi-square tests followed by nonparametric tests was observed in American surgical articles.\[[@ref28]\] The selection of test depends partly on types of study designs.

The use of normality tests was mentioned in only 1.5% of articles could explain in part by the inappropriate presentation of skewed data in mean (SD) and inappropriate use of parametric methods for skewed data. Checking the normality was lower than in another study.\[[@ref26]\] Multiple comparisons were used only in 4/7 of the reported ANOVA tests; these results were higher than findings by Olsen in 2003\[[@ref20]\] and partly consistent with the results of ignoring or misusing the method of multiple pair-wise comparisons in ANOVA in the analysis of Chinese articles.\[[@ref29]\] The presentation of unidentified method is an error and deficiency in both documentation and presentation. However, these unidentified methods were excluded from further assessment.

Basic analyses were used slightly more in articles than advanced analyses. These results were nearly consistent with other studies.\[[@ref28][@ref30]\] Pet *et al*., 2014,\[[@ref30]\] mentioned that the sophistication of statistical methods are going to be increased over time and avoiding use of advanced techniques may miss many possible important inferences from the same data. The difference in selection of inferential statistical tests depends on study designs, the main study hypothesis, type of data, and independence of variables.\[[@ref22]\]

One of two RCTs was with no sample size calculation. This point is crucial to detect treatment effects.\[[@ref19][@ref21]\] If no sample size calculation was used the study size must be justified, for example, all available patients in two centers were included and a sample size calculation was not relevant. Although the method of randomization/allocation to intervention was not clearly stated in 3.2% of all articles which represented all searched RCT articles. A full explanation of the method of randomization and sampling should be mentioned as all inferential statistical techniques are valid only for random samples.\[[@ref19]\]

Unfortunately, incompatibility of statistical test with the type of data examined and the inappropriate use of parametric methods on skewed data was higher than in British and Australian Clinical Articles,\[[@ref16][@ref17]\] and the latter item was higher than others.\[[@ref5][@ref27]\] The use of unpaired tests for paired data was nearly similar to others.\[[@ref5][@ref17]\] The improper use of Pearson\'s Chi-square test instead of the McNemar test was found in analysis of correlated and dependent categorical variables, and this may lead to misleading conclusions and recommendations.\[[@ref31][@ref32]\]

Failure to proof or report that the *t*-test assumptions and not including appropriate multiple comparison α-level correction was lower than in other studies.\[[@ref5][@ref16][@ref17]\] Correcting the alpha level by dividing 0.05 by times of multiple comparisons maintains a "family wise" error rate of 5% likelihood of Type I error.\[[@ref33]\] Most of the errors related to application and reporting regression models in the current study were related to multiple linear regression. The check of the assumptions in regression analysis was not mentioned in most of the articles using multiple linear regression, this error was higher than in another study.\[[@ref18]\] Chi-square was incorrectly used when expected cells \<5 in 8.1% of the articles. These results were similar to the Indian study\[[@ref5]\] and lower than in the articles of New England Journal of Medicine and Chinese articles.\[[@ref16][@ref34]\]

All the errors related to statistical analysis could be due to the use of new statistical software by nonexperts. Hoekstra *et al*., 2012,\[[@ref35]\] set four possible explanations for failing to check for violations of assumptions such as lack of knowledge of the assumptions, methods of checking the assumptions, the problem of possible violation of an assumption, and lack of knowledge of an alternative if an assumption was violated.

Multiple and different deficiencies in documentation of the used statistical methods were nearly similar to others.\[[@ref16][@ref17][@ref27]\] Failure to state number of tails was higher than other studies.\[[@ref16][@ref17]\] Hypothesis tests whether one- or two-sided with *P* value were the most unreported while fail to mention the name of software by which analyzed the data was lower than other study.\[[@ref4][@ref30]\] Deficiencies in documentation mean nonadherence to the guidelines of reporting statistics.

Clear statistics should be reported, either through labels in the table or as a footnote.\[[@ref21]\] Reporting *P* value only without test statistics in at least one table was in 53.3% of the articles. These results were in agreement with the study by Hassan *et al*., 2015.\[[@ref18]\] It is recommended to report observed values of test statistics (e.g., *t*-test, χ²-test) with tabulated values and *P* value.\[[@ref23]\] From the reported observed test statistics, tabulated values and its degrees of freedom, it is possible to compute the observed *P* value with most statistical packages and check the congruence of the results.\[[@ref36]\]

Inappropriate reporting of mean (SD) to describe ordinal/nonnormal data for nonparametric tests was higher than in other studies\[[@ref5][@ref16][@ref18]\] this could be related to no checking of the assumption of normality. No reporting of CI for main effect size measures was lower than in other articles.\[[@ref16][@ref17][@ref18]\] This deficiency could be due to difference in the study designs and the used statistical tests. CIs provide an alternate approach to quantifying the role of chance in research.\[[@ref37]\]

Numerical results and *P* values given to too many (or too few) decimal places were shown in nearly one-tenth of the articles. This error was not detected in other studies.\[[@ref16][@ref17]\] Too many digits clutter a table and make it more difficult for the eye to brain connection to extract the relevant trends.\[[@ref19]\] *P* = nonsignificant (NS) *P* \< 0.05, *P* \> 0.05, etc., instead of reporting exact *P* values was lower than in prestigious journals in other studies.\[[@ref16][@ref17][@ref18]\]

Drawing conclusions not supported by the study data was in a number of the articles were mostly due to the conclusions based on wrong test of significance. Significance claimed without data analysis or statistical test mentioned, and missing discussion of the problem of multiple significance testing was shown in few of the reviewed articles these results varied in other studies.\[[@ref5][@ref16][@ref17][@ref26]\] The variation could be due to difference in skills of interpretations by authors, their statistical background, and ignoring the interpretation of NS results in the examined articles.

The researcher received formal training in statistics and research; a member in FM research continuous quality improvement and had experience in teaching, the assumptions of most common statistical tests and errors in FM research.

Strengths and limitations {#sec2-10}
-------------------------

Strengths: This is the first study about statistics in FM research (Suez Canal University-Egypt) and will provide a base for continuous quality improvement in FM research. Most of the reported statistical errors by this study provide a teaching tool in FM research education. Limitations: The reviewed articles were published in a wide range of medical journals and were not classified in this article into PubMed indexed or not, National or International Journals. Items of study design evaluation in the checklist were more specific to longitudinal studies than those listed in STROBE one, but the checklist was applicable, more comprehensive, and covers many other statistical areas. Although most of FM articles were shared publication with authors from other specialties, some journals/authors did not provide adequate author information to identify the share of statisticians.

Conclusion {#sec1-5}
==========

The use of inferential statistical tests was reported in the majority of FM articles. Omission and inadequate documentation of the statistical methods; failure to mention test statistics in the results with only *P* values and the incorrect use of statistical tests in statistical analysis. Frequency and quality of using statistical methods in FM research articles are nearly comparable to other research articles in different disciplines. This study calls for future education interventions based on the detected statistical errors to improve the quality of statistics in FM research. Adherence to statistical guidelines and review by all professionals, editors, and journals are also recommended.

Financial support and sponsorship {#sec2-11}
---------------------------------

Nil.

Conflicts of interest {#sec2-12}
---------------------

There are no conflicts of interest.

![](JFMPC-5-24-g006.jpg)
