Statistical tests are used to test the fit between a hypothesis and the data. Choosing the right statistical test is not a trivial task. The choice of the test depends on many properties of the research question. The vast majority of studies can be addressed by 30 of the 100 or so statistical tests in use.
Explanation of properties
Scaling of data: One of the properties of the tests is the scale of the data, which can be interval-based, ordinal or nominal. Nominal scale is also known as categorical. Interval scale is also known as numerical. When categorical data has only two possibilities, it is called binary or dichotomous.
Assumptions, parametric and non-parametric: There are two groups of statistical tests, parametric and non-parametric. The choice between these two groups needs to be justified. Parametric tests assume that the data follow a particular distribution, typically a normal distribution, while non-parametric tests make no assumptions about the distribution. Non-parametric tests have the advantage of being more resistant to misbehaviour of the data, such as outliers. They also have the disadvantage of being less certain in the statistical estimate.
Type of data: Statistical tests use different types of data. Some tests perform univariate analysis on a single sample with a single variable. Others compare two or more paired or unpaired samples. Unpaired samples are also called independent samples. Paired samples are also called dependent. Finally, there are some statistical tests that perform analysis of relationship between multiple variables like regression.
Number of samples: The number of samples of data.
Exactness: A test can be exact or be asymptotic delivering approximate results.
List of statistical tests
+ Test name Scaling Assumptions Data Samples Exact Special case of Application conditions One sample t-test interval normal univariate 1 No Location test Paired difference test paired 2 Location test Unpaired t-test interval normal unpaired 2 No Location test Homoscedasticity Welch's t-test interval normal unpaired 2 No Location test Paired t-test interval normal paired 2 No Location test F-test interval normal unpaired 2 Z-test (one mean) interval normal univariate 1 No variance is known Z-test (two means) interval normal 2 No variances are known Permutation test interval non-parametric unpaired ≥2 Yes Kruskal-Wallis test ordinal non-parametric unpaired ≥2 Yes small sample size Mann–Whitney test ordinal non-parametric unpaired 2 Kruskal-Wallis test Wilcoxon signed-rank test interval non-parametric paired ≥1 Location test Sign test ordinal non-parametric paired 2 Friedman test ordinal non-parametric paired >2 Location test test nominal non-parametric No Contingency table, sample size > ca. 60, any cell content ≥ 5, marginal totals fixed Pearson's test nominal/ordinal non-parametric No test Median test ordinal non-parametric No Pearson's test Multinomial test nominal non-parametric univariate 1 Yes Location test McNemar's test binary non-parametric paired 2 Yes Cochran's test Cochran's test binary non-parametric paired ≥2 Binomial test binary non-parametric univariate 1 Yes Multinomial test Siegel–Tukey test ordinal non-parametric unpaired 2 Chow test interval parametric linear regression 2 No Time series Fisher's exact test nominal non-parametric unpaired ≥2 Yes Contingency table, marginal totals fixed Barnard's exact test nominal non-parametric unpaired 2 Yes Contingency table Boschloo's test nominal non-parametric unpaired 2 Yes Contingency table Shapiro–Wilk test interval univariate 1 Normality test sample size between 3 and 5000 Kolmogorov–Smirnov test interval 1 Normality test distribution parameters known Shapiro-Francia test interval univariate 1 Normality test Simplification of Shapiro–Wilk test Lilliefors test interval 1 Normality test
See also List of probability distributions
References