Home > E-book list > Significance Testing: Parametric & Nonparametric

Overview A statistical significance coefficient is the chance that a relationship as strong or stronger than the one observed was due to the chance of random sampling. Thus if a correlation coefficient is significant at exactly the .05 level, this means there is 5% chance that a correlation as strong or stronger than the observed one would result from an unusual random sampling of data when in fact the correlation was zero. There are many, many specific significance tests. Common tests are listed below, but in addition each statistical procedure has associated significance tests which are discussed in the respective Statistical Associates "Blue Book" volumes dealing with each procedure.

The full content is now available from Statistical Associates Publishers. Click here.

```SIGNIFICANCE TESTING
Overview 10
Significance Testing	15
Overview	15
Types of significance tests	15
Parametric tests	15
Key Concepts and Terms	16
When significance testing applies	16
Significance and Type I Errors	19
Confidence limits	19
Power and Type II Errors	20
One-tailed vs. two-tailed tests	20
Assumptions of significance testing in general	22
Random sampling	22
Significance is not importance	22
À priori testing	23
Appropriate alpha significance level	23
Absence of intervening and common anteceding causes	23
How is significance related to effect size?	24
Should I "fail to accept" or should I "reject" the null hypothesis?	24
If cases in my sample are weighted, am I getting accurate significance values?	24
Is significance the same for multistage random samples as for simple random samples?	24
Binomial Test of Significance	26
Overview	26
Key Concepts and Terms	26
Implementing the binomial test in SPSS	27
Selections	27
Normal approximation of the binomial test	30
Assumptions of the binomial test	31
Dichotomous distribution	31
Data distribution	31
Random sampling	31
Student's t-Test of Difference of Means	32
Overview of the t-test	32
SPSS t-test types	32
Key Concepts and Terms	33
Formula	33
Critical value	33
Confidence limits	34
One-sample t-test	34
Example	34
Interpretation	35
Independent sample t-test	35
Example	36
Interpretation (with Levene's test)	36
The independent samples assumption	37
Paired sample t-test for non-independent samples	37
Overview	37
Example	38
Interpretation	38
Assumptions for t-tests	39
Normal distribution	39
Random sampling	39
Similar variances	39
Dependent/independent samples	40
Effect size measures	40
What are common alternatives to the t-test?	40
What non-parametric test do I use instead of the t-test if my data cannot meet the assumption of normality?	41
Normal Curve Tests of Means and Proportions	42
Overview	42
Key Concepts and Terms	42
Deviation scores	42
Standard deviation	42
Variance	43
Standard error	43
Confidence limits	43
Binomial distribution	44
Normal distribution	44
Normal curve means tests ("hypothesis tests")	46
Confidence interval	47
Manual computation of z values for normal curve tests	49
Assumptions for normal curve tests	50
Interval data	51
Sample size should not be small	51
Homogeneity of variances	51
Random sampling	51
Chi-Square Significance Tests	52
Overview	52
Pearson's chi-square	52
Overview	52
SPSS output	53
Yates correction for continuity	56
Crosstabulation control variables and chi-square	57
Chi-square goodness-of-fit test	58
Likelihood ratio chi-square	61
Mantel-Haenszel (linear by linear) chi-square	62
Assumptions for chi-square tests	63
Random sampling	63
Independence	64
Known distribution	64
Non-directional hypotheses	64
Finite values	64
Normal distribution of deviations	64
Data level	64
How is Pearsonian chi-square calculated manually for tabular data?	65
My statistics program prints out the chi-square contribution of each cell in the table. Can this be used to establish the significance of each cell?	66
Fisher Exact Test of Significance	67
Overview	67
Key Concepts and Terms	67
Notation	67
Manual computation of Fisher's exact test	68
Assumptions of Fisher's exact test	69
Random sampling	69
Directionality	69
Independent observations	70
Dichotomous level of measurement	70
Fixed marginals	70
Frequently asked questions regarding the Fisher exact test	70
What is Tocher's modification of the Fisher exact test?	70
Is the two-tailed test statistic for Fisher's exact test simply double the one-tailed value?	71
Runs Test of Randomness	72
Overview	72
Key Concepts and Terms	72
Runs	72
Runs test	72
Runs test for serial randomness	73
Cut points	73
Example	73
The data	73
Requesting the runs test	74
Runs test output in SPSS	76
SPSS legacy dialog features	77
Assumptions of the runs test	80
Data order	80
Numeric data	80
Data level	80
Data distribution	80
One-Sample Kolmogorov-Smirnov  Goodness-of-Fit Test	82
Overview	82
Key Concepts and Terms	82
Observed vs. hypothetical distribution	82
Example	84
What is tested	84
SPSS set-up	84
Statistical output for the Kolmogorov-Smirnov test in SPSS	86
Kolmogorov-Smirnov test assumptions	88
Random sampling	88
Level of data	88
Could I use the Kolmogov-Smirnov goodness-of-fit test to test my data not only against the normal distribution, but also against a series of other distributions?	89
Where is the Kolmogorov-Smirnov exact test found in SPSS?	89
Could goodness-of-fit be assessed graphically instead of by the Kolmogorov-Smirnov test?	90
Tests for two independent samples:	91
The Mann-Whitney U test	91
Overview	91
Key Concepts and Terms	91
Independent samples	91
Data level	91
Computation of Mann-Whitney U	91
A manual example	92
SPSS setup for the Mann-Whitney U test	93
Example	93
SPSS user interface	93
SPSS statistical output for the Mann-Whitney U test	95
The hypothesis test summary table	95
The SPSS model viewer	96
Wilcoxon rank sum W	97
Assumptions of the Mann-Whitney U test	97
Data level	97
Independent samples	97
Random sampling	98
Data pairs	98
Sample size	98
Frequently asked questions: Mann-Whitney U test	98
Where is the Mann-Whitney U exact test found in SPSS?	98
Tests for two independent samples:	100
The Wald-Wolfowitz runs test	100
Overview	100
SPSS inputs for the Wald-Wolfowitz runs test	100
SPSS output for the Wald-Wolfowitz runs test	102
The hypothesis test summary table	102
The model viewer	103
Frequently asked questions: Wald-Wolfowitz runs test	104
Where is the Wald-Wolfowitz exact test found in SPSS?	104
Assumptions of the Wald-Wolfowitz runs test	105
Data level	105
Independent samples	105
Random sampling	105
Data pairs	106
Data distribution	106
Sample size	106
Wald-Wolfowitz median test	106
SPSS inputs for the median test	106
SPSS output for the independent samples median test	107
Tests for two independent samples:	111
The Kolmogorov-Smirnov Z test	111
Overview	111
Computation	111
SPSS inputs for the independent samples Kolmogorov-Smirnov test	111
SPSS output for the independent samples Kolmogorov-Smirnov test	112
The hypothesis test summary table	112
The SPSS model viewer	113
Assumptions of the Kolmogorov-Smirnov test	114
Data level	114
Random sampling	114
Data pairs	115
Data distribution	115
Sample size	115
Frequently asked questions: Kolmogorov-Smirnov test	115
Where is the Kolmogorov-Smirnov exact test found in SPSS?	115
Tests for two independent samples:	117
Moses Extreme Reactions Test	117
Overview	117
Computation	117
Example	118
SPSS inputs for the Moses test	118
SPSS output for the Moses test	120
The hypothesis summary table	120
The SPSS model viewer	121
Assumptions of the Moses extreme reactions test	123
Control group	123
Data level	124
Random sampling	124
Data pairs	124
Data distribution	124
Sample size	124
Frequently asked questions: Moses extreme reactions test	124
Where is the Moses extreme reactions exact test found in SPSS?	124
Tests for more than two independent samples: The Kruskal-Wallis H test	126
Overview	126
Computation	126
Example	127
SPSS input for the Kruskal-Wallis H test	127
SPSS output for the Kruskal-Wallis H test	129
The hypothesis summary table	129
The SPSS model viewer	130
Pairwise comparisons	131
Frequently asked questions: Kruskal-Wallis test	132
Where is the Kruskal-Wallis exact test found in SPSS?	132
Assumptions of the Kruskal Wallis H test	134
Data level	134
Data pairs	135
Data distribution	135
Sample size	135
Random sampling	135
Tests for more than two independent samples: The multiple samples median test	136
Overview	136
Computation	136
Example	136
SPSS input for the median test	136
SPSS output for the median test	138
The hypothesis summary table	138
The SPSS model viewer	139
Pairwise comparisons	140
Frequently asked questions: median test	141
Where is the median exact test found in SPSS?	141
Assumptions of the median test	142
Random sampling	143
Central tendency	143
Data level	143
Data pairs	143
Data distribution	143
Sample size	143
Tests for more than two independent samples: The Jonckheere-Terpstra test	144
Overview	144
Example	144
SPSS input for the Jonkheere-Terpstra test	145
SPSS output for the Jonkheere-Terpstra test	146
The hypothesis summary table	146
The SPSS model viewer	147
Pairwise comparisons	148
Frequently asked questions: Jonkheere-Terpstra test	149
Where is the Jonkheere-Terpstra exact test found in SPSS?	149
Assumptions of the Jonkheere-Terpstra test	151
Random sampling	151
Data level	152
Data distribution	152
Sample size	152
Significance tests for two dependent samples:  The McNemar test	153
Overview	153
Key Concepts and Terms	154
Dependent samples	154
Pairs	154
Similar distributions	154
Symmetry	154
Computation	155
Significance tests for two dependent samples:  The marginal homogeneity test	167
Overview	167
Example	167
SPSS input for the marginal homogeneity test	167
SPSS output for the marginal homogeneity test	169
The hypothesis summary table	169
The SPSS model viewer	170
Frequently asked questions: Marginal homogeneity test	172
Where is the Marginal homogeneity exact test found in SPSS?	172
Assumptions of the Marginal homogeneity test	173
Data level	173
Data distribution	174
Sample size	174
Random sampling	174
Significance tests for two dependent samples:  The sign test	175
Overview	175
Computation	175
Example	175
SPSS input for the sign test	176
SPSS output for the sign test	178
The hypothesis summary table	178
The SPSS model viewer	179
Frequently asked questions: Sign test	180
Where is the sign exact test found in SPSS?	180
Assumptions of the Sign test	182
Data level	182
Data distribution	182
Sample size	183
Random sampling	183
Significance tests for two dependent samples:  The Wilcoxon signed-rank test	184
Overview	184
Computation	184
Example	184
SPSS input for the Wilcoxon signed-rank test	185
SPSS output for the Wilcoxon signed-rank test	187
The hypothesis summary table	187
The SPSS model viewer	188
Frequently asked questions: Wilcoxon signed-rank test	190
Where is the sign exact test found in SPSS?	190
Assumptions of the Wilcoxon signed-rank test	191
Data level	192
Data distribution	192
Sample size	192
Random sampling	192
Significance tests for two independent or dependent samples:  Hodges-Lehman confidence intervals	193
Overview	193
Example	193
Frequently asked questions: the Hodges-Lehman procedure	194
Is the Hodges-Lehman procedure available in the legacy dialog of SPSS?	194
Is there additional SPSS output for the Hodges-Lehman procedure in the SPSS model viewer?	194
Significance Tests for K Dependent Samples: Friedman Test	195
Overview	195
Computation	195
Example	196
SPSS input for the Friedman test	196
SPSS output for the Friedman test	197
The SPSS model viewer	198
Frequently asked questions: Friedman test	201
Where is the Friedman exact test found in SPSS?	201
Assumptions of the Friedman test	202
Significance Tests for K Dependent Samples: Kendall's W test	204
Overview	204
Computation	204
Example	204
SPSS input for Kendall's W test	204
SPSS output for Kendall's W test	206
The SPSS model viewer	207
Frequently asked questions: Kendall's W test	210
Where is Kendall's W exact test found in SPSS?	210
Assumptions of Kendall's W test	211
Significance Tests for K Dependent Samples: Cochran's Q test	213
Overview	213
Computation	213
Example	213
SPSS input for Cochran's Q test	214
SPSS output for Cochran's Q test	215
The SPSS model viewer	216
Frequently asked questions: Cochran's Q test	219
Where is Cochran's Q exact test found in SPSS?	219
Assumptions of Cochran's Q test	220
Dependent samples	220
Data level	221
Bootstrapping and Resampling	222
Overview	222
Key Concepts and Terms	223
Bootstrapping	223
Resampling	223
Monte Carlo estimation	223
Histogram of z values	224
Bootstrapping in SPSS	224
Overview	224
Exact tests	226
SPSS syntax (command) mode	227
Assumptions	228
Random sampling	228
Nonparametric	229
Sample size	229