Home > E-book list > GLM Univariate

Univariate GLM is the general linear model now often used to implement such long-established statistical procedures as regression and members of the ANOVA family. It is "general" in the sense that one may implement both regression and ANOVA models. One may also have fixed factors, random factors, and covariates as predictors. Also, in GLM one may have multiple dependent variables, as discussed in a separate section on multivariate GLM and one may have linear transformations and/or linear combinations of dependent variables. Moreover, one can apply multivariate tests of significance when modeling correlated dependent variables, not relying on individual univariate tests as in multiple regression. GLM also handles repeated measures designs. Finally, because GLM uses a generalized inverse of the matrix of independent variables' correlations with each other, it can handle redundant independents which would prevent solution in ordinary regression models.

Data requirements. In all GLM models, the dependent(s) is/are continuous. The independents may be categorical factors (including both numeric and string types) or quantitative covariates. Data are assumed to come from a random sample for purposes of significance testing. The variance(s) of the dependent variable(s) is/are assumed to be the same for each cell formed by categories of the factor(s) (this is the homogeneity of variances assumption).

Regression in GLM is simply a matter of entering the independent variables as covariates and, if there are sets of dummy variables (ex., Region, which would be translated into dummy variables in OLS regression, for ex., South = 1 or 0), the set variable (ex., Region) is entered as a fixed factor with no need for the researcher to create dummy variables manually. The b coefficients will be identical whether the regression model is run under ordinary regression (in SPSS, under Analyze, Regression, Linear) or under GLM (in SPSS, under Analyze, General Linear Model, Univariate). Where b coefficients are default output for regression in SPSS, in GLM the researcher must ask for "Parameter estimates" under the Options button. The R-square from the Regression procedure will equal the partial Eta squared from the GLM regression model.

The advantages of doing regression via the GLM procedure are that dummy variables are coded automatically, it is easy to add interaction terms, and it computes eta-squared (identical to R-squared when relationships are linear, but greater if nonlinear relationships are present). However, the SPSS regression procedure would still be preferred if the reseacher wishes output of standardized regression (beta) coefficients, wishes to do multicollinearity diagnostics, or wishes to do stepwise regression or to enter independent variables hierarchically, in blocks. PROC GLM in SAS has a greater range of options and outputs (SAS also has PROC ANOVA, but it handles only balanced designs/equal group sizes).

The full content is now available from Statistical Associates Publishers. Click here.

```GLM UNIVARIATE: GENERAL LINEAR MODELS
Overview	11
Key Concepts	15
Why testing means is related to variance in analysis of variance	15
One-way ANOVA	16
Simple one-way ANOVA in SPSS	16
Simple one-way ANOVA in SAS	20
Two-way ANOVA	23
Two-way ANOVA in SPSS	24
Two-way ANOVA in SAS	27
Multivariate or n-way ANOVA	29
Regression models	29
Parameter estimates (b coefficients) for factor levels	31
Parameter estimates for dichotomies	32
Significance of parameter estimates	32
Research designs	32
Between-groups ANOVA design	32
Completely randomized design	34
Full factorial ANOVA	34
Balanced designs	35
Latin square designs	36
Graeco-Latin square designs	37
Randomized Complete Block Design (RCBD ANOVA)	37
Split plot designs	39
Mixed design models	39
Random v. fixed effects models	41
In SPSS	41
In SAS	42
Linear mixed models (LMM) vs. general linear models (GLM)	43
Effects	43
Treating a random factor as a fixed factor	44
Mixed effects models	44
Nested designs	44
Nested designs	45
In SPSS	46
In SAS	49
Treatment by replication design	49
Within-groups (repeated measures) ANOVA designs	49
Counterbalancing	50
Reliability procedure	51
Repeated measures GLM in SPSS	51
Repeated measures GLM in SAS	51
Interpreting repeated measures output	52
Variables	53
Types of variables	53
Dependent variable	53
Fixed and random factors	54
Covariates	54
WLS weights	54
Models and types of effects	55
Full factorial models	55
Effects	56
Main effects	56
Interaction effects	56
Residual effects	59
Effect size measures	60
Effect size coefficients based on percent of variance explained	60
Partial eta-squared	60
Omega-squared	61
Herzberg's R2	62
Intraclass correlation	62
Effect size coefficients based on standardized mean differences	62
Cohen's d	62
Glass's delta	64
Hedge's g	65
Significance tests	65
F-test	65
Example 1	66
Example 2	66
Significance in two-way ANOVA	67
Computation of F	67
F-test assumptions	67
Lack of fit test	68
Power level and noncentrality parameter	69
Hotelling's T-Square	70
Planned multiple comparison t-tests	70
Simple t-test difference of means	72
Sidak test	74
Dunnett's test	74
HSU's multiple comparison with the best (MCB) test	74
Post-hoc multiple comparison tests	74
The q-statistic	75
Output formats: pairwise vs. multiple range	76
Tests assuming equal variances	76
Least significant difference (LSD) test	76
The Fisher-Hayter test	77
Tukey's test, a.k.a. Tukey honestly significant difference (HSD) test	78
Tukey-b test, a.k.a. Tukey's wholly significant difference (WSD) test	79
S-N-K or Student-Newman-Keuls test	80
Duncan test	81
Ryan test (REGWQ)	81
The Shaffer-Ryan test	83
The Scheffé test	83
Hochberg GT2 test	85
Gabriel test	87
Waller-Duncan test	87
Tests not assuming equal variances	87
Tamhane's T2 test	87
Games-Howell test	88
Dunnett's T3 test and Dunnett's C test	89
The Tukey-Kramer test	89
The Miller-Winer test	89
More than one multiple comparison/post hoc test	89
Example	89
Contrast tests	91
Overview	91
Types of contrasts	92
Deviation contrasts	92
Simple contrasts	92
Difference contrasts	92
Helmert contrasts	92
Repeated contrasts	92
Polynomial contrasts	93
Custom hypothesis tables	93
Custom hypothesis tables index table	93
Custom hypothesis tables	94
Estimated marginal means	96
Overview	96
EMM Estimates table	98
Other EMM output	101
EMM Pairwise comparisons table	101
EMM Univariate tests table	101
Profile plots	101
GLM Repeated Measures	102
Overview	102
Key Terms and Concepts	103
Within-subjects factor	103
Repeated measures dependent variables	104
Between-subjects factors	105
Covariates	105
Models	106
Type of sum of squares	107
Balanced vs. unbalanced models	107
Estimated marginal means	108
Pairwise comparisons	109
Statistics options in SPSS	110
Descriptive statistics	110
Hypothesis SSCP matrices	111
Partial eta-squared	111
Within-subjects SSCP matrix and within-subjects contrast effects.	112
Multivariate tests.	113
Univariate vs. multivariate models	114
Box's M test	115
Mauchly's test of sphericity	115
Univariate tests of within-subjects effects	116
Parameter estimates	118
Levene's test	119
Residual plots	120
Lack of fit test	122
General estimable function	122
Post hoc tests	122
Overview	122
Profile plots for repeated measures GLM	125
Example	125
Contrast analysis for repeated measures GLM	127
Types of contrasts for repeated measures	128
Simple contrasts example	129
Saving variables in repeated measures GLM	130
Cook's distance	131
Leverage values	131
Assumptions	132
Interval data	132
Homogeneity of variances	132
Homogeneity of variance	133
Appropriate sums of squares	137
Multivariate normality	138
Equal or similar sample sizes	139
Random sampling	139
Orthogonal error	140
Data independence	140
Recursive models	140
Categorical independent variables	140
The independent variable is or variables are categorical.	140
Continuous dependent variables	140
Non-significant outliers	140
Sphericity	141
Assumptions related to ANCOVA:	142
Limited number of covariates	142
Low measurement error of the covariate	142
Covariates are linearly related or in a known relationship to the dependent	142
Homogeneity of covariate regression coefficients	143
No covariate outliers	143
No high multicollinearity of the covariates	144
Assumptions for repeated measures	144
How do you interpret an ANOVA table?	146
Isn't ANOVA just for experimental research designs?	148
Should I standardize my data before using ANOVA or ANCOVA?	148
Since orthogonality (uncorrelated independents) is an assumption, and since this is rare in real life topics of interest to social scientists, shouldn't regression models be used instead of ANOVA models?	148
Couldn't I just use several t-tests to compare means instead of ANOVA?	148
How does counterbalancing work in repeated measures designs?	149
How is F computed in random effect designs?	150
What designs are available in ANOVA for correlated independents?	150
If the assumption of homogeneity of variances is not met, should regression models be used instead?	151
Is ANOVA a linear procedure like regression? How is linearity related to the "Contrasts" option?	151
What is hierarchical ANOVA or ANCOVA?	151
Is there a limit on the number of independents which can be included in an analysis of variance?	152
Which SPSS procedures compute ANOVA?	152
I have several independent variables, which means there are a very large number of possible interaction effects. Does SPSS have to compute them all?	152
Do you use the same designs (between groups, repeated measures, etc.) with ANCOVA as you do with ANOVA?	152
How is GLM ANCOVA different from traditional ANCOVA?	153
What are paired comparisons (planned or post hoc) in ANCOVA?	153
Can ANCOVA be modeled using regression?	153
How does blocking with ANOVA compare to ANCOVA?	153
What is the SPSS syntax for GLM repeated measures?	154
What is a "doubly repeated measures design"?	155
Bibliography	156
Pagecount: 160
```