05a Testing Model Fit 2005

Testing Model Fit

SOC 681

Limitations of Fit Indices

• Values of fit indices indicate only the average or overall fit of a model. It is thus possible that some parts of the model may poorly fit the data.

• Becauwe a single index reflects only a particul;ar aspect oof model fit, a favorable value of that

Limitations of Fit Statistics

• Fit indices do not indicate whether the results are theoretically meaningful.

• Values of fit indices that suggest adequate fit do not indicate the predictive power of the model is also high.

Assessment of Model Fit

• Examine the parameter estimates

• Examine the standard errors and significance of the parameter estimates.

• Examine the squared multiple correlation coefficients for the equations

• Examine the fit statistics

Measures of Fit

• Measures of fit are provided for three models: – Default Model – this is the model that you specified – Saturated Model – This is the most general model

possible. No constraints are placed on the population moments It is guaranteed to fit any set of data perfectly. – Independence Model – The observed variables are

GPA

HEIGHT

WEIGHT

RATING

ACADEMIC

ATTRACT

Overall measures of Fit

• NPAR is the number of parameters being estimated (q)

• CMIN is the minimum value of the discrepancy function between the sample covariance matrix and the estimated covariance matrix.

• DF is the number of degrees of freedom and equals the p-q

– p=the number of sample moments

Overall measures of Fit

• CMIN is distributed as chi square with df=p-q

• P is the probability of getting as large a discrepancy with the present sample

• CMIN/DF is the ratio of the minimum

Chi Square:



2

• Best for models with N=75 to N=100

• For N>100, chi square is almost always significant since the magnitude is

affected by the sample size

Chi Square to df Ratio:



2

/df

• There are no consistent standards for what is considered an acceptable

model

• Some authors suggest a ratio of 2 to 1 • In general, a lower chi square to df ratio

Transforming Chi Square to Z

CMIN

Model NPAR CMIN DF P CMIN/DF

Default model 22 10.335 14 .737 .738 Saturated model 36 .000 0

Independence

RMR, GFI

• RMR is the Root Mean Square Residual. It is the square root of the average amount that the sample variances and covariances differ from their estimates. Smaller values are better

• GFI is the Goodness of Fit Index. GFI is

GFI and AGFI

(LISREL measures)

• The AGFI takes into consideration the df available to test the model.

• Values close to .90 reflect a good fit.

• These indices are affected by sample size and can be large for poorly specified models. • These are usually not the best measures to

RMR, GFI

• AGFI is the Adjusted Goodness of Fit

Index. It takes into account the degrees of freedom available for testing the model. Acceptable values are above 0.90.

RMR, GFI

Model RMR GFI AGFI PGFI

Default model .003 .975 .935 .379 Saturated model .000 1.000

Independence

Comparisons to a Baseline Model

• NFI is the Normed Fit Index. It compares the improvement in the minimum

discrepancy for the specified (default) model to the discrepancy for the

Bentler-Bonett Index or

Normed Fit Index (NFI)

• Define null model in which all correlations are zero:

2 (Null Model) - 2 (Proposed Model) 2

(Null Model)

• Value between .90 and .95 is acceptable; above .95 is good

Comparisons to a Baseline Model

• RFI is the Relative Fit Index This index takes the

degrees of freedom for the two models into account. • IFI is the incremental fit index. Values close to 1.0

indicate a good fit.

• TLI is the Tucker-Lewis Coefficient and also is

known as the Bentler-Bonett non-normed fit index (NNFI). Values close to 1.0 indicate a good fit.

• CFI is the Comparative Fit Index and also the

Tucker Lewis Index or

Non-normed Fit Index (NNFI)

• Value: 2/df(Null Model) - 2/df(Proposed Model)

2/df(Null Model)

• If the index is greater than one, it is set to1. • Values close to .90 reflects a good model fit. • For a given model, a lower chi-square to df

Comparative Fit Index (CFI)

• If D= 2 - df, then:

D(Null Model) - D(Proposed Model) D(Null Model)

• If index > 1, it is set to 1; if index <0, it is set to 0 • A lower value for D implies a better fit

• If the CFI < 1, then it is always greater than the TLI • The CFI pays a penalty of one for every parameter

Baseline Comparisons

Model Delta1NFI

RFI

Default model .958 .915 1.016 1.034 1.000 Saturated model 1.000 1.000 1.000 Independence

Parsimony Adjusted Measures

• PRATIO is the Parsimony Ratio. It is the number of constraints in the model being evaluated as a fraction of the number of constraints in the independence model.

• PNFI is the result of applying the PRATIO to the NFI.

Parsimony-Adjusted Measures

Model PRATIO PNFI PCFI Default model .500 .479 .500 Saturated model .000 .000 .000 Independence

Measures Based on the Population

Discrepancy

• NCP is an estimate of the noncentrality parameter obtained by fitting a model to the population moments rather than to the sample moments.

NCP

Model NCP LO 90 HI 90

Default model .000 .000 7.102 Saturated model .000 .000 .000 Independence

The Minimum Sample Discrepancy

Function

FMIN

Model FMIN F0 LO 90 HI 90 Default model .107 .000 .000 .073 Saturated model .000 .000 .000 .000 Independence

Root Mean Square Error of

• Good models have values of < .05; values of > .10 indicate a poor fit.

• It is a parsimony-adjusted measure.

PCLOSE

RMSEA

Model RMSEA LO 90 HI 90 PCLOSE Default model .000 .000 .072 .877 Independence

Information Theoretic Measures

• These indices are composite measures of badness of fit and complexity.

• Simple models that fit well receive low

scores. Complicated poorly fitting models get high scores.

• These indices are used for model

AIC

Model AIC BCC BIC CAIC

Default model 54.335 58.835 111.204 133.204 Saturated model 72.000 79.364 165.059 201.059 Independence

Akaike Information Criterion

(AIC)

• Value: 2 + k(k-1) - 2(df)

where k= number of variables in the model • A better fit is indicated when AIC is smaller • Not standardized and not interpreted for a

given model.

Information Theoretic Measures

• BCC is the Browne-Cudeck Criterion • BIC is Bayes Information Criterion. • CAIC is the consistent AIC

• ECVI except for a constant scale factor it is the same as AIC.

ECVI

Model ECVI LO 90 HI 90 MECVI Default model .560 .598 .671 .607 Saturated model .742 .742 .742 .818 Independence

Diminished SES

Neurological Dysfunction

Low Morale

Illness Symptoms

Poor

Difference in Chi Square

Value: X

2

diff

= X

2 model 1

-X

2 model 2

Diminished SES

Neurological Dysfunction

Low Morale

Illness Symptoms

Poor

Diminished SES

Neurological Dysfunction

Low Morale

Illness Symptoms

Poor

Miscellaneous Measures

• HOELTER is the largest sample size for

Hoelter Index

• Value: (N-1)*2(crit) + 1 2

Where 2 (crit) is the critical value for the chi-square statistic

Hoelter Index (2)

• If the critical value is unknown, can approximate: [ (1.645 + (2df-1) ]2 + 1 2 2/ (N-1)

• For both formulas, one rounds down to the nearest integer

Hoelter Index (3)

• In other words, how small one’s sample size would have to be for chi square to no longer be significant

• Hoelter Recommends values of at least 200

HOELTER

Model HOELTER.05 HOELTER.01

Default model 223 274

Independence

Which Fit Indices to Report?/

• Chi Square and df • RMSEA

• CFI • AGFI

• Hierarchical Models: Difference in Chi Square