Significance Tests for Covariates in LCA and LTA
I am performing LCA and wondered how to test the significance of my covariates. I understand that I need a test statistic and its corresponding degrees of freedom (df) to perform the test, but I don’t know how to get this information from my output. ––Signed, Feeling Insignificant
A significance test can be helpful for judging whether there is sufficient evidence to state that a particular covariate is related to latent class membership. Significance tests provide p-values, which tell us how likely we would be to observe a relation between a covariate and latent class membership as strong as, or stronger than, the one in our sample, if in fact there is no relation between the two. A small p-value (say, p<.05) is generally taken as a relation being “statistically significant,” i.e., as evidence that a covariate is in fact related to latent class membership in the population. Follow these steps to test for statistical significance:
- Fit a model with the covariate of interest; save the loglikelihood as 'alt_loglike.'
- Fit a model with the covariate removed; save the loglikelihood as 'null_loglike.'
- Compute the likelihood ratio test statistic: 'delta_2ll = 2(alt_loglike – null_loglike).'
- Determine the df for the likelihood ratio test by determining how many additional parameters are estimated in Step 1 compared to Step 2. This is easily done by counting up the number of additional regression coefficients estimated in Step 1 compared to Step 2. Because the intercept coefficients are estimated in both the null and the alternative models, they do not need to be counted; that is, only the “slope parameters” need to be counted.
For example, for LCA with multinomial logistic regression, df = (number of latent classes – 1) * (number of groups). This is because the model with the covariate (in Step 1) estimates an additional slope parameter, compared to the model without the covariate (in Step 2), for the effect of the covariate on each latent class compared to a reference latent class, within each group. For LCA with binary logistic regression, df = (number of groups). This is because the model with the covariate estimates one additional slope parameter, compared to the model without the covariate, for the effect of the covariate on a target latent class compared to all other latent classes combined for the reference, within each group. Remember that in LTA many more slope parameters are estimated when you predict transitions; be sure to count them all.
- Evaluate the cumulative distribution function of a chi-square distribution for the value 'delta_2ll' with the corresponding df. This is the p-value for that covariate. You can use an online calculator for this computation.
For example, there is a high-quality online calculator at https://www.fourmilab.ch/rpkp/experiments/analysis/chiCalc.html. Using this calculator, you would put in 'delta_2ll' for X2 and 'df' for d. The calculator then reports the p-value for you.
Good luck in your research!