SEM APPROACH FOR THE DETECTION OF RESPONSE SHIFT EXAMPLE SYNTAX LISREL ! We use an example of a three-factor model with on nine indicators, measured at two occasions ! S1.t1, S2.t1, and S3.t1 are three measures of social health measured at time 1 ! S1.t2, S2.t2, and S3.t2 are the same three measures of social health measured at time 2 ! M1.t1, M2.t1, and M3.t1 are three measures of mental health measured at time 1 ! M1.t2, M2.t2, and M3.t2 are the same three measures of mental health measured at time 2 ! P1.t1, P2.t1, and P3.t1 are three measures of physical health measured at time 1 ! P1.t2, P2.t2, and P3.t2 are the same three measures of physical health measured at time 2 ! The data is captured in a [covariance matrix], [means vector], and [number of observations] STEP 1 : ESTABLISHING A MEASUREMENT MODEL ! Specification of the data ! ni = number of observed variables da ni=18 no=[number of observarions] ma=cm cm fi=[covariance matrix] me fi=[means vector] LA S1.t1 S2.t1 S3.t1 M1.t1 M2.t1 M3.t1 P1.t1 P2.t1 P3.t1 S1.t2 S2.t2 S3.t2 M1.t2 M2.t2 M3.t2 P1.t2 P2.t2 P3.t2 ! Specifications of the model-matrices mo ny=18 ne=6 ly=fu,fr ps=sy,fr te=sy,fr al=fu,fi ty=fu,fr LE SOCIAL.T1 MENTAL.T1 PHYSICAL.T1 SOCIAL.T2 MENTAL.T2 PHYSICAL.T2 ! SPECIFICATION OF COVARIANCE STRUCTURE ! matrix of common factor loadings ! all free to be estimated (no restrictions over time) pa ly 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 ! matrix of common factor (co)variances ! includes longitudinal relations ! identification is done through fixing the factor variances at 1 (at both occasions) ma ps 1 0 1 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 pa ps 0 1 0 1 1 0 1 1 1 0 1 1 1 1 0 1 1 1 1 1 0 ! matrix of residual (co)variances ! includes covariances of the same residual factors over time pa te 1 0 1 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 ! SPECIFICATION OF MEAN STRUCTURE ! vector of common factor means ! identification through fixing common factor means at zero (at both occasions) pa al 0 0 0 0 0 0 ! vector of intercepts pa ty 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 pd ou ml mi so ! -> When model has adequate fit and is interpretable, continue with STEP2 STEP 2 : OVERALL TEST OF RESPONSE SHIFT ! Specification of the data da ni=18 no=[number of observarions] ma=cm cm fi=[covariance matrix] me fi=[means vector] LA S1.t1 S2.t1 S3.t1 M1.t1 M2.t1 M3.t1 P1.t1 + P2.t1 + P3.t1 S1.t2 S2.t2 S3.t2 M1.t2 M2.t2 M3.t2 P1.t2 + P2.t2 + P3.t2 ! Specifications of the model-matrices mo ny=18 ne=6 ly=fu,fr ps=sy,fr te=sy,fr al=fu,fr ty=fu,fr LE SOCIAL.T1 MENTAL.T1 PHYSICAL.T1 SOCIAL.T2 MENTAL.T2 PHYSICAL.T2 ! SPECIFICATION OF COVARIANCE STRUCTURE ! matrix of common factor loadings ! all factor loadings are constrained to be equal across occasions using the 'eq' command pa ly 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 eq ly 1 1 ly 10 4 eq ly 2 1 ly 11 4 eq ly 3 1 ly 12 4 eq ly 4 2 ly 13 5 eq ly 5 2 ly 14 5 eq ly 6 2 ly 15 5 eq ly 7 3 ly 13 6 eq ly 8 3 ly 14 6 eq ly 9 3 ly 15 6 ! matrix of common factor (co)variances ! identification is done through fixing the factor variances at 1, only at the first occasion ma ps 1 0 1 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 pa ps 1 1 1 1 1 1 1 1 1 0 1 1 1 1 0 1 1 1 1 1 0 ! matrix of residual (co)variances pa te 1 0 1 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 ! SPECIFICATION OF MEAN STRUCTURE ! vector of common factor means ! identification through fixing the common factor means at zero, only at the first occasion ma al 0 0 0 0 0 0 pa al 0 0 0 1 1 1 ! vector of intercepts ! all intercepts are constrained to be equal across occasions using the 'eq' command pa ty 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 eq ty 1 ty 10 eq ty 2 ty 11 eq ty 3 ty 12 eq ty 4 ty 13 eq ty 5 ty 14 eq ty 6 ty 15 eq ty 7 ty 16 eq ty 8 ty 17 pd ou ml mi so ! -> Compare the model fit of the model from STEP2 with the model fit of the model from STEP1 ! -> When the omnibus test for response shift is significant, this indicates the (overall) presence of response shift ! -> Continue with STEP3 to identify specific indications of response shift STEP 3 : DETECTION OF RESPONSE SHIFT ! -> Use an iterative procedure to modify the model from STEP2 (where all possible response shifts are considered one at a time) to identify specific response shift effects. ! -> Test whether response shift is significant ! -> Look at parameter estimates to interpret detected response shift ! Calculate effect-size for impact of response shift on change in the observed variable ! -> See Verdam, Oort, & Sprangers, 2017 (JCE, 85, 37-44) STEP 4: TRUE CHANGE ASSESSMENT ! -> In the final model from STEP3: Look at the parameter estimates of the common factor means at the second occasions (i.e. SOCIAL.T2, MENTAL.T2 and PHYSICAL.T2) ! -> Calculate effect-size for the estimated true change using Cohen's d: mean_post - mean_pre / sqrt(variance_post + variance_pre - 2correlation_postpre*sd_post*sd_pre) ! -> To evaluate the impact of response shift on true change, compare these estimates to the same estimates of the 'no response shift model' from STEP2