On the Asymptotic Relative Efficiency of Planned Missingness Designs

Abstract

In planned missingness (PM) designs, certain data are set a priori to be missing. PM designs can increase validity and reduce cost; however, little is known about the loss of efficiency that accompanies these designs. The present paper compares PM designs to reduced sample (RN) designs that have the same total number of data points concentrated in fewer participants. In 4 studies, we consider models for both observed and latent variables, designs that do or do not include an “X set” of variables with complete data, and a full range of between- and within-set correlation values. All results are obtained using asymptotic relative efficiency formulas, and thus no data are generated; this novel approach allows us to examine whether PM designs have theoretical advantages over RN designs removing the impact of sampling error. Our primary findings are that (a) in manifest variable regression models, estimates of regression coefficients have much lower relative efficiency in PM designs as compared to RN designs, (b) relative efficiency of factor correlation or latent regression coefficient estimates is maximized when the indicators of each latent variable come from different sets, and (c) the addition of an X set improves efficiency in manifest variable regression models only for the parameters that directly involve the X-set variables, but it substantially improves efficiency of most parameters in latent variable models. We conclude that PM designs can be beneficial when the model of interest is a latent variable model; recommendations are made for how to optimize such a design.

Appendix A: Translating Relative Efficiency into Change in Power

Let $\theta$ be the parameter of interest. We want to test the null $H_0 :\theta \le 0$ against the alternative $H_1:\theta >0$. Let ${\hat{\theta }}_\mathrm{CD}$ be the complete data estimator based on a given $n$. We assume that the sampling distribution of this estimator is approximately $N\left( {\theta ,\frac{1}{n}\mathrm{SE}_{\theta ,\mathrm{CD}}^2}\right)$. Similarly, let ${\hat{\theta }}_\mathrm{PM}$ be the incomplete data estimator (from a PM design) based on a given $n$, with the approximate sampling distribution $N\left( {\theta ,\frac{1}{n}\mathrm{SE}_{\theta ,\mathrm{PM}}^2}\right)$. Without loss of generality, and for simplicity, we assume $n=1$ below (none of the relevant equations depend on $n$). Additionally, we assume for simplicity that standard errors are known, to avoid complicating the expressions with the “hat” notation. The expressions are then approximate and become exact asymptotically. Finally, the most serious but necessary simplifying assumption is that the variance of the estimator does not depend on the population value; i.e., $\mathrm{SE}_{\theta ,\mathrm{CD}}=\mathrm{SE}_\mathrm{CD}$ and $\mathrm{SE}_{\theta ,\mathrm{PM}}=\mathrm{SE}_\mathrm{PM}$ for all values of $\theta$.

The Wald test statistic is given by $z_\mathrm{CD}=\frac{{\hat{\theta }}_\mathrm{CD}}{\mathrm{SE}_\mathrm{CD}}$, and its critical value for a one-tailed test at a given significance level $\alpha$ is $z_\mathrm{CRIT}=\Phi ^{-1}(1-\alpha )$. Suppose the true value is $\theta _0$. Then, power is given by

$$\begin{aligned} \pi _{\theta _0 ,\mathrm{CD}}&= 1-\Pr \left( {z_\mathrm{CD} \le z_\mathrm{CRIT} |\theta =\theta _0 } \right) =1-\Pr \left( {\frac{\hat{{\theta }}_\mathrm{CD} }{\mathrm{SE}_\mathrm{CD} }\le z_\mathrm{CRIT} |\theta =\theta _0 } \right) \\&=1-\Pr \left( {\frac{\hat{{\theta }}_\mathrm{CD} -\theta _0 }{\mathrm{SE}_\mathrm{CD} }\le z_\mathrm{CRIT} -\frac{\theta _0 }{\mathrm{SE}_\mathrm{CD} }|\theta =\theta _0 } \right) =1-\Phi \left( {z_\mathrm{CRIT} -\frac{\theta _0 }{\mathrm{SE}_\mathrm{CD} }} \right) . \end{aligned}$$

It follows that

$$\begin{aligned} 1-\pi _{\theta _0 ,\mathrm{CD}}&= \Phi \left( {z_\mathrm{CRIT} -\frac{\theta _0 }{\mathrm{SE}_\mathrm{CD} }} \right) \nonumber \\ \Phi ^{-1}\left( {1-\pi _{\theta _0 ,\mathrm{CD}} } \right)&= z_\mathrm{CRIT} -\frac{\theta _0 }{\mathrm{SE}_\mathrm{CD} }=\Phi ^{-1}(1-\alpha )-\frac{\theta _0 }{\mathrm{SE}_\mathrm{CD} }\nonumber \\ \mathrm{SE}_\mathrm{CD}&= \frac{\theta _0 }{\Phi ^{-1}(1-\alpha )-\Phi ^{-1}\left( {1-\pi _{\theta _0 ,\mathrm{CD}} } \right) }. \end{aligned}$$

(2)

Similarly, for incomplete data,

$$\begin{aligned} \mathrm{SE}_\mathrm{PM} =\frac{\theta _0 }{\Phi ^{-1}(1-\alpha )-\Phi ^{-1}\left( {1-\pi _{\theta _0 ,\mathrm{PM}} } \right) }. \end{aligned}$$

(3)

The square-root of the (adjusted) relative efficiency (RE or ARE; recall that these are identical for PM designs) is the ratio of (2) and (3), and plugging these expressions in, we obtain

$$\begin{aligned} \sqrt{\mathrm{ARE}_\theta }=\frac{\mathrm{SE}_\mathrm{CD} }{\mathrm{SE}_\mathrm{PM} }=\frac{\Phi ^{-1}(1-\alpha )-\Phi ^{-1}\left( {1-\pi _{\theta _0 ,\mathrm{PM}} } \right) }{\Phi ^{-1}(1-\alpha )-\Phi ^{-1}\left( {1-\pi _{\theta _0 ,\mathrm{CD}} } \right) }. \end{aligned}$$

After some algebraic manipulation, we obtain the final expression for the power of the incomplete data design as a function of relative efficiency and the power of the complete data design:

$$\begin{aligned}&\left[ {\Phi ^{-1}(1-\alpha )-\Phi ^{-1}\left( {1-\pi _{\theta _0 ,\mathrm{CD}} } \right) } \right] \sqrt{\mathrm{ARE}_\theta }=\Phi ^{-1}(1-\alpha )-\Phi ^{-1}\left( {1-\pi _{\theta _0 ,\mathrm{PM}} } \right) \\&\Phi ^{-1}(1-\alpha )-\left[ {\Phi ^{-1}(1-\alpha )-\Phi ^{-1}\left( {1-\pi _{\theta _0 ,\mathrm{CD}} } \right) } \right] \sqrt{\mathrm{ARE}_\theta }=\Phi ^{-1}\left( {1-\pi _{\theta _0 ,\mathrm{PM}} } \right) \\&\pi _{\theta _0 ,\mathrm{PM}} =1-\Phi \left\{ {\Phi ^{-1}(1-\alpha )-\left[ {\Phi ^{-1}(1-\alpha )-\Phi ^{-1}\left( {1-\pi _{\theta _0 ,\mathrm{CD}} } \right) } \right] \sqrt{\mathrm{ARE}_\theta }} \right\} . \end{aligned}$$

Or equivalently,

$$\begin{aligned} \pi _{\theta _0 ,\mathrm{PM}} =1-\Phi \left\{ {\Phi ^{-1}(1-\alpha )(1-\sqrt{\mathrm{ARE}_\theta })+\Phi ^{-1}\left( {1-\pi _{\theta _0 ,\mathrm{CD}} } \right) \sqrt{\mathrm{ARE}_\theta }} \right\} . \end{aligned}$$