• Why randomize? \(\checkmark\)

• Standard randomized designs \(\checkmark\)

• How to analyze data from randomized trials \(\times\)

  • Deviations from randomization

  • Adjusting for covariates

• Primary analysis for many trials is often direct comparison of outcome in treatment groups, i.e. t-test, log-rank test, permutation test. Secondary analyses may adjust for specific covariates

• Question: Patients enroll and are randomized. Some deviate from assignment and move to the other arm. Do you (i) analyze using randomized assignment (say \(T_R\)) or (ii) actual treatment (\(T\))?

• Recall that we estimate \(E[O_i(A)|T_i=A]-E[O_i(B)|T_i=B]\) and implicitly equate to \(E[O_i(A)-O_i(B)]\)

• Suppose there is no \(T\)-\(O\) association in truth. In intended DAG, \(T_R\equiv T\). Does it matter which of these we use?

  • \(E[O_i(A)|T_i=A]-E[O_i(B)|T_i=B]\)

  • \(E[O_i(A)|T_{iR}=A]-E[O_i(B)|T_{iR}=B]\)

• In actual DAG, \(T_R\) and \(T\) may differ. Now does it matter which we use?

\[ \begin{aligned} &E[O_i(A)|T_i=A]-E[O_i(B)|T_i=B] \\ &= E[U_i|T_i~=~A]-E[U_i|T_i=B] \\ &\neq 0 \end{aligned} \]

(because of indirect path through \(U\)) \[ \begin{aligned} &E[O_i(A)|T_{iR}=A]-E[O_i(B)|T_{iR}=B]\\ &= E[U_i|T_{iR}=A]-E[U_i|T_{iR}=B] \\ &= 0 \end{aligned} \]

• 'As-Treated' analysis may undo the effects of randomization if there is significant deviation from protocol

• 'Intent-to-Treat' analysis is appropriate one. Formally, want to test for effect of being randomized to treatment rather than effect of treatment itself

• Reasons to consider stratifying randomization based upon covariate values:

1. Pre-specified subgroup analyses

2. Negative public perception of large imbalances

   • Probability of large overall treatment imbalance decreases with sample size, but chance imbalances on specific covariates still likely

3. Increased precision (?)

1. Identify one or two strong prognostic factors

2. Create separate strata defined by these prognostic factors

3. Within each stratum, conduct separate, independent permuted block randomization

Block size \(b=4\); after 2 patients

Section 4.3, Rosenberger and Lachin (2004)

Block size \(b=4\); after 3 patients

Block size \(b=4\); after 4 patients

Block size \(b=4\); after 5 patients

Block size \(b=4\); after 10 patients

Block size \(b=4\); after 13 patients

Block size \(b=4\); after 30 patients

• Subsequent analysis should then adjust for strata, e.g.

  • Stratified Mantel-Haenszel, log-rank tests
  • Putting covariates that defined strata directly in model
  • Random intercept for each stratum
  • Individual baseline hazards (for time-to-event)

• Question: when analyzing data from randomized trial (not necessarily stratified randomization), should analysis be adjusted for baseline factors?

• Two potential analysis models:

  1. \(O = \alpha^* + \delta^*1_{[T=A]} + \beta V + U\); \(U|\{T,V\} \sim N(0,\sigma^2)\)

  2. \(O = \alpha + \delta 1_{[T=A]} + U\); \(U|T \sim N(0,\tau^2)\)

• Which to use?

• Question 1: When can both models hold, i.e. both satisfy all assumptions for linear regression?

• Question 2: Assuming both models hold, when are treatment effects the same, i.e. \(\delta^* = \delta\)?

• Question 3: When both models hold, when is there gain in precision (smaller variance) by adjusting for \(V\)?

• From model 1: \(E[O|T] = \alpha^* + \delta^*1_{[T=A]} + \beta E[V|T]\), which matches regression mean according to model 2 when \(E[V|T] = \mu + \gamma 1_{[T=A]}\), for some \(\mu\) and \(\gamma\), so that \(\alpha = \alpha^* + \beta\mu\) and \(\delta = \delta^* + \beta\gamma\)

• And because \(\tau^2=\mathrm{var}[O|T] = \beta^2 \mathrm{var}[V|T] + \sigma^2\), matches constant variance according to model 2 when \(\mathrm{var}[V|T] = \omega^2\), so that \(\tau^2 = \beta^2 \omega^2 + \sigma^2\)

Thus, answer to Q1 is when \(V|T \sim N(\mu + \gamma 1_{[T=A]}, \omega^2)\), i.e. when \(V\) given \(T\) is normal with mean equal to linear function of \(T\) and constant variance

• In that case, \[ \begin{aligned} E[O|T] &= \alpha^* + \delta^*1_{[T=A]} + \beta (\mu + \gamma 1_{[T=A]})\\ &= (\alpha^* + \beta\mu) + (\delta^* + \beta\gamma) 1_{[T=A]}\\ \end{aligned} \]

• So, answer to Q2 is \(\delta^* = \delta\) when \(\beta=0\) or \(\gamma=0\)

• For Q3, we compare \(\mathrm{var}(\hat\delta^*)\) to \(\mathrm{var}(\hat\delta)\)

• For brevity, we'll use \(T=1\) as equivalent to \(1_{[T=A]}\). Then, \[ \begin{aligned} \mathrm{var}(\hat\delta) = \tau^2\begin{pmatrix} 1 & ET\\ ET & ET^2 \end{pmatrix}^{-1}_{[2,2]} = \dfrac{\beta^2\omega^2 + \sigma^2}{\mathrm{var}[T]} \end{aligned} \]

\[ \begin{aligned} \mathrm{var}(\hat\delta^*) &= \sigma^2\begin{pmatrix} 1 & ET & EU \\ ET & ET^2 & ETV \\ EU & ETV & EV^2 \end{pmatrix}^{-1}_{[2,2]} \\ & = ... \\ & = \dfrac{\mathrm{var}[V] \sigma^2}{\mathrm{var}[V]\mathrm{var}[T] - \mathrm{cov}^2(T,V)} \end{aligned} \]

• Then, asymptotic relative efficiency of \(\hat\delta^*\) to \(\hat\delta\) is

\[ \begin{aligned} \dfrac{\mathrm{var}(\hat\delta)}{\mathrm{var}(\hat\delta^*)}& = \dfrac{\beta^2\omega^2 + \sigma^2}{\mathrm{var}[T]}\dfrac{\mathrm{var}[V]\mathrm{var}[T] - \mathrm{cov}^2(T,V)}{\mathrm{var}[V] \sigma^2}\\ & = \dfrac{\beta^2\omega^2 + \sigma^2}{\sigma^2}\dfrac{\mathrm{var}[V]\mathrm{var}[T] - \mathrm{cov}^2(T,V)}{\mathrm{var}[V]\mathrm{var}[T]}\\ & = \dfrac{\beta^2\omega^2 + \sigma^2}{\sigma^2}\left(1- \mathrm{cor}^2(T,V)\right)\\ & = \dfrac{1- \mathrm{cor}^2(T,V)}{1 - \dfrac{\beta^2\omega^2}{\beta^2\omega^2 + \sigma^2}} \end{aligned} \]

• \(\dfrac{\beta^2\omega^2}{\beta^2\omega^2 + \sigma^2}\) resembles 'coefficient of variation' or proportion of variance explained, \(R^2\):

\[ \begin{aligned} \dfrac{\beta^2\omega^2}{\beta^2\omega^2 + \sigma^2} = \dfrac{\mathrm{var}(\alpha^* + \delta^*T + \beta V|T)}{\mathrm{var}(\alpha^* + \delta^* T + \beta V + U|T)} \end{aligned} \]

• This is proportion of variance of \(O\) explained by \(V\) after conditioning on \(T\)

• Often write \(\sqrt{\dfrac{\beta^2\omega^2}{\beta^2\omega^2 + \sigma^2}} = \mathrm{cor}(V,O|T)\), the partial correlation between \(V\) and \(O\) given \(T\)

\[ \begin{aligned} \dfrac{\mathrm{var}(\hat\delta)}{\mathrm{var}(\hat\delta^*)} = \dfrac{1- \mathrm{cor}^2(T,V)}{1 - \mathrm{cor}^2(V,O|T)} \end{aligned} \]

• If \(\mathrm{cor}(V,O|T)=0\), meaning \(V\) and \(O\) have no linear association once \(T\) is accounted for, then \(\dfrac{\mathrm{var}(\hat\delta)}{\mathrm{var}(\hat\delta^*)} = 1- \mathrm{cor}^2(T,V) \leq 1\)

• Adjusting for \(V\) when has no prognostic ability beyond treatment will tend to increase variance of test statistic

• If \(\mathrm{cor}(T,V)=0\), meaning \(T\) is unrelated to \(V\) (encouraged by proper randomization) \(\dfrac{\mathrm{var}(\hat\delta)}{\mathrm{var}(\hat\delta^*)} = (1- \mathrm{cor}^2(V,O|T))^{-1} \geq 1\)

• Adjusting for prognostic covariate will decrease variance of test statistic

• Note also that, related to Q2,
  • \(\mathrm{cor}(V,O|T) = 0 \Leftrightarrow \beta=0\)
  • \(\mathrm{cor}(T,V) = 0 \Leftrightarrow \gamma=0\)
• Thus

• In logistic regression, adding other covariates always increases variance of \(\hat\delta\). However, \(\hat\delta\) may be less biased (Robinson and Jewell, 1991)

• Similar but less definitive findings for time-to-event outcomes (Chastang, Byar and Piantadosi, 1988)

• (from strongest to most tentative)

1. Pre-specify analyses: state what primary analysis, secondary analysis will be before trial starts. Be clear where reported analyses deviate from planned analyses

2. Stratify only on few, strong prognostic covariates. However, if multi-center trial, center should always be stratification factor

3. Do not add covariates only to improve precision

4. Statistically acceptable to ignore imbalances occuring by chance; may not be palatable to broader research community

• Randomization key to making causal claims: properly conducted, ensures there is no unmeasured confounding, but cannot address external validity concerns, e.g. Simulation 5 preferentially enrolled patients having larger treatment benefit, over-estimated population-average treatment effect. Under-estimate of treatment effect also possible

• Restricted randomization used to encourage balance between treatment groups

• Chance of large imbalance decreases with sample size; bias due to unmeasured confounding does not