Test 2 Practice Questions

Use these questions to practise for Test 2. They cover the same topic blocks as the test: heterogeneous treatment effects, policy trees and judgement, outcome-wide reporting, and measurement.

Write short answers without notes first. Then check your answer against the relevant lecture, lab, Reporting Guide, and Test 2 Study Sheet.

Heterogeneous Treatment Effects (Week 8)

  1. State the difference between the average treatment effect and the conditional average treatment effect (CATE). When is the distinction substantively important?
  2. A regression model with treatment-by-age and treatment-by-gender interactions reports both interactions as non-significant. Does this rule out heterogeneous treatment effects? Why or why not?
  3. Explain honest splitting in a causal forest. What problem is it designed to prevent?
  4. What does it mean for an estimator to be doubly robust? Give one practical advantage.
  5. A causal forest reports RATE-AUTOC = 0.04 with a 95% CI of $[0.01, 0.07]$. State, in plain language, what this tells you about heterogeneity. Compare with RATE-AUTOC = 0.04 with CI $[-0.02, 0.10]$.
  6. Sketch a Qini curve. Label the axes. Describe the shape of the curve when targeting helps and when it does not.
  7. Why do we expect causal forest estimates of $\hat{\tau}(x)$ to be noisy in regions of the covariate space with little data?
  8. A reviewer says "If the heterogeneity is real, a regression with the right interaction terms will find it." Reply in two sentences.

Policy Trees, Fairness, and Judgement (Week 9)

  1. State, in one or two sentences, what a policy tree does that a conditional average treatment effect estimate alone does not. Your answer should use the idea of utility or policy value over allocation rules.
  2. Explain the parsimony rule for choosing tree depth. Why is the simpler rule often preferable even when the more complex rule has a slightly larger estimated utility or policy value?
  3. A depth-2 policy tree treats the leaf "deprivation index $> 1.2$ and age $\leq 40$". Translate this into a sentence a community organiser could repeat.
  4. Name three things you would check before using a policy tree to allocate a programme.
  5. Explain how a split on deprivation can affect social groups differently even when group membership does not appear in the tree.
  6. State, in one or two sentences, why statistical evidence cannot by itself decide whether an allocation rule is just.
  7. Describe one scenario in which a community organiser should override a policy tree recommendation. What information would they have that the model does not?
  8. State the difference between a ranking rule (for example, treat the top 20% by $\hat{\tau}$ when a budget is fixed) and the course policy-tree workflow, which fits depth-1 and depth-2 candidates and chooses depth-2 only if it clears the prespecified gain threshold. In your answer, distinguish estimating CATEs from estimating the utility of allocation rules. Give one scenario in which the ranking is preferable.

Outcome-Wide Reporting

  1. State the four causal estimands an outcome-wide design implies for one exposure and four outcomes.
  2. Why is multiple-testing correction necessary in an outcome-wide design? Describe the Bonferroni correction at $\alpha_{FW} = 0.05$ for four outcomes.
  3. Explain, in plain language, what an E-value of 2.0 means.
  4. A forest plot orders four outcomes by effect magnitude. The largest two effects survive Bonferroni; the smaller two do not. Write a one-paragraph interpretation suitable for a non-specialist audience.

Measurement (Week 10)

  1. Briefly distinguish reflective and formative measurement models. State one reason each is awkward for causal inference.
  2. Define measurement invariance. Describe how scalar invariance can fail across cultures, and why this threatens cross-cultural causal comparisons.
  3. Why is "mindfulness intervention" vulnerable to the multiple-versions-of-treatment problem? What does this imply for consistency?
  4. Draw a causal directed acyclic graph with $A$, $Y^\ast$, $Y$, and $U_Y$ representing differential measurement error. Explain how the path $A \to U_Y \to Y$ threatens identification.
  5. Explain why including the baseline measurement of the outcome ($Y_0$) in the adjustment set provides strong confounding control in a three-wave panel.
  6. State one design response and one analytic response when measurement invariance fails across groups.

Synthesis Questions

  1. An investigator estimates the effect of weekly volunteer work on four wellbeing outcomes using a causal forest. Set out the full causal workflow: estimands, identification assumptions, estimator, multiple-testing correction, sensitivity analysis, presentation.
  2. A community wellbeing programme wants to use heterogeneous treatment effect estimates. Walk through the steps from conditional average treatment effect estimation to a readable policy-tree summary. Explain that policy learning estimates utility over allocation rules, name the fairness check, and state one value judgement the model cannot settle. State how the answer would change if the programme also had a fixed budget.
  3. A cross-cultural study uses the K6 to compare the effect of a school-based mindfulness intervention in two countries. Identify the measurement, treatment-version, and identification threats, and propose one response to each.
  4. A reviewer challenges your outcome-wide forest plot with: "Three of these confidence intervals cross zero after Bonferroni. Are you sure your story holds?" Reply in one paragraph.