Week 4: Interaction, Measurement Bias, and Selection Bias

Seminar

Learning outcomes

By the end of this week, you will be able to:

1. Distinguish effect modification from confounding and from interaction.
2. Identify the four types of measurement error bias in a causal diagram.
3. Explain how selection bias arises from conditioning on a collider at baseline.
4. Define the distinction between a target population and a source population, and explain why it matters for transportability.

Effect modification

Effect modification occurs when the causal effect of $A$ on $Y$ differs across strata of a third variable $V$. In our DAG conventions, effect modification is indicated by a blue arrow with a circle point from $V$ to the $A\to Y$ path. Importantly, effect modification is not a source of bias. It is a feature of the world: some treatments work differently for different people.

Effect modification differs from confounding. A confounder is a common cause of $A$ and $Y$ that must be conditioned on to eliminate bias. An effect modifier is a variable that changes the magnitude (or direction) of the causal effect. We do not adjust for effect modifiers to remove bias; we examine them to understand for whom the treatment works.

Effect modification also differs from statistical interaction, although the two are related. Interaction is a property of a statistical model (e.g., a product term $A\times V$ in a regression). Effect modification is a causal property of the data-generating process. A statistical interaction may reflect effect modification, confounding, or an artefact of the scale on which the outcome is measured. Distinguishing these requires a causal diagram.

Common causal questions presented as causal graphs

The following figure shows how several standard research designs can be represented as causal DAGs. Each design answers a different causal question, and the DAG makes explicit what is being estimated and what assumptions are required.

A typology of measurement error bias

Measurement error is pervasive in psychology. We rarely observe the true value of a variable; instead, we observe a measured version that may differ from the truth. When measurement error is systematic rather than purely random, it can introduce bias into our causal estimates. The following figure presents four structural types of measurement error, classified along two dimensions.

Understanding these types matters because the standard advice to "use reliable measures" addresses only the simplest case (independent, uncorrelated error). The other three types require structural reasoning: you need a DAG to determine whether measurement error in your study is a source of bias and, if so, in which direction.

Selection bias and transportability

Selection bias arises when the composition of the study sample differs systematically from the target population in ways that affect the treatment-outcome relationship. The most common structural source of selection bias is conditioning on a collider at baseline.

Consider a study of the effect of exercise on blood pressure. Suppose health-conscious people are more probable to both exercise and volunteer for health studies, and suppose high socioeconomic status (SES) independently predicts both better health and study participation. Participation is a collider of health consciousness and SES. By restricting analysis to study participants, we condition on this collider and induce a spurious association between health consciousness and SES, which can distort the estimated effect of exercise on blood pressure.

The acronym "WEIRD" (Western, Educated, Industrialised, Rich, Democratic) has drawn attention to the narrow demographic base of much psychological research. From a causal perspective, the problem is not that WEIRD samples are unrepresentative per se, but that effect modifiers may be distributed differently in the target population. If the treatment effect varies across levels of an effect modifier, and that modifier's distribution differs between sample and target, the sample ATE will not transport to the target. The simulation guide for this course demonstrates this point concretely using a transportability simulation.

Measurement invariance (which we cover in Week 10) connects to measurement error bias in cross-cultural research. If the same questionnaire measures a construct differently in different populations, then what appears to be a cross-population difference in the outcome may instead be a difference in how the outcome is measured. This is a form of correlated, directed measurement error, and it cannot be detected by standard reliability statistics.

Lab materials: Lab 4: Regression and Confounding Bias