Psych 434: Conducting Research Across Cultures

Conducting Research About Cultures

Author
Affiliation

Joseph Bulbulia

Victoria University of Wellington, New Zealand

Published

February 25, 2025

Course Outline

Week 1 - Course Introduction - Introduction to R

Focus

  • Introduce course objectives and outline
  • R setup

Lab

  • Getting started with R/R-studio: installation and package management
Readings
  • No Readings

Week 2 - Causal Diagrams: Five Elementary Causal Structures

Focus

  • Understanding causal diagrams: definitions and applications
  • Introduction to five elementary structures and four rules in causal inference
  • Introduction to R interface and data simulation

Readings

Lab

  • Practical exercises in R: Using the interface and simulating data

Week 3 - Causal Diagrams: The Structures of Confounding Bias

Focus

  • Confounding bias using causal diagrams
  • Application of regression and simulation in R

Lab

  • Practical exercises in R: regression and ggdag

Readings

Optional Readings

Week 4 - Causal Diagrams: The Structures of Interaction/Effect Modification, Measurement Bias, Selection Bias

Focus

  • Key concepts of interaction, measurement bias, and selection bias understood through causal diagrams
  • Both External and Internal Validity clarified by Causal Graphs
  • Advanced regression and simulation exercises in R

Lab

  • Continuation of regression and simulation exercises in R

Readings

Optional Readings

Week 5 - Causal Inference: Average Treatment Effects

Focus

  • Key concepts of Average Treatment Effect (ATE)
  • Application of regression and simulation in R to obtain ATE estimation

Readings

Extra Readings

Lab

  • Regression and simulation exercises in R focussed on estimating the ATE

Week 6 - Causal Inference and Effect Modification

Optional Readings

Focus

  • Effect Modification: Definine your Causal Estimand
  • Distinguishing Cultural Effect-Modification from the confused and conflated concepts of “Moderation”, “Mediation”, “Interaction.”
  • Detour into Causal Mediation

Lab

  • Analysis step 1: data wrangling and descriptive tables/graphs

Week 7 - IN CLASS TEST (25%)

Focus

  • Assessment covering key terms and concepts taught so far.

Week 8 - Causal Inference: ESTIMATING Marginal Structural Models; Inverse Probability of Treatment Weighting; Conditional Average Treatment Effects; IPTW when Groups are Compared.

Focus

  • Workflow for causal question formulation, population statement, and causal diagram creation
  • Marginal Structural Models: propensity scores and Inverse Probability of Treatment Weighting (IPTW)
  • IPTW when estimating conditional causal effects
  • Estimation techniques evaluating evidence for group-wise effect modification using R.

Readings

Optional Readings

Lab

  • Estimation ATE; CATE

Week 9 - Hands On Analysis

Focus

  • Second assessment covering advanced topics in causal inference
  • Topics include ATE, Effect-Modification, fundamental assumptions of causal inference, experiments, and real-world confounding

Lab

  • Preparing your analysis: Hands On Study!

Week 10 - Hands On Working With Quarto Manuscript

  • No readings, do your take-home assignment (see course details).

Lab

  • Creating and managing Quarto documents for publication quality research workflows

Week 11 - Measurement Matters

Part 1: Classical Measurement Models

Focus

  • Factor analysis, confirmatory factor analysis (CFA), multigroup CFA, partial invariance
  • Worked example on configural, metric, and scalar equivalence

Readings

Optional Readings

Part 2 Problems With Classical Measurement Models: External Validity in Causal Inference

Focus

  • Understanding causal assumptions of measurement theory
  • Guidance on your final assessment.

Readings

  • [Tyler J. VanderWeele (2022)][link](https://www.dropbox.com/scl/fi/mmyguc0hrci8wtyyfkv6w/tyler-vanderweele-contruct-measures.pdf?rlkey=o18fiyajdqqpyjgssyh6mz6qm&dl=0)

  • [Bulbulia (2024d)][link](https://www.cambridge.org/core/journals/evolutionary-human-sciences/article/methods-in-causal-inference-part-3-measurement-error-and-external-validity-threats/4D35FFDECF32B2EFF7557EC26075175F)

Suggested Readings

Week 12: Student Presentations

References

Appendix Optional Videos

Johannas Karl on Getting Started In R

Richard McElreath on Causal Inference

Miguel Hernan: How to We learn What Works

Tyler VanderWeele on Measurement Contructs

  • Abstract. Psychosocial constructs can only be assessed indirectly, and measures are typically formed by a combination of indicators that are thought to relate to the construct. Reflective and formative measurement models offer different conceptualizations of the relation between the indicators and what is sometimes conceived of as a univariate latent variable supposedly corresponding to the construct. I argue that the empirical implications of these models will often be violated by data since the causally relevant constituents will generally be multivariate, not univariate. In fact, the assumption of an underlying univariate structural latent variable is so strong that it has empirically testable implications, even though the latent is unobserved. Formal statistical tests can be developed to reject this assumption, but factor analysis, as typically practiced, is not adequate to do so. Factor analysis also suffers from the inability to distinguish associations arising from causal versus conceptual relations. I put forward an outline for a new model of the process of measure construction and propose a causal interpretation of associations between constructed measures and subsequent outcomes that is applicable even if the usual assumptions of reflective and formative models fail. I discuss the practical implications of these observations and proposals for the provision of definitions, the selection of items, item-by-item analyses, the construction of measures, and the causal interpretation of regression analyses.

Stijn Vansteelandt on the Problem of Mediation Analysis

Stijn Vansteelandt and Besty Ogburn on Causal Inference (Workflows)

Abstract: Causal inference research has shifted from being primarily descriptive (describing the data-generating mechanism using statistical models) to being primarily prescriptive (evaluating the effects of specific interventions). The focus has thereby moved from being centered on statistical models to being centered on causal estimands. This evolution has been driven by the increasing need for practical solutions to real-world problems, such as designing effective interventions, making policy decisions, and identifying effective treatment strategies. It has brought enormous progress, not solely in terms of delivering more useful answers to the scientific questions at stake, but also in providing a more hygienic inference that targets a well-understood causal estimand. However, many causal questions are not readily translated into the effects of specific interventions, and even if they can, scientists may be reliant on help from an expert statistician to make that translation, may not find the considered interventions feasible or of immediate interest, or may find too little information in the data about the considered estimand. In this talk, I will reflect on this and argue that hygienic causal inference thinking therefore comes with a price. I will next propose a compromise solution at the intersection of descriptive and prescriptive causal inference. It borrows the flexibility of statistical modeling, while tying model parameters to causal estimands in order to ensure that we understand what is being estimated and obtain valid (data-adaptive) inference for it, even when the model is wrong. Examples on structural (nested) mean models, instrumental variables estimation, target trials, … will be used to provide insight.

References

Bulbulia, J. A. 2024a. “A Practical Guide to Causal Inference in Three-Wave Panel Studies.” OSF. https://doi.org/10.31234/osf.io/uyg3d.
———. 2024b. “Methods in Causal Inference Part 1: Causal Diagrams and Confounding.” Evolutionary Human Sciences 6: e40. https://doi.org/10.1017/ehs.2024.35.
———. 2024c. “Methods in Causal Inference Part 2: Interaction, Mediation, and Time-Varying Treatments.” Evolutionary Human Sciences 6: e41. https://doi.org/10.1017/ehs.2024.32.
———. 2024d. “Methods in Causal Inference Part 3: Measurement Error and External Validity Threats.” Evolutionary Human Sciences 6: e42. https://doi.org/10.1017/ehs.2024.33.
Fischer, Ronald, and Johannes A Karl. 2019. “A Primer to (Cross-Cultural) Multi-Group Invariance Testing Possibilities in r.” Frontiers in Psychology, 1507.
Greifer, Noah. 2023. WeightIt: Weighting for Covariate Balance in Observational Studies.
Harkness, J. [et al]. 2003. “Questionnaire Translation.” In Cross-Cultural Survey Methods, 35–56. NJ: Wiley.
Harkness, Janet A, Fons JR Van de Vijver, and Timothy P Johnson. 2003. “Questionnaire Design in Comparative Research.” Cross-Cultural Survey Methods, 19–34.
He, Jia, and Fons van de Vijver. 2012. “Bias and Equivalence in Cross-Cultural Research.” Online Readings in Psychology and Culture 2 (2). https://doi.org/10.9707/2307-0919.1111.
Hernan, M. A., and J. M. Robins. 2024. Causal Inference: What If? Chapman & Hall/CRC Monographs on Statistics & Applied Probab. Taylor & Francis. https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/.
Hernán, M. A. 2017. “Invited Commentary: Selection Bias Without Colliders | American Journal of Epidemiology | Oxford Academic.” American Journal of Epidemiology 185 (11): 1048–50. https://doi.org/10.1093/aje/kwx077.
Hernán, Miguel A., and Stephen R. Cole. 2009. “Invited Commentary: Causal Diagrams and Measurement Bias.” American Journal of Epidemiology 170 (8): 959–62. https://doi.org/10.1093/aje/kwp293.
Hernán, Miguel A., Sonia Hernández-Díaz, and James M. Robins. 2004. “A Structural Approach to Selection Bias.” Epidemiology 15 (5): 615–25. https://www.jstor.org/stable/20485961.
Hoffman, Katherine L., Diego Salazar-Barreto, Kara E. Rudolph, and Iván Díaz. 2023. “Introducing Longitudinal Modified Treatment Policies: A Unified Framework for Studying Complex Exposures,” April. https://doi.org/10.48550/arXiv.2304.09460.
Neal, Brady. 2020. “Introduction to Causal Inference from a Machine Learning Perspective.” Course Lecture Notes (Draft). https://www.bradyneal.com/Introduction_to_Causal_Inference-Dec17_2020-Neal.pdf.
Suzuki, Etsuji, Tomohiro Shinozaki, and Eiji Yamamoto. 2020. “Causal Diagrams: Pitfalls and Tips.” Journal of Epidemiology 30 (4): 153–62. https://doi.org/10.2188/jea.JE20190192.
VanderWeele, Tyler J. 2009. “On the Distinction Between Interaction and Effect Modification.” Epidemiology, 863–71.
VanderWeele, Tyler J. 2022. “Constructed Measures and Causal Inference: Towards a New Model of Measurement for Psychosocial Constructs.” Epidemiology 33 (1): 141. https://doi.org/10.1097/EDE.0000000000001434.
VanderWeele, Tyler J., and Miguel A. Hernán. 2012. “Results on Differential and Dependent Measurement Error of the Exposure and the Outcome Using Signed Directed Acyclic Graphs.” American Journal of Epidemiology 175 (12): 1303–10. https://doi.org/10.1093/aje/kwr458.
VanderWeele, Tyler J, Maya B Mathur, and Ying Chen. 2020. “Outcome-Wide Longitudinal Designs for Causal Inference: A New Template for Empirical Studies.” Statistical Science 35 (3): 437–66.
VanderWeele, Tyler J., and James M. Robins. 2007. “Four types of effect modification: a classification based on directed acyclic graphs.” Epidemiology (Cambridge, Mass.) 18 (5): 561–68. https://doi.org/10.1097/EDE.0b013e318127181b.
Vijver, Fons J. R. van de, Kwok Leung, Velichko H. Fetvadjiev, Jia He, and Johnny R. J. Fontaine. 2021. Edited by Velichko H. Fetvadjiev, Jia He, and Johnny R. J. Fontaine. 2nd ed. Culture and Psychology 116. Cambridge: Cambridge University Press. https://doi.org/10.1017/9781107415188.

Reuse