Lecture I (2025-10-30): Statistical Reflections

In this lecture, we review statistical fundamentals – such as the origins of the t-test, the meaning of type-I and type-II error (and alternative terminology for both, such as false positive rate and false negative rate) and the connection to statistical power (sensitivity). We review the Receiver Operating Characteristic (ROC) curve and give a qualitative description of where it gets its shape in a hypothesis test. We close with a validation example (from Lecture H) where we use a power analysis on a one-sample t-test to help justify whether we have gathered enough data to trust that a simulation model is a good match for reality when it has a similar mean output performance to the real system. Peppered throughout the lecture are also comments about why normality is required for t-tests, why there is a minimum expected count for chi-squared tests, and how to avoid statistical inference issues when making multiple comparisons.