Our Winter Quarter at Northwestern just ended, and I want to write about a new class I taught, how it went, and why I think we should be teaching more courses like this in statistics and data science.

The Course

The course was called “STAT359: Social Statistics, Science, and Society” and was a special topics course (though I hope to make it into a formal standalone course).

The overview:

This interdisciplinary course examines the critical role that statistics plays in understanding social phenomena and informing public policy. The course begins with a foundation in the history of social statistics, the sociology of science, and ethical frameworks that guide research practice. From this historical and theoretical grounding, students will examine contemporary issues in data collection and analysis, many of which are made increasingly complex by the integration of artificial intelligence. Key topics include ethical problems in clinical trials, government data collection practices, and the use and misuse of statistical evidence in policy decisions. Special attention is given to issues of bias, representation, and ethical considerations in data analysis, particularly regarding marginalized populations and algorithmic decision-making. The course culminates with presentations of student-selected research projects.

I taught the course because I wanted to talk about these things and had no idea if and who would enroll - in the end, I had a class of about 24 undergraduate junior and senior statistics and/or data science majors/minors, 2 stats PhD students, and 4 graduate students from social science departments.

The class was entirely reading and discussion based, with a tiny bit of lecturing by me (when there were areas where I thought students needed extra background). I structured the course so that the first few classes focused on foundations that could carry us through the course, including: (1) ethical frameworks broadly in science and society; (2) ethical frameworks specific to statistics and data science; (3) the “average” man concept of Quetelet and the beginning of social statistics; (4) the rise of eugenics (Galton, Pearson, Fisher) and its connections to statistics; (5) the role that statistics plays in the social sciences, including concepts like “cargo cult” statistics; (6) the use of statistical/data science algorithms in society, including the history of credit scores and the SAT. From there, we moved to case studies. I put together two: (1) the role of the Census and Federal Statistical Agencies; (2) Algorithmic fairness. Students in the course were required to write term papers on a topic of their choosing - I encouraged them to pick topics relevant to their lives and interests. Most of the seniors had jobs lined up already - in tech, AI, finance, banking, data science - and many chose topics related to the industries they were going into. From their proposals, I grouped students into topic areas, and they led the final 7 class sessions, finding papers to read and developing discussion questions. These ended up including topics: (1) Who is in research and questions of generalizability; (2) Surveillance and algorithmic exploitation; (3) Race and algorithmic bias; (4) Measuring and classifying children (focused on IQ testing); (5) The quantification process: How measures are created and manipulated; (6) The replication crisis; (7) The politicization of statistics and its use in state formation.

Why this approach

I have long thought that we need to be teaching and talking about ethics with our statistics students, but this feels more urgent to me as our field shifts towards training students to work in the development of algorithms in the technologies of our daily lives. But talking about this as ‘ethics’ alone feels too abstract to me, too philosophical. As you can see in this course, my strategy was to combine: Ethics + History + Sociology as a way to approach this:

1. The ethical foundations provide guidelines and frameworks for how to approach what is ‘good’. By showing that there are competing frameworks, it helps students see tensions here, but also these provide frameworks that can be carried forward all quarter.

2. The history of statistics topics - average man and the origin of the normal distribution - help students see that statistical developments and the social world have been intertwined from the beginning and how the politics of those developing these methods informed how they played out in society - i.e., the issues we are seeing with AI are not new.

3. The sociology of statistics topics - how statistics are used in social science research and in social applications - provide ways for students to think about the difference between how statistics is optimally used (how it is conceived by statisticians) and how it is used (by real people) and how this difference can lead to problems.

These three frameworks then provided the grounding for all case studies in the remainder of the course.

How it went

This class really blew me away. At the beginning of the quarter, it was clear that many of these students hadn’t really had an opportunity to think critically about the tools they were being equipped to use in the world, the power they wield, and how they have the potential for harm. But over the course of the quarter, I could see that they were deeply engaged with these questions, wrestling with what this would mean for their own work in the world. Their final papers indicated this even more - these papers included deep dives into topics regarding pharmaceutical company practices to how algorithms affect markets to issues of statistical decision making gone wrong.

This is not to say this was an easy lift. I’m used to teaching traditional stats classes - I haven’t taught a discussion-based class before, and it’s been decades since I even was in one. I asked around to get ideas and even asked Claude.ai for ideas for how to do this. Ultimately, the rhythm we got into for an 80-minute class was about this: Intro, little lecture, Discussion #1, lecture, Discussion #2, lecture, Discussion #3, wrap up. Discussions were about 10 minutes in groups of 5 students (next time I’d do better about mixing students up), with 5 minutes of full class synthesis. This meant that I needed to prepare 3-6 good discussion questions per class that were related to the topic and readings and that helped connect these readings to previous themes in the course. In general, I was impressed that students stayed engaged and on topic (especially after I cracked down on screens in the class).

The other change is that I’m not used to grading term papers and I was really worried about AI. Based upon feedback I got from colleagues, I had students do this in stages: (1) they submitted abstracts (I gave feedback on these). (2) They submitted “rough drafts” - I ultimately decided these should be more like a combination of annotated bibliographies plus some early drafting. The fact that the expectation was that this wouldn’t be very polished at this stage helped a lot here. Next time, I think I’d give a few more guidelines about the number and type of references needed. I gave substantial feedback on these rough drafts. (3) They submitted final papers. Overall, I had a good sense of student voice by this point and the progression of the work - so these papers were in general rather strong.

Some topics they chose to write about included: * The Robinhood app and gamification and how it exploits users’ misunderstandings of statistical uncertainty * History of cognitive testing and identification of learning disabilities in education * Predictive algorithms used in university admissions and implications for equity and fairness * How the death count was used to justify the Vietnam war and the errors that resulted * ACL injuries in women’s soccer and how this relates to issues of the ‘average’ man and generalizability, as well as ethics of research * Greenhouse gas reporting and corporate greenwashing and the role of self-reporting * Statistical reasoning errors in the Challenger explosion and what it teaches us about the role of statistical communication in ethics * Recommender systems in apps and social media, ethical harms, and the role for regulation

The Statistician’s Oath

Early in the quarter - when talking about ASA ethical guidelines - I had the students draft a version of a Hippocratic Oath for statisticians/ data scientists. I structured it by taking the first lines of the actual Hippocratic Oath and then leaving blanks. Students worked in groups and submitted these to me. I synthesized them (with the help of Claude.ai) and in our last class session, the whole class worked in groups to revise and come to consensus about what an oath should say. There was a lot of debate! We ended the quarter with them standing and taking the oath together.