I’ve just finished teaching Sociology 504, which is the second and final required statistics course for all of the Ph.D. students in the Sociology department. The students have been great, and I’ve learned a lot about statistics this semester. But, it has been challenging to teach a group of students that is so diverse, in terms of both interests and technical background. Now that the semester is over I thought that I would reflect on what what worked well and what I could do better next time.
Things that worked well:
- R is getting easier and easier. When I taught this class in the spring of 2009 many students struggled with R, but this time it worked much better. I think there are two main reasons for that: 1) R Studio, a great open-source R development environment and 2) better help on the web (thank you Stack Overflow for saving the students for the R-help email list). Also, here are some tips that I would recommend if you are going to use R in your course. First, explain why you are doing it. R will be difficult at first, and your students need to understand why you are making this decision. I tend to stress flexibility (R empowers you do what you want, rather than just what Stata Corp has already coded up) and openness (most students support open-source). Second, ask you students to follow the Google’s R Style Guide. I’ve found that students not used to coding find these kind of clear instructions very helpful, and the fact that this is exactly what they do at Google makes it cool. Finally, ask the students to post a link to their code for each homework assignment. We do this on our course Piazza page so that all the other students can see it, and I think this system encourages the students to think of code as an important part of the research process. Logistically, Gist makes this kind of code sharing easy and elegant.
- Negative is positive. Throughout the course I took an extremely critical approach to statistics as it is frequently practiced in sociology. Ordinarily, it would not make sense for a teacher to be relentlessly negative about the course material, but I think it worked well in this case. Roughly, there were two groups of student in my course: those that wanted to learn statistics and those that dislike statistics. I think that my negative approach actually worked well for both groups. For the group that wanted to learn statistics, they were excited to learn about the limitations of the methods that we were studying so that they could do better. For the group that didn’t like statistics, they were excited to learn about the limitations of the methods that we were studying because it helped them articulate what they already believed: that a lot of mediocre research is hiding behind apparently fancy statistics.
- Didactic methodological articles are great. This semester, in addition to the textbook (we used Fox), I assigned a few journal articles that I would call “didactic methodological articles.” In other words, these are methodological articles, but they do not introduce new methods, rather they comment on the use and misuse of existing methods. The students really loved these, and I would agree. I think they were both clearer and tighter than most of what one sees in a textbook, probably because the amount of time spent per word is much higher for a journal article than a textbook. Here are some of the articles that the students and I really thought worked well:
- Brambor, T., Clark, W.R., and Golder, M. (2006). Understanding Interaction Models: Improving Empirical Analyses. Political Analysis 14:63-82.
- Gelman, A., Pasarica, C., and Dodhia, R. (2002). Let’s Practice What We Preach: Turning Tables into Graphs. The American Statistician 56(2):121-130.
- Kastellec, J.P. and Leoni, E.L. (2007). Using Graphs Instead of Tables in Political Science. Perspectives on Politics 5(4):755-771. [see also the code repository ]
- King, G. Tomz, M., and Wittenberg, J. (2000). Making the Most of Statistical Analyses: Improving Interpretation and Presentation. American Journal of Political Science, 44(2):341-355.
- Hamner, M.J. and Kalkan, K.O. (2013). Behind the Curve: Clarifying the Best Approach to Calculating Predicted Probabilities and Marginal Effects from Limited Dependent Variable Models. American Journal of Political Science, 57(1)263-277.
- Mood, C. (2010). Logistic Regression: Why We Cannot Do What We Think We Can Do, and What We Can Do About It. European Sociological Review, 26(1):67-82.
- Berry, W.D., DeMeritt, J.H.R., and Esarey, J. (2010). Testing for Interaction in Binary Logit and Probit Models: Is a Product Term Essential? American Journal of Political Science, 54(1):248-266.
Things I can do better next time:
- Create tighter integration between reading, class, lab, and homework. Our class has four main ways that student can learn new material: reading, class, lab, and homework. In general, I tried to introduce students to the new material in that order. That is, they would read about an idea, hear me talk about it in class, practice doing it in the lab, and then do it on their own for homework. However, at times this structure broke down, and sometimes the homework assignments got a bit decoupled from the rest (which I think did hang together OK). Next time, I’d like to have a better integration of the homework with the rest of the class. Also, I’d like to occasionally have homework assignments that build on each other so that students can see how much they are learning.
- Less lecturing, more coaching. During the semester I met with Jeff Himpele from the McGraw Teaching Center about my undergraduate course, and he told me something that changed how I did my teaching in my graduate class. He advised me to think of myself as a coach for the students. While this sounds extremely simple, it really changed how I approached my course. I started to compare what I do in class to what some of my favorite soccer coaches have done. Surely, none of them told me for 90 minutes how to kick a soccer ball, nor did they make me watch them kick the ball over and over. Rather, they would talk a bit about the skill they wanted us to learn, and then they would do a quick demonstration. Next, we would all start doing a drill that was designed for us to practice the skill, and the coach would observe us giving encouragement and advice. I tried that in my statistics class, and it was actually extremely hard to get right. Designing a good question or series of questions that pushes the students to learn what you want is not easy. But, when it worked, it seems to really work much better than any lecturing that I can do. I’ll try it again next time.
- Have the students replicate a published paper. Rather than the standard final paper, I had the students participate in a prediction contest much like the Netflix Prize. In this case, we withheld the income of 500 people from the GSS, and the contest was to see who could predict these held-out incomes best. The one twist was that everyone had to make their code available for everyone else. My hope is that it would turn into a friendly competition where the students would be learning from each others code and then gradually discover the value of some of the techniques that we had been learning about such as transformations and interactions. In the end, it did not quite work out that way. For some reason the students were not that excited about it, possibly because it was not well integrated with the rest of the course. In any case, next time, I think I will ask them to replicate a published paper. This piece by Gary King seems like a good place to start on how to structure a replication paper assignment. And, who knows, my students might find an Excel error in a famous paper and end up on the Colbert Report.
- Get more regular feedback on how the class is going. At the end of the class, I gave students a short writing assignment with three questions: What do you think are the most important things you learned in this class? What would you have liked to learn more about? If you had to create three “statistics commandments” for other sociologists to follow what would they be? Overall, the results from this feedback were very reassuring. It seems like the students were actually learning what I wanted them to be learning. But, I should not have waited until the last class to find that out. As far as the statistical commandments, the wordle at the top of this post summarizes what they wrote. Then we spend some time in the last class trying to combine everyone’s ideas into three commandments we would all follow. We came up with:
- Know your data.
- Know your model.
- Be transparent (i.e., clearly describe what you did, report results on interpretable scales and as graphs, if possible, make your data and code available to others).
Obviously those commandments are a bit terse and simplistic, but I think that you could do a lot worse. If you have any advice for the next time I teach the class, please let me know.