Evaluating the strength of evidence in noisy data is a critical step in scientific thinking that typically relies on statistics. Students without statistical training will benefit from heuristic models that highlight the logic of statistical analysis. The likelihood associated with various coin-tossing outcomes gives students such a model. There is a point at which a majority of students will doubt the underlying fairness of a coin-tossing game, and that student-generated cutoff can be used as a critical value in a nonparametric analysis of paired data. Following this approach, student lab reports come to more closely model scientific thinking.
There is broad consensus that students should be learning scientific thinking (e.g., National Research Council, 2003; National Science Teachers Association, 2004, 2011), and the science-education literature contains a wealth of information on inquiry-based teaching. However, while guidance on generating questions, designing experiments, and collecting data is common, relatively little published information helps one guide students through data interpretation. Biological systems are inherently variable, and biologists use statistics to evaluate noisy data, but many of our students have little or no statistical training. Giving students comprehensible tools with which to interpret their data is thus a critical step in teaching scientific thinking.
I’ve observed that most students readily grasp the “If-And-Then-Therefore” logic (Lawson, 1999) of the scientific method but are frequently befuddled by the logic of data analysis. For example, I often ask students whether antibacterial soap works better than regular soap, and they predictably produce scientifically sound experimental designs. But they also predictably struggle when, predictably, antibacterial soap outperforms in some trials while regular soap outperforms in others. A typical student report might discuss one of the following: (1) the experiment failed, (2) the data are inconclusive, or (3) student error(s) resulted in “outliers.” What they fail to do is trust the validity of their varied results and contemplate how those results might enhance their understanding of the natural world. Yet biologists deal with varied results all the time.
For the past few years I’ve employed a simple, 20-minute coin-tossing exercise that seems to greatly improve the analytical sophistication in lab reports. I start by repeatedly flipping a coin and asking the students to predict each outcome. The catch is that I create the illusion of many identical results by calling “heads” every time. Importantly, I quietly keep mental notes as students express skepticism. When a supermajority doubts the legitimacy of the game, I begin a discussion of their initial assumption (the coin is fair) and the strength of evidence that might lead them to reject that assumption. Typically, a group of students will agree that four to five identical tosses triggers serious doubt. I then ask them to consider 18 simultaneous tosses (assuming 18 students in a lab), and we discuss the implications of various possible outcomes. Typically 14 or 15 “heads” in 18 tosses will create a consensus of doubt about the fairness of the game. In conclusion, I extend the discussion to the analysis of their data. If two treatments are no different, and 18 students replicate the protocol, then only a 14:4 or greater split in the results should constitute strong evidence for a treatment effect. Interestingly, the actual probabilities of four-consecutive-heads and four-or-fewer-heads-in-18-tosses are 0.0625 and 0.0481, respectively. Thus, the cutoffs that students intuitively home in on end up quite close to the generally accepted significance value of 0.05.
This exercise is adaptable to a wide variety of investigations, provided that the data are paired, and it can enhance activities over a wide range of grade levels. Paired data are produced when each student completes one set of trials and the resulting data are aggregated into a class data set. I’ve used some modification of this coin-toss activity with several variations of a leaf-shredder lab (after Sparkes et al., 2008). The action of leaf shredders can be tested over different leaf treatments, or for comparisons of food preference. At an advanced level, the coin-toss activity can be extended to the tossing of dice and provide a foundation for analysis of variance. At an introductory level, the activity can be used to build an additional layer of sophistication into data analysis that might otherwise be limited to summary statistics such as means and medians, and visual representations such as histograms.
I have found this exercise to be beneficial from several perspectives. First, the students choose their own critical value via a highly intuitive process without dependence on calculations or jargon. Implicit in this approach is the null-hypothesis concept, yet a discussion of the same isn’t required. In lab reports, I have seen greater focus on the strength of evidence for a particular effect, and less on proving one’s hypothesis or on putative methodological errors. This, in turn, sets the stage for students to seek information that might bolster (or stand in contrast to) their case. All in all, when lab reports are framed from the perspective of the coin-tossing exercise, I find that they much more accurately reflect the way in which a scientist might think about a problem.
I am grateful to the students and faculty in the Department of Natural Sciences at Lyndon State College, especially the secondary science education students. Additionally, two anonymous reviewers provided insightful, helpful comments.