Repeatability underpins a basic assumption in science which students must learn in order to evaluate others’ research findings as well as to communicate the results of their own research. By attempting to repeat the methods of published studies, students learn the importance of clear written communication, while at the same time developing research skills. I describe three examples of published field studies that can be used as the basis for course exercises on the repeatability of methodology, as well as field sampling techniques, all grounded in the overall topic of environmental change. Two of the exercises returned students to the exact location of the past research that they had previously read from the primary literature, making it possible to clarify the difference between reproducibility and repeatability in field-based research. When student-collected data differed from published results, students explored, through both post-project discussions and written work, factors that could explain this variation, including methodology, ecological succession, and climate change. Assessments and student comments on course evaluations showed that these exercises have a positive impact on students’ communication skills and engagement with the scientific process.

Introduction

Science educators are faced with the curricular dilemma of teaching an ever increasing body of knowledge while still ensuring that students gain basic skills for understanding and doing science (AAAS, 2011). Fieldwork can be designed both to develop skills and to teach important concepts, while at the same time engaging students in a way that fosters deep understanding and curiosity. The projects I describe here use guided fieldwork to address the concept of repeatability in the scientific process while also helping students understand the important details to include when writing methodology, all with a backdrop of how and why ecosystems change over time and space.

Field-based course activities facilitate students’ understanding of complex systems as they are immersed in experiential learning (Allen, 2011). Fieldwork, although sometimes regarded as old-fashioned (Allen, 2014), has been shown to increase learning in the affective domain and improve skills development in a scientific discipline (Boyle et al., 2007). Indeed, Simon et al. (2013) argued that field-based activities can deepen learning outcomes in courses using systems approaches to environmental science. Thus, developing curricular innovation in field settings contributes to the overall toolkit available for teaching in STEM disciplines.

Reproducibility and repeatability are key parts of the knowledge-building process in science and have received recent attention in fields like psychology and chemistry (see Pashler & Wagonmakers, 2012; Laird, 2014). While true reproducibility may be impossible in an ecological field study because of a large number of time- and site-specific variables, having enough information to allow for repeatability is still essential (Cassey & Blackburn, 2006; Ellison, 2010; Shapiro & Báldi, 2012; Bruna, 2014). Field biologists must effectively communicate additional key information about their methodology, beyond the typical details for a bench-based research project. Factors such as dates, weather conditions, and site descriptions provide context for interpreting field data and repeating research at other times or locations for temporal and spatial comparisons. Developing scientists must learn how to clearly communicate essential parts of their methodology if their research is to be repeatable, yet students often have a poor grasp of how to write out procedures because of limited experience or understanding (Campbell et al., 2000). Showing students examples of written research reports can facilitate the learning of skills needed for writing methodology (Tilstra, 2001). Because the methods section of a research report is often the key to framing the rest of the communication of the research (Smagorinsky, 2008), focusing on learning how to construct a sufficiently detailed methods section should be a central part of teaching scientific writing at the undergraduate level.

One advantage of attempting to repeat published methodology in ecological field studies is that data are generated by the students as they practice field-based procedures. Students can then be asked to compare their data to the published source and consider what accounts for any variation between their data set and that of the previous study. Variation can arise from problems with their understanding and, therefore, their success in repeating the original methods. It also could be a result of poor execution of what was a clear methodology. But, perhaps more likely, if students have done a careful job, their data will reveal changes over time or between locations. The latter can lead to an interesting exploration of factors that could underlie these changes, including different management regimes, changing abiotic conditions, or impacts of disturbance. Because students are fully immersed in producing their own new data (i.e., generating new knowledge) in these projects, their engagement with the key concepts embodied in these alternative hypotheses may reach a deeper level than what often occurs in a traditional lecture on management issues or environmental change.

Regional Context

The three laboratory exercises described below were designed for the Southern Lake Michigan ecoregion where I am based but could be adapted in other regions with similar ecological conditions. Ecosystems in this region include everything from deep forest to savanna and prairie. The region has a diverse mix of land uses, including heavy industry, agriculture, urban and suburban communities, and scattered parks and natural areas. As with many locations in the United States, we have issues with animal populations (both too large and too small), invasive species, and impacts from pollution and a changing climate (Greenberg, 2004). As is typical with many laboratory exercises, these were constrained by the 3-hour window of class time that had to include travel to and from the local field site, orientation to the landscape, and data collection.

Constructing the Task

The exercises all require some scaffolding of skills prior to lab day, as well as a post-exercise discussion of methodology and analysis of variability in data. For the examples given here, this included learning plant identification skills in these or previous courses. In addition, extensive preparation was done in the course sessions immediately before the actual data-collection laboratory, including reading the assigned focal article, deciding as a class what part of the research could be repeated by students in the context of class time, and work done in small groups to generate lists of needed supplies, with a discussion of these lists across groups. In some cases, we practiced methodology on campus first before traveling to the actual research site. When possible, the exercises were based on research done at accessible locations that we could visit during class time. After collecting data using the published protocols, additional class time was spent compiling data and discussing how the results compared to the published ones, as well as to previous semesters of student data from the same project. Students were asked to consider alternative hypotheses for any variation observed and challenged to critique the clarity of the published methodology. In some cases (i.e., example 2 below), student data from a project were provided to a community partner for use in ecological restoration planning.

Example 1: Tree Density & Fire Ecology in Sand Dune Habitat

Inland Marsh, now called Tolleston Dunes, is a unit in the Indiana Dunes National Lakeshore that contains a mix of upland oak savanna and wetter areas with trees like black gum and aspen. Frequent fire shaped this ecosystem, and the history and importance of wild fires was the focus of a study done in 1981 by Henderson and Long. In their paper, published in 1984, they reported measurements of tree density in the upland part of the habitat. In this activity, we attempted to repeat a small portion of their methodology in order to compare data and examine changes over the past 30 years. Students were instructed to repeat the methods described in the following excerpt from Henderson & Long (1984):

Each study area was sampled during the summer of 1981, utilizing a stratified random design of line transects and circular plots. Five transect lines spanned the entire north–south distance of each area, perpendicular to the general orientation of the dune ridges. The centers of five circular plots, 20 m in diameter, were established at random distances along each transect line. Within each plot, trees (dbh ≥ 5 cm) were identified as to species.… The live/dead status was noted, and sprouts were counted on each stem.

Working in teams of three, the students sampled as many plots as possible in the time allotted (~1 hour). I have found that even though they know that dbh = diameter at breast height, or what “stratified random design” means, actually applying this knowledge in the field is challenging and serves to clarify the methodology. For example, students have often struggled with how to define and measure “sprouts/stem” because no additional details are offered in the published paper, beyond a data table that reports “basal sprouting” – thus identifying an ambiguity in the original description of methods. Comparisons of data from this project with the data from Henderson and Long's (1984) results revealed possible changes over time in this forest (Table 1). Students explored the potential reasons for an increased tree density seen in our data during a post-exercise discussion and in a writing assignment. Hypotheses included variation in methods (even though we have tried to repeat them exactly), variation in area of the forest sampled (students usually cannot access wetter areas), reduced fire frequency (raises issues of management using controlled fires in the National Park), and changes in climate patterns.

Table 1
Data from published research and students’ efforts, showing number of trees per hectare at Inland Marsh, Indiana Dunes National Lakeshore.
Species1981 (Henderson & Long, 1984)2012 (Student Generated)2013 (Student Generated)2014 (Student Generated)
Black oak 102.2 530.5 471.1 418.4 
White oak 10.2 233.4 133.7 113.7 
Red maple 7.6 
Sassafras 2.7 29.7 85.9 18.2 
Black gum 1.3 12.7 35.0 50.0 
Dead trees 87.9 46.7 232.4 31.8 
Species1981 (Henderson & Long, 1984)2012 (Student Generated)2013 (Student Generated)2014 (Student Generated)
Black oak 102.2 530.5 471.1 418.4 
White oak 10.2 233.4 133.7 113.7 
Red maple 7.6 
Sassafras 2.7 29.7 85.9 18.2 
Black gum 1.3 12.7 35.0 50.0 
Dead trees 87.9 46.7 232.4 31.8 

Example 2: Deer Browse Damage & Deer Control in Black Oak Savanna

White-tailed deer management is a contentious issue in our region because large populations of deer can do extensive damage to sensitive vegetation (Porter & Underwood, 1999). In this exercise, we worked at John Merle Coulter Nature Preserve, an oak savanna owned by a local land trust. We repeated the following method used to study deer browse damage in an Illinois upland oak–hickory forest habitat (Strole & Anderson, 1992):

[B]rowsing habits and intensity were measured after the winter browsing period was finished but when it was still possible to determine the current season's browsing damage (March, April, and the first week of May 1987). At each site, 60 circular quadrats, 1 m in diameter, were located using a stratified random sampling procedure. Along North-south paced compass lines, points were located at approximately 5-m intervals. Two-digit random numbers were used to determine the distances beyond the points (first digit) and left or right of the line (second digit, even number to the right and odd to the left) for the location of quadrats. Within each quadrat, all browsable twigs of woody species were counted. Browsable twigs were defined as growth from the previous season ≥1 cm that occurred between 0 and 1.25 m above the ground…. Each twig was scored as browsed or unbrowsed and the species was recorded.

Students were asked to consider whether deer at Coulter Preserve have different preferred browse species compared to the Illinois site and also whether the deer management being carried out at Coulter Preserve is correlated with changes in deer browse damage. They had to follow additional methods in the Strole and Anderson paper to calculate from their data relative abundance, use, and percentage of available twigs browsed (Table 2).

Table 2
Percentage of available twigs browsed by deer of selected species in a published study in Illinois and student data from Coulter Nature Preserve. The number of deer taken during the previous fall hunting season at Coulter is also noted.
SpeciesStrole & Anderson, 1992 Student Generated
20092010201120122013
Oak 51 26 35 28 12 21 
Sassafras – 69 50 – 33 54 
Maple 100 – 27 
Cherry 62 59 60 12 
Ash 72 83 – – – 
All species 14 43 46 19 17 18 
Deer taken – 19 15 13 12 27 
SpeciesStrole & Anderson, 1992 Student Generated
20092010201120122013
Oak 51 26 35 28 12 21 
Sassafras – 69 50 – 33 54 
Maple 100 – 27 
Cherry 62 59 60 12 
Ash 72 83 – – – 
All species 14 43 46 19 17 18 
Deer taken – 19 15 13 12 27 

As students processed these data they considered why, overall, there was a sudden shift in the percentage of all twigs browsed after 2010, how the presence of different species in the landscape helps determine what species deer will favor (e.g., they like sassafras), and how loss of species (e.g., ash to an invasive insect pest or maple to forest management) might change browsing habits. Once again, the challenges in replicating methods and how these can influence data comes up in the discussions of year-to-year variation.

Example 3: Seedling Recruitment & Environmental Change in Old-Growth Maple Beech Forest

A long-term study of a remnant old-growth beech–maple forest (Poulson & Platt, 1996) forms the basis for a study of changes in seedling recruitment. Poulson and Platt compared their data to an even older study (Cain, 1935), which lets students see that scientists repeat methods as part of research studies and that this has utility beyond simply a course exercise. In order to calculate the density of maple and beech seedlings and juveniles in the forest now and compare it with data from 1933 and 1980, students attempted to repeat the following sampling techniques (Poulson & Platt, 1996):

Our study site includes the undisturbed areas that others have studied.… It is an irregularly shaped 16-ha area of flat upland that is 0.03–1.5 m above the water table in spring (we do not report here on areas with standing water in spring). The northern border of our site runs parallel to and 5 m south of Warren Woods Road. The western boundary is 20 m from a field edge. The eastern and southern boundaries are the edge of a slope down to the Galien River.… In 1980 we sampled 25 quadrats of 100 m2 in the same approximate area studied by Cain (1935). Trees were assigned to the same five size classes reported by Cain: seedlings (<30 cm tall), juveniles (>30 cm tall but <2.5 cm dbh)….

One modification I made in this exercise is to use circular sampling plots rather than the 10 × 10 m square quadrats used in the original study. This alteration led to a discussion of what is doable in a field situation with vegetation-covered sampling areas. And because of time issues, the class only looked at the smallest of the five tree size classes. Students considered why our data showed higher densities than past studies and speculated on whether these differences were a result of actual changes in the ecology of the forest or methodological variations (Table 3).

Table 3
Density (number/m2) of seedling and juvenile maple and beech trees in Warren Woods from two published studies and student data.
YearSugar Maple DensityAmerican Beech Density
SeedlingsJuvenilesSeedlingsJuveniles
1933 (Cain, 1935) 1.6 0.4 0.1 0.1 
1980 (Poulson & Platt, 19961.5 1.7 0.1 0.2 
2009 (student data) 4.0 0.7 1.1 0.4 
2010 (student data) 3.3 0.4 1.0 0.3 
2012 (student data) 1.7 0.8 2.5 0.1 
YearSugar Maple DensityAmerican Beech Density
SeedlingsJuvenilesSeedlingsJuveniles
1933 (Cain, 1935) 1.6 0.4 0.1 0.1 
1980 (Poulson & Platt, 19961.5 1.7 0.1 0.2 
2009 (student data) 4.0 0.7 1.1 0.4 
2010 (student data) 3.3 0.4 1.0 0.3 
2012 (student data) 1.7 0.8 2.5 0.1 

Conclusions

All three of these exercises, although used in different courses at my institution, lead students to consider the effects of methodological variation on resulting data and how that affects the repeatability of a study. As they struggle with trying to replicate published methodologies, students can see the importance of including essential details in a good Methods section of a research paper. Following these exercises, students were assigned to write research reports using their own data, and the focal published methodologies served as helpful examples for these writing assignments. In a comparison of pre- and post-project assessment questions in one recent course, there was a 30% increase in the number of students choosing details that are important to include in reports of field methodology and a 33% drop in students choosing inappropriate information (Table 4).

Table 4
Results from a Field Biology course (n = 11) from before and after the research projects described in examples 1 and 2. Students were asked to choose which types of information should ideally be included in a Methods write-up of field research from a different project on squirrel behavior.
Type of InformationStudent Choices Pre-ProjectStudent Choices Post-Project
Relevant details for “Methods”:
  • Weather conditions (temperature, wind, cloud cover, etc.)

  • Date of study

  • Experimental setup

  • Description of study sites

  • Behaviors recorded

  • Statistical test used

 
47 61 
Details that should not be in “Methods”:
  • Day of the week

  • Data from the experiment

  • Background literature citations for experimental question

  • Names of field assistants

 
12 
Type of InformationStudent Choices Pre-ProjectStudent Choices Post-Project
Relevant details for “Methods”:
  • Weather conditions (temperature, wind, cloud cover, etc.)

  • Date of study

  • Experimental setup

  • Description of study sites

  • Behaviors recorded

  • Statistical test used

 
47 61 
Details that should not be in “Methods”:
  • Day of the week

  • Data from the experiment

  • Background literature citations for experimental question

  • Names of field assistants

 
12 

Although these lab exercises were originally designed to build skills, they have also served to immerse students in course content through reflection on how ecosystems can change over time. This falls within Content Area 5 in the AAAS (2011) Vision and Change recommendations for biology education. Thus, I believe that each of these exercises accomplished three goals: illustrating the principle of repeatability in scientific methods, practicing that repeatability and learning about the essential contents of the Methods section of a research paper, and reflecting on the importance of changing conditions in ecosystems and how these can influence what lives there. Judging from student comments made on end-of-course evaluations from several courses that included these exercises (see below), some students found these projects to be transformative.

The three examples described here can be used as models for developing similar laboratory exercises in other locations by noting several key factors that led to their success. Each published study had both a methodology and a results piece that could be extracted from the research paper for use as the basis of a field study. The focal published research must be accomplishable during the time allotted by the course schedule and appropriate for the seasonal conditions. For example, one year I tried repeating a study with students on winter bird abundance in conservation reserve cropping systems in local agricultural fields. On the one day of our laboratory curriculum that I had set aside for the study, the weather was terrible and we spent 3 cold hours and saw no birds. Clearly, this data collection protocol required a flexibility in response to weather conditions that I had not built into my course. In all examples detailed above, course-based exercises were accomplished by leveraging the power of a class of students working in small groups (of three or four) and combining data. It was especially effective, in terms of student engagement, to return to the exact location of the original study, although this sometimes presented additional challenges when research permits were required, such as for the National Park site at Inland Marsh. However, such challenges are often worthwhile because park management decisions are informed by science, and student data can be useful to park managers, adding a service-learning component to the laboratory exercises (Reynolds & Ahern-Dodson, 2010). For example, in the deer browse study (example 2), a representative of the land trust met us in the field and talked about their resource management challenges and how our data would help them. The student-generated data from the exercise were then provided to the land trust to help in their understanding of how deer are affecting woody vegetation at the property. This type of service to the community through coursework serves to increase student motivation and engagement, as is evident from the comments below. The exercises described here were used in biology courses, but this type of activity would be conducive to other field-based science disciplines such as chemistry, geography, and environmental science.

Students’ Comments about Field Methods in Ecology & Field Biology Classes, 2009–2014

  • I liked the experience of reading from primary research repeatedly throughout the semester. Overall, I appreciated how much [of this] related to scientific writing.

  • By the end of the semester, we were all comfortable discussing scientific literature at a high intellectual level.

  • Discussions [of primary literature] helped connect course material to real-life issues and always induced stimulating conversation.

  • The field portion of this class helped make me a well-rounded scientist, giving me actual research field experience.

  • I feel as though I have learned so much in this class, and am better prepared to be a more well-informed environmental scientist should I pursue a job with fieldwork.

  • I gained skills in this class that I will hold on to forever, and I truly believe they have better prepared me for whatever I might choose to do when I graduate.

  • I loved all of the local applications of ecological concerns the class explored….

  • [The course] was wonderful both in the content of the class and in the lab and field team techniques we practiced. Completely changed my mind about what I want to do after graduation!

  • It was enjoyable collecting data, knowing that it would be put to good use rather than just for class.

Multiple sections of students in Ecology and Field Biology courses at Valparaiso University contributed to the data presented here. Jonathan Bauer provided the seeds for the idea of researching deer browse damage at John Merle Coulter Nature Preserve; and Paul Quinlan, with the Shirley Heinz Land Trust, made our work possible there. The National Park Service allowed us to work in the Indiana Dunes National Lakeshore (permit no. INDU-00321). D. Dew, P. Martin, and S. Martin-Eberhardt helped in the preparation of the manuscript.

References

References
AAAS
(
2011
).
Vision and Change in Undergraduate Biology Education: A Call to Action
.
Washington, DC
:
AAAS
.
Allen, C.D. (
2011
).
Concept mapping validates fieldwork's capacity to deepen students’ cognitive linkages of complex processes
.
Research in Geographic Education
,
13
,
30
51
.
Allen, C.D. (
2014
).
The need for fieldwork in science
.
Journal of College Science Teaching
,
43
,
10
11
.
Boyle, A., Maguire, S., Martin, A., Milsom, C., Nash, R., Rawlinson, S., Turner, A., Wurthmann, S. & Conchie, S. (
2007
).
Fieldwork is good: the student perception and the affective domain
.
Journal of Geography in Higher Education
,
31
,
299
317
.
Bruna, E.M. (
2014
).
Reproducibility & repeatability in tropical biology: a call to repeat classic studies
.
Biotropica: The Editor's Blog
, May 29. http://biotropica.org/reproducibility-repeatability/.
Cain, S.A. (
1935
).
Studies on virgin hardwood forest: III. Warren's Woods, a beech–maple climax forest in Berrien County, Michigan
.
Ecology
,
16
,
500
513
.
Campbell, B., Kaunda, L., Allie, S., Buffler, A. & Lubben, F. (
2000
).
The communication of laboratory investigations by university entrants
.
Journal of Research in Science Teaching
,
37
,
839
853
.
Cassey, P. & Blackburn, T.M. (
2006
).
Reproducibility and repeatability in ecology
.
BioScience
,
56
,
958
959
.
Ellison, A.M. (
2010
).
Repeatability and transparency in ecological research
.
Ecology
,
91
,
2536
2539
.
Greenberg, J. (
2004
).
A Natural History of the Chicago Region
.
Chicago, IL
:
University of Chicago Press
.
Henderson, N.R. & Long, J.N. (
1984
).
A comparison of stand structure and fire history in two black oak woodlands in northwestern Indiana
.
Botanical Gazette
,
145
,
222
228
.
Laird, T. (
2014
).
Editorial reproducibility of results
.
Organic Process Research & Development
,
18
,
921
.
Pashler, H. & Wagenmakers, E.-J. (
2012
).
Editors’ introduction to the Special Section on Replicability in Psychological Science: a crisis of confidence?
Perspectives on Psychological Science
,
7
,
528
530
.
Porter, W.F. & Underwood, H.B. (
1999
).
Of elephants and blind men: deer management in the U.S. national parks
.
Ecological Applications
,
9
,
3
9
.
Poulson, T.L. & Platt, W.J. (
1996
).
Replacement patterns of beech and sugar maple in Warren Woods, Michigan
.
Ecology
,
77
,
1234
1253
.
Reynolds, J.A. & Ahern-Dodson, J. (
2010
).
Promoting science literacy through research service-learning, an emerging pedagogy with significant benefits for students, faculty, universities, and communities
.
Journal of College Science Teaching
,
39
(
6
),
24
29
.
Shapiro, J.T. & Báldi, A. (
2012
).
Lost locations and the (ir)repeatability of ecological studies
.
Frontiers in Ecology and the Environment
,
10
,
235
236
.
Simon, G.L., Wee, B.S., Chin, A., Tindle, A.D., Guth, D. & Mason, H. (
2013
).
Synthesis for the interdisciplinary environmental sciences: integrating systems approaches and service learning
.
Journal of College Science Teaching
,
42
(
5
),
42
49
.
Smagorinsky, P. (
2008
).
The method section as conceptual epicenter in constructing social science research reports
.
Written Communication
,
25
,
389
411
.
Strole, T.A. & Anderson, R.C. (
1992
).
White-tailed deer browsing: species preferences and implications for central Illinois forests
.
Natural Areas Journal
,
12
,
139
142
.
Tilstra, L. (
2001
).
Using journal articles to teach writing skills for laboratory reports in General Chemistry
.
Journal of Chemical Education
,
78
,
762
764
.