A favorite college professor can make a lasting impact. They can encourage and inspire us, challenge and stretch us. When they stand before a classroom of students, there is just something about what they do that sets them apart from others.
But what, exactly, is that something?
A team of professors at the University of Virginia are leveraging technology across Grounds to begin identifying specific practices professors use that make their teaching effective.
“There has been a significant amount of work in evaluating effective teaching in K-12 classrooms,” said Karen Inkelas, associate professor in the Curry School of Education and Human Development. “The success of those efforts is due in part to K-12 class settings being more uniform in size and curriculum. However, college classrooms have incredible variety, making it incredibly difficult to determine what exactly is effective teaching.”
Indeed, classes at UVA range in size from five students to more than 300, in location from an auditorium of seats to a lab filled with electric guitars, and in type of instruction from online lectures to on-Grounds group projects.
Inkelas, along with Lindsay Wheeler, a Curry School alumna and assistant director of STEM education initiatives at the UVA Center for Teaching Excellence, and Michael Redwine, coordinator of instructional technologies in the UVA School of Engineering and Applied Sciences, are piloting a new web-based observational tool to identify what exactly is happening during a given class at UVA.
“Our hope this semester is to identify every incident that happens during the course of a class – from minutes lectured to questions asked, from group discussions to individual work,” Inkelas said.
Inkelas developed the initial version of the observation protocol. Redwine optimized the tool for an online setting. Wheeler provided expertise in using observation protocols to study teaching.
The ROCA tool allows researchers to pull up an image of the particular classroom being used. That image is overlaid with a grid that breaks the classroom into smaller units, where the researchers can track in precise detail where in the classroom activities happen.
“The grid allows us to track a number of things, including for example where questions are being asked or where the professor moves through the class,” Inkelas said. “Sometimes it feels like a particular class was especially lively and included robust class discussion, when the reality is it was only a handful of students who were especially engaged. With this tool we can parse out what is exactly happening during a class session.”
The ROCA involves observers recording both instructor-led and student-led activities. Within those two categories, observers can also provide greater detail on the activities occurring at the moment. For example, for instructor-led activities, observers would note whether the instructor was lecturing, leading a class discussion, using media, etc.
Further, the ROCA can capture at any given moment: what specific things the instructor is doing, what specific things the students are doing, what technology is being used, and any questions that are asked and/or answered. And, as noted previously, the ROCA can show by using a heat map where in the classroom those activities are taking place.
Twelve faculty members across UVA, all of whom teach in a classroom already outfitted with recording technology, have volunteered to share recorded class sessions with the team.
Inkelas and Wheeler hope that this semester’s pilot study will allow the team to test the reliability and validity of the ROCA and to, subsequently, see if there is any correlation between certain behaviors and outcomes like student evaluations, student grades, or recognitions like teaching awards.
“In the Center for Teaching Excellence, we assess the impact of various faculty development programs, so we also hope to connect ROCA data to our faculty programs,” Wheeler said. “We know the more STEM faculty engage in CTE programs, the more active learning we observe, but is that true beyond STEM courses?”
For Redwine, this project is a great example of how researchers can leverage technology to improve teaching and learning.
“The way we have envisioned melding technology and observations together is allowing us to peer deeper into what happens during classroom meetings,” Redwine said. “This detailed look, plus the ability to meld this data together with other analytical data, will provide educational specialists with not only what is happening today, but also how small changes made periodically during the semester are affecting student learning outcomes simultaneously.”
After the team finishes the pilot testing of the ROCA, they will revise the online tool and begin working on how to leverage the data obtained by the ROCA to better understand teaching practices that enhance the student learning experience.