Working Group E
Institutional Competence in Evaluating Efficacy Research

What level of expertise do institutional decision makers have to critically evaluate and understand efficacy research and whose responsibility should it be to prepare institutional leaders and educators to make EdTech purchasing decisions? What are the schools of education doing to prepare institutional leaders and educators to make EdTech purchasing decisions? What are the best practices in preparing institutional leaders and educators to request, interpret and use efficacy research to make decisions?

Working Group E: Blog
Institutional Competence in Evaluating Efficacy Research
October 5, 2016

A recent article in the New York Times contained an interesting anecdote related to a new ed-tech innovation. Facebook is working with Summit Public Schools in California to refine and extend a personalized-learning platform called “Summit Basecamp.” This platform is currently being piloted in a number of schools, and thus far at least one principal is enthused with the results:
“There were many points where we weren’t sure the Summit Basecamp model was what our students needed,” said Claire Fisher, the principal of Urban Promise Academy, a public middle school in Oakland, Calif., which introduced the platform in its sixth-grade classes.

By the end of the school year, however, 31 percent of the school’s sixth graders were reading at or above their grade level, compared with just 9 percent in the fall. That was a larger improvement in reading than students in seventh and eighth grades, which did not use the platform, Ms. Fisher said.

Is Principal Fisher right to conclude that Summit Basecamp is working based on this evidence? Bigger question: When it comes to evaluating efficacy, do school leaders, teachers, philanthropists, ed-tech providers, and researchers approach these sorts of questions the same way?

Existing evidence suggests perhaps not. A recent report identified no less than 30 different measures that so-called next-generation educators use to evaluate the impact of education innovations. These measures range from using state tests and external evaluations to relying upon family conversations or student progress on individual goals — to name only a few.

All of this highlights the research-practice divide in education, and the lack of common agreement around basic terms. So our Working Group hopes to add one small brick to bridging that divide by gathering some data on how different participants in the education system think about evidence and efficacy. We then hope to offer some suggestions on how we might prepare an educator workforce that will be able to scientifically — or at least semi-scientifically — evaluate technology-based education innovations.


Comments
  • Nice post. I learn something new and challenging on blogs I stumbleupon on a daily basis.
    It’s always helpful to read articles from other authors and practice a little
    something from their websites.



Post a Comment

Your email address will not be published. Required fields are marked *

Working Group Leaders
Benjamin Riley
Executive Director
Deans for Impact
Robert C. Pianta
Dean
Curry School of Education, University of Virginia
Research Lead
Glen Bull
Co-Director
Center for Technology & Teacher Education, Curry School of Education, University of Virginia
Working Group Members
Amy Nowell
Director of Research
LEAP Innovations
Benjamin Gaines
Principal
Rancho Minerva Middle School, Vista Unified School District (CA)
Bob Wise
President
Alliance for Excellent Education
Brad Lane
Co-founder & CEO
RIDGE-LANE Limited Partners
Dave Edyburn
Associate Dean for Research
College of Education and Human Performance, University of Central Florida
David Steiner
Director
Institute for Education Policy, Johns Hopkins University
Ellen Meier
Director
Center for Technology & School Change, Teachers College, Columbia University
J. Michael Spector
Professor
University of North Texas
Jeffrey Cohen
CEO
Catapult Learning
Jeremy Shorr
Director of Technology Innovation and Early Childhood STEM Education
Teaching Institute for Excellence in STEM
Joseph South
Karen Johnson
Senior Program Officer
Bill & Melinda Gates Foundation
Kate Kazin
Chief Academic Officer
College for America at Southern New Hampshire University
Kay Persichitte
Professor of Instructional Technology
University of Wyoming; Assoc. for Educational Communications & Technology, President
Kinshuk
Dean
College of Information at the University of North Texas
Lou Pugliese
Senior Innovation Fellow and Managing Director, the Action Lab
EdPlus at Arizona State University
Teresa Foulger
Associate Professor of Educational Technology
Arizona State University
Logistics Lead
Ileen Warner
Vice President of Communications
Jefferson Education Accelerator, University of Virginia
Benjamin Riley
Executive Director
Deans for Impact
Benjamin is the founder of Deans for Impact, a national nonprofit organization working to transform the way we prepare teachers in the U.S. He formerly worked as policy director for NewSchools Venture Fund and as Deputy Attorney General for the state of California.
Amy Nowell
Director of Research
LEAP Innovations
Before joining the LEAP team, Amy was an independent consultant for program evaluation and data systems after almost a decade with Chicago Public Schools. At CPS, she led the effort to produce comprehensive school scorecards and designed processes for principal eligibility and evaluation.
Benjamin Gaines
Principal
Rancho Minerva Middle School, Vista Unified School District (CA)
Through his 4 years at RMMS, Benjamin has led the school as a Digital Promise Verizon Innovative Learning School, where students received 24/7 connectivity as a 1:1 iPad School. Under his leadership, Rancho Minerva has been in the national spotlight as a P21 Exemplar School for their unique approach to teaching and learning.
Benjamin Riley
Executive Director
Deans for Impact
Benjamin is the founder of Deans for Impact, a national nonprofit organization working to transform the way we prepare teachers in the U.S. He formerly worked as policy director for NewSchools Venture Fund and as Deputy Attorney General for the state of California.
Bob Wise
President
Alliance for Excellent Education
Bob served a combined twenty-four years as governor, member of the U.S. House of Representatives, and state legislator, Led by Gov. Wise since 2005, the Alliance for Excellent Education has become a respected advocate for the Common Core State Standards, deeper learning, Linked Learning, digital learning, adolescent literacy, and other key education policy issues.
Brad Lane
Co-founder & CEO
RIDGE-LANE Limited Partners
R. Brad Lane is the Chief Executive Officer of RIDGE-LANE Limited Partners, an investment-bank founded with The Honorable Thomas J. Ridge, which sponsors public-private partnerships to improve the nation’s infrastructure, economic development programs to spur growth in metropolitan cities and specialty venture development for private growth companies. He founded Lane Financial Corporation, a multi-family office providing global wealth management strategies for high-net-worth individuals, where he served as Managing Partner of Atlas Capital, the $12 billion subsidiary B/D structuring initial public offerings in Technology & Healthcare and alternative investments in Real Estate.
Dave Edyburn
Associate Dean for Research
College of Education and Human Performance, University of Central Florida
Dave is the Associate Dean for Research in the College of Education and Human Performance at the University of Central Florida. His work focuses on the use of technology to enhance teaching, learning, and performance. Dave also serves as editor of the Journal of Research on Technology in Education.
David Steiner
Director
Institute for Education Policy, Johns Hopkins University
David previously served as Commissioner of Education for New York State, as the Klara and Larry Silverstein Dean at the Hunter College School of Education, and as Director of Education at the National Endowment for the Arts. His insistence on including major funding for curricula in the State's Race to the Top grant led to the launch of EngageNY, the nation’s most consulted on-line curriculum resource.
Ellen Meier
Director
Center for Technology & School Change, Teachers College, Columbia University
Ellen is also a faculty member in the Department of Mathematics, Science, and Technology. She is co-chair of the University of the State of New York Council on Technology Policy and Practice. Ellen began her career as a teacher in a federal experimental school program in Minnesota.
Glen Bull
Co-Director
Center for Technology & Teacher Education, Curry School of Education, University of Virginia
Glen is also a professor of instructional technology in the Curry School of Education. He developed one of the nation’s first statewide K-12 Internet systems, Virginia’s Public Education Network (PEN). Glen is a founding member and past president of the Society for Information Technology and Teacher Education (SITE).
Ileen Warner
Vice President of Communications
Jefferson Education Accelerator, University of Virginia
Ileen is a marketing consultant focused in education where she develops and implements programs that bring together the greatest minds in the industry to collaborate and develop solutions through seminars, networking and sharing best practices. Ileen is also the Director of the YPO Higher Education Group.
J. Michael Spector
Professor
University of North Texas
Michael is a professor of learning technologies. He previously served as president of AECT, editor of ETR&D, editor of the SAGE Encyclopedia of Educational Technology, and editor of the Handbook of Research on Educational Technology and Communications (3rd, 4th eds.).
Jeffrey Cohen
CEO
Catapult Learning
Jeffrey previously served as the President of Catapult Learning from 2001 to 2006, when the company was a division of Educate, Inc. In 2008, he oversaw the sale of the business to JMI Equity and The Carlyle Group. Prior to returning, Jeffrey served as the President and CEO of Sylvan Learning for seven years.
Jeremy Shorr
Director of Technology Innovation and Early Childhood STEM Education
Teaching Institute for Excellence in STEM
Jeremy previously worked at Mentor Public Schools as Director of Innovation and Educational Technology. He is the Chair of the International Society for Technology in Education's (ISTE) Technology Coordinator's Network and the Past-President of the Ideastream Advisory Council. In 2015, Jeremy was recognized as an Apple Distinguished Educator.
Joseph South
Karen Johnson
Senior Program Officer
Bill & Melinda Gates Foundation
Karen's work centers around K-12 personalized learning, including measuring product efficacy and spurring development of ed tech tools that meet the needs of teachers and students. Previously, Karen spent a decade marketing and building K-12 digital tools in social studies, science, math and language arts.
Kate Kazin
Chief Academic Officer
College for America at Southern New Hampshire University
Kate helped create CfA and designed CfA’s project-based CBE model. She is a NEASC/CIHE Commissioner and directs a $3.93 million FITW grant project to develop CBE alternatives to developmental education. Kate came to SNHU from ETS, where she was Executive Director of Higher Education.
Kay Persichitte
Professor of Instructional Technology
University of Wyoming; Assoc. for Educational Communications & Technology, President
Kay is Professor of Instructional Technology at the University of Wyoming (UW) and the current President of the Association for Educational Communications and Technology (AECT). Previously, she taught high school math and served in multiple higher ed administration roles including dean of the UW College of Education.
Kinshuk
Dean
College of Information at the University of North Texas
Kinshuk is the Dean of the College of Information at the University of North Texas. His research interests include learning analytics; mobile, ubiquitous and location-aware learning; and cognitive profiling.
Lou Pugliese
Senior Innovation Fellow and Managing Director, the Action Lab
EdPlus at Arizona State University
Lou has more than 25 years of experience developing a wide variety of technology solutions to radically improve teaching and learning outcomes. He previously served as founding CEO at Blackboard, as a senior vice president at ETS, and as president and CEO of Moodlerooms.
Robert C. Pianta
Dean
Curry School of Education, University of Virginia
Robert is a nationally-recognized expert in both early childhood education and K-12 teaching and learning. His research and policy interests focus on teacher-student interactions and relationships and their contributions to students’ learning and development.
Teresa Foulger
Associate Professor of Educational Technology
Arizona State University
Teresa S. Foulger is an Associate Professor of Educational Technology at Arizona State University. She is an expert in professional development and organizational structures that create lasting change in education, especially where technology is integrated into teaching and learning opportunities.
In The News
What people are saying about our Symposium and the importance of efficacy in EdTech.