Working Group J
Crowdsourcing Efficacy Research and Product Reviews

Given that “Consumer Reports for Education” does not currently exist, which organizations (if any) are aggregating product reviews and efficacy research, to help making decisions about which EdTech products to consider, pilot, and procure? Are states or regional cooperatives of institutions sharing product reviews and efficacy research? What promising efforts may be underway in the private sector and the media to aggregate and disseminate this type of information?

Working Group J: Blog
Crowdsourcing Efficacy Research and Product Reviews
October 5, 2016

The EdSurge Product Index currently lists more than two thousand different education technology products and services.  Many of these EdTech programs, systems, tools, games, and apps belong to product categories that did not exist even a decade ago. Many schools (K-12 and Higher Ed) feel completely overwhelmed by this abundance of choice, and have no reliable way to determine which products are most likely to be a good fit for their local needs.

As a result, it appears that faculty and staff at our institutions are collectively spending millions of hours reinventing the wheel as they and their individual institutions perform similar analysis in parallel:  They poke around on the web, read what they can find, identify a few candidate solutions, hopefully do a pilot, and then implement an EdTech product or service without ever documenting or sharing the results of their efforts with anyone outside their institutions.

Our Working Group is exploring this problem as well as potential solutions.  We seek to discover what efforts may be underway around the country to aggregate and share EdTech product reviews, analysis of pilots, and efficacy research.

If you are aware of any efforts to aggregate and/or disseminate this type of information, we would be grateful if you would spend a moment letting us know of them via this
Google Form
.  Our Working Group can then examine these efforts to identify best practices, draw attention to worthy efforts, and potentially assist with coordination between them.  Please feel free to share the link to our Google Form broadly with your colleagues so that others may share their perspectives as well.


Comments
  • For us, the problem starts earlier. We have difficulty getting schools/teachers to pilot our supplemental materials with fidelity and therefore the data they gather (if they manage to gather it) is not useful for sharing. Although teachers/schools are glad to hear of the positive data already gathered, the general criteria for piloters so far has been that if it seems fun and the kids like it then it’s worth talking about. In-class, in-school evidence seems to be a low priority once they are convinced to try a program. Your thoughts?

  • Rosemarie Piccioni

    This is very important work. Identifying all of the new products being launched each month, categorizing all functionalities of each, cross-referencing competitive products and services, and gathering valid feedback from actual users are all needed. I would like to be part of this work.



Post a Comment

Your email address will not be published. Required fields are marked *

Working Group Leaders
Bart Epstein
Founding CEO; Research Associate Professor
Jefferson Education Accelerator; University of Virginia, Curry School of Education
Chris Rush
Co-Founder and Chief Program Officer; Pahara-Aspen Education Fellow
New Classrooms Innovation Partners; Aspen Institute
Research Lead
David Slykhuis
Professor
James Madison University; President, Society for Information Technology and Teacher Education
Working Group Members
Betsy Corcoran
Co-founder and CEO
EdSurge
George Kane
General Manager, Education Ventures
Emerson Collective
Harold Levy
Executive Director
Jack Kent Cooke Foundation
Jennifer Medbery
Founder & CEO
Kickboard
John Bailey
Vice President of Policy and Executive Director
Digital Learning Now
Matt Greenfield
Managing Partner
Rethink Education
Michael Feldstein
Partner
MindWires Consulting
Michael Nagler
Superintendent
Mineola School District (NY)
Nick White
Chair
WCET Steering Committee, WICHE Cooperative for Educational Technologies
Nicole Neal
CEO
Noodle Markets
Rahim Rajan
Senior Program Officer, Postsecondary Success
Bill & Melinda Gates Foundation
Rebecca Griffiths
Senior Researcher
Center for Technology in Learning, SRI International
Richard Culatta
Chief Innovation Officer
Rhode Island Department of Education
Steve Fleischman
CEO
Education Northwest
Logistics Lead
Madeline Clements
Central Initiatives Associate
New Classrooms
Bart Epstein
Founding CEO; Research Associate Professor
Jefferson Education Accelerator; University of Virginia, Curry School of Education
Bart and Jefferson Education seek to improve our nation’s education system by helping the most promising growth-stage education companies grow and do real research, and by bringing together education leaders and stakeholders to tackle complex questions such as how can we make merit, not marketing, be the basis of more decision-making.
Bart Epstein
Founding CEO; Research Associate Professor
Jefferson Education Accelerator; University of Virginia, Curry School of Education
Bart and Jefferson Education seek to improve our nation’s education system by helping the most promising growth-stage education companies grow and do real research, and by bringing together education leaders and stakeholders to tackle complex questions such as how can we make merit, not marketing, be the basis of more decision-making.
Betsy Corcoran
Co-founder and CEO
EdSurge
Elizabeth “Betsy” Corcoran is co-founder and CEO of EdSurge , an award-winning news and information source on education technology. EdSurge delivers weekly newsletters, runs a vibrant jobs board, a series of powerful Summits that bring together educators and entrepreneurs who are decision makers. Through EdSurge's "Concierge" service, schools across the US are finding better edtech tools to serve their students. Ms. Corcoran is an award winning journalist. She spent 10 years with Forbes, where she served as Executive Editor for technology at Forbes Media. She’s also been a staff writer for the Washington Post, a member of the Board of Editors of Scientific American and a Fellow in the Knight Science Journalism program at MIT. She holds a bachelor’s degree in economics from Georgetown University and has been a Knight Science Journalism Fellow at the Massachusetts Institute of Technology. She’s been a featured commentator on programs including Forbes on Fox, CNBC, NBC's "Press Here.” She was named a Fellow of the Pahara Institute in 2016. Her sons attend public school in California.
Chris Rush
Co-Founder and Chief Program Officer; Pahara-Aspen Education Fellow
New Classrooms Innovation Partners; Aspen Institute
Most recently, Chris led the overall conceptualization, design and implementation of the School of One/Teach to One personalized learning programs which were named one of Time Magazine's Top 50 Inventions of the Year. He previously worked with the NYCDOE, co-leading the design of the of their citywide parent, teacher and administrator longitudinal data system.
David Slykhuis
Professor
James Madison University; President, Society for Information Technology and Teacher Education
David has been a science education professor at James Madison University for twelve years. He graduated with his PhD in science education from North Carolina State University in 2004.
George Kane
General Manager, Education Ventures
Emerson Collective
George was responsible for Apple's ecosystem of education apps and content, product management at School Specialty, business development and several product lines at Pearson, and content and websites at Blackboard.
Harold Levy
Executive Director
Jack Kent Cooke Foundation
Harold is best known for having served as chancellor of the New York City school system for three years beginning in 2000. Prior to working for the Cooke Foundation, he headed the edtech investment practice of a venture capital firm and was special counsel to a special situations/distressed debt hedge fund. He also served as the executive vice president of Kaplan, Inc.
Jennifer Medbery
Founder & CEO
Kickboard
Jennifer served as a corps member in the Teach for America program before becoming a founding teacher at New Orleans Charter Science and Mathematics Academy. In 2009 she founded Kickboard, a software and coaching provider for improving school culture.
John Bailey
Vice President of Policy and Executive Director
Digital Learning Now
John co-founded Whiteboard Advisors and previously served at the White House as Special Assistant to the President for Domestic Policy. He has also worked at the Bill and Melinda Gates Foundation, served as a top technology and innovation advisor to the Secretary of Commerce, and served as the nation’s second Director of Educational Technology.
Madeline Clements
Central Initiatives Associate
New Classrooms
Prior to joining New Classrooms, Madeline graduated from George Washington University in cultural anthropology. While in school, she volunteered for One Common Unity working with high school students. She also interned at the American Association of University Women and spent almost a year in South America promoting literacy and working with children.
Matt Greenfield
Managing Partner
Rethink Education
Matt is also on the boards of Ace Learning, BrightBytes, and Hapara and serves as an advisor to University Ventures and Laureate Education. His non-profit affiliations include the leadership council of NewSchools Venture Fund and the board of Mouse.org.
Michael Feldstein
Partner
MindWires Consulting
Michael is an educational technology consultant and a lifelong educator. Previously, he has been the senior program manager of MindTap at Cengage Learning and principal product strategy manager for Academic Enterprise Solutions at Oracle Corporation. Michael has also served as an assistant director at the SUNY Learning Network.
Michael Nagler
Superintendent
Mineola School District (NY)
Michael has been the superintendent at Mineola for seven years. He graduated from Teachers College at Columbia University.
Nick White
Chair
WCET Steering Committee, WICHE Cooperative for Educational Technologies
Nick also serves as the director of competency based learning solutions at Capella University. He has worked with Capella University since 2005, graduating with his Master's degree from the university in 2008. Prior to Capella Nick was an information developer at Avaya.
Nicole Neal
CEO
Noodle Markets
Nicole is a visionary leader and pioneer at building transformative educational products. As the CEO and Co-Founder of Noodle Markets, she leads the company in its mission to enhance efficiency and transparency within the education landscape by revolutionizing K-12 purchasing. Prior to joining the company, Nicole was the President of the Education Solutions Group at CORE Education Consulting and Solutions, Inc., a publicly traded company, where she oversaw a staff of 300 employees and managed a portfolio of formative assessment, intervention, and content solutions for districts and states serving the Pre-K, K-12, Employability, Special Education and Higher Education sectors. Nicole also served as the Senior Vice President of Major Accounts and State Services for Pearson Education Inc., one of the world’s largest education publishing and technology companies. Prior to Pearson, she served as Senior Vice President of Client Services for Schoolnet, Inc.
Rahim Rajan
Senior Program Officer, Postsecondary Success
Bill & Melinda Gates Foundation
Rahim leads investments and grant making in courseware, learning technologies, and innovations in the delivery of learning and knowledge that will help advance the success of college students. He previously helped launch and grow a number of not-for-profit technology organizations in New York City, created by the Andrew W. Mellon Foundation, and grew adoption of these resources in parts of the developing world.
Rebecca Griffiths
Senior Researcher
Center for Technology in Learning, SRI International
Rebecca specializes in blending qualitative and quantitative methods to understand the efficacy and cost-effectiveness of technology-enhanced instruction in postsecondary education. Griffiths has led large-scale research and evaluation projects involving numerous institutional partners and stakeholders.
Richard Culatta
Chief Innovation Officer
Rhode Island Department of Education
Prior to becoming the Chief Innovation Officer for the State of Rhode Island, Culatta was the Director of the Office of Educational Technology for the US Department of Education. Before that, he served as policy advisor to US Senator Patty Murray and as Chief Technology Officer at CIA University.
Steve Fleischman
CEO
Education Northwest
Steve is an expert on evidence-based policy and practice in education. A former classroom teacher, Steve has also been a vice president at the American Institutes for Research (AIR), where he created and led a number of U.S. Department of Education-funded school improvement projects.
In The News
Click on the stories below to read what people are saying about our Symposium and the importance of efficacy in EdTech.