Carnegie Mellon University researchers dive into edtech pilots at innovative districts


Matching schools with effective edtech involves a bit of trial and error. But when elementary, middle and high schools are spending $6.6 billion on technology tools, according to Carnegie Mellon University estimates, experts say it’s important to know if that money is delivering results.

Carnegie Mellon University is trying to find out. The university just completed a yearlong study on how specific technologies impacted student learning and engagement in specific schools, and released a report in collaboration with Digital Promise.

Carnegie Mellon researchers studied three school districts: Avonworth, South Fayette Township and Elizabeth Forward, all in Pennsylvania and all members of Digital Promise’s League of Innovative Schools, a network of districts across the country that share innovative best practices.

Some of the products hit or missed teachers satisfaction thresholds for student engagement, while other products displayed stronger outcomes for student learning and teacher support, according to the report.

Avonworth piloted two products – Puzzlets, a hybrid learning game with a programming interface that develops early computational thinking for grades K-2, and eSpark, a personalized learning platform on iPads for first-grade English Language Arts classes.

Researchers found that students were not mastering content – but still getting promoted to the next level – and were overwhelmingly failed quizzes after using the eSpark platform. One teacher commented, “The content is beyond productive struggle. I don’t see what value [students] are getting because it is just too hard for them.”

Experts concluded that a tool like eSpark works better for high-achieving students and in station rotation models, when students work independently while the teacher helps others.

Elizabeth Forward, which uses a game-based curriculum, piloted a suite of games from Amplify during the 2015-16 school year for middle school students in English, math and science. The district wanted to see whether Amplify games served as a better alternative to non-educational games that students play outside of school, and whether it encouraged competition among students and created a “culture of gaming.”

Kids at the district, which puts a device in every student’s hand, were able to play the games on their iPads and, through interviews with students, they saw strong engagement. One student claimed to play the game for fun after class hours.

Teachers praised the suite for providing an extensive list of games that satisfy different student interests, and games are user friendly enough for students to figure them out on their own.

But, there were also problems.

“Many others shared the concern that the games
do not align well with their curriculum, and
that they would use it more in the classroom
if they did,” according to the report. “The English Language Arts teachers liked that the Amplify library provides access to a variety of books; however, they found that students were spending more time playing games.”

Researchers noted that if
the school had chosen one game to be piloted in a single
grade, it would have narrowed the focus and allowed for a more thorough analysis.

At South Fayette, where computer science is the foundation of the entire curriculum, two products were piloted: INVENTORcloud curriculum
and 3D printer for 8th graders,
and Microsoft OneNote, a platform to share instructional
content between teachers and students in 7th grade science and social
studies classrooms.

The district, whose digital learning efforts are led by technology and innovation director Aileen Owens, already has a robust staff dedicated to researching different edtech products.

“South Fayette gives teachers the flexibility to personalize their curricula, rather than implement them en-masse in the classroom,” researchers wrote.

INVENTORcloud, developed by a makerspace, provides virtual, remote access to CAD instruction, design file review and rapid prototyping equipment like 3D printers and laster cutters. Teachers wanted to introduce students to the fundamentals of entrepreneurship.

The program turned out to be a success – after using INVENTORcloud, more than half of students strongly agreed that they could do the design challenges they received in class, up from 13 percent of students before using the tools.

Researchers said South Fayette “chose a product well-suited to their goals and objectives,” and there was “100 percent teacher buy-in from Day 1.”

Reach the reporter at and follow her on Twitter @clestch and @edscoop_news.