In a striking piece in Education Next, Laurence Holt dives into a series of research studies that show strong results for edtech math products Khan Academy, DreamBox Learning, i-Ready, and IXL—when the programs are used as recommended.
The results across the studies are great—0.26 standard deviations (equivalent to several months of additional learning), 0.20 SD, 0.22 SD, and 0.14 SD, respectively.
The problem? As Holt shows, in each of the studies, roughly 5% of students used each program at the minimum level prescribed. That’s a stunning—and depressing—convergence. To give an idea of what that signifies, just 4.7% of the students in the research study on Khan Academy, for example, use it a minimum of 30 minutes per week. Not a lot of time.
The other 95% of students not properly using the programs see minimal gains at best. This helps explain why, despite the rapid adoption of digital math programs in the United States, we don’t see the growth in math achievement that you might expect based on the research.
Holt offers some theories as to what’s going on here, but I have a couple myself that could lead to more of the other 95% using the programs as prescribed.
First, as we wrote as far back as Disrupting Class in 2008, it’s not the presence of technology alone that will move learning. It’s the use of technology to support a novel model of learning that will move the needle. What matters most is the model.
A central reason why technology isn’t a silver bullet in education is that when it’s crammed into the existing classroom model, at its best it can only serve as an additional resource to bolster that model’s existing processes and priorities. That means it can make an operation more efficient or allow it to take on additional tasks, but it can’t reinvent the model in and of itself. It also means that in many cases it will conflict with the organization’s processes and priorities and therefore go largely unused.
That could explain what’s going on here. The tech is just an add-on to the whole-class instruction going on. It’s not core to the model. And it’s not that different from Larry Cuban’s research back in the late 1990s showing that fifth graders reported using computers for programs like “Franklin Learns Math” or “Math Rabbit” just 24 minutes a week.
If these edtech vendors instead spent the time and resources to help the schools and classrooms set up even a basic Station Rotation model of blended learning, they could ensure that students would visit the online-learning station for a defined block of time each day in which students would do the digital math program. Then they’d all but guarantee that students would reach the minimum usage levels.
Edtech needs to think about motivation and learning differences more
Second, another way to engage more students is to make the learning more intrinsically motivating for each student. In Disrupting Class, we suggested that this could occur, in part, by personalizing based on a variety of characteristics documented in research.
Holt does a terrific job of showing the potential power of this approach in another article in Education Next, “The Orchid and the Dandelion.” The piece explores a link between a genetic variation and how students respond to teaching.
The basic idea is that some students seem to respond best when they receive more feedback and stimulation to get a good feeling from the dopamine rush. With “normal” levels of feedback and stimulation, they get bored and tune out. Others respond differently; too much feedback could cause them to get overstimulated.
Customizing for these different profiles, as some research shows, may be crucial. But how many edtech providers are building their products to take these sorts of findings into account? This is to say nothing of all the other ways one might want to provide different hooks for students based on their background knowledge and interests to get them excited by different programs or approaches to learning math.
Building for many different profiles and backgrounds is obviously hard. This is why in Disrupting Class we hypothesized that so long as the content in digital learning software is built by a single entity, there will be limits to how much customization is possible. Instead, we argued, the ultimate correct amount of customization might only occur when a platform emerges with authoring tools that allow teachers, students, parents, and more to easily create different modules for learning different concepts. Think YouTube but with far more interactivity than video.
To create a world in which more students use programs at their prescribed minimum level, this may be a necessary step. I get that we’re unlikely to undo the reality that as humans we like to avoid hard work. But anything edtech providers can do to make sure the work students are assigned is at the right level of ease and that students have the ability to navigate to content that gets them excited, the better the chance we have at making progress.
Because while there’s lots of research that shows promise, the 95% of students not using the products shows us we have a long way to go.
