A very honest take. They tried something, it showed a very small improvement, and they acknowledged it. Acknowledging that the cause appears to be a lack of engagement agrees with my own experience. You can give students great tools but that doesn’t make them interested in math. Khanmigo might be a good tutor but it won’t help unless you’re chaining kids to a desk and not letting them leave until they finish some quota of problems.
Conclusion: "The binding constraint appears to be engagement: realizing the promise of AI tutoring will require getting students to use it, not just giving them access."
I have found this with my kids as well. I have an AI writing tutor that they use but only one of the two actually likes engaging with it. The other gets easily disengaged.
Some kids will be able to learn at a record pace with AI, others will need to be brought along in the usual ways via good teaching that fosters engagement.
My casual interpretation is that most kids didn't want to use the AI tutor. The authors estimate that time spent using the AI tutor is as effective as time spent studying without AI assistance, which is around one fourth as effective as studying with a real tutor. The real tutors are obviously quite expensive, comparatively.
So, if some kids do prefer studying with this AI tutor, that's as good as studying without it. Whichever approach gets them to put the study time in.
Some kids will be able to learn at a record pace with AI, others will need to be brought along in the usual ways via good teaching that fosters engagement.
So, if some kids do prefer studying with this AI tutor, that's as good as studying without it. Whichever approach gets them to put the study time in.