A Semester With AI Did Not Automatically Improve Learning

1 The study and what it measured

The Journal of Computer Assisted Learning published the paper on 10 September 2026, by Chrysanthi Melanou, Maik Beege and Martin Kimmig. It investigated how generative AI influences knowledge gain, motivation, cognitive load, critical thinking and reflective use across a semester, and whether prior knowledge and AI experience moderated those effects, an explicit check for Matthew effects.

Wikipedia defines learning broadly, as acquiring understanding, knowledge, skills, values and preferences; the study measured a semester-length slice of it.

2 The design, stated plainly

It was a quasi-experimental design with three parallel classes of business informatics students, N = 87, assigned to different instructional conditions: tutor-AI use, unguided AI, and a control group without AI, with data from three measurement points, T1 through T3.

Quasi-experimental means intact classes, not random assignment, so any pre-existing difference between classes travels with the result.

3 What came back: gains, no extra

Students showed significant knowledge gains across all groups, but no additional benefits of AI integration. Gains in the control group are the crux: because every condition improved, the improvement cannot be credited to AI. Motivation remained largely stable. Germane cognitive load increased and was positively associated with knowledge gain. Critical thinking remained stable, while reflective use was higher in the AI condition and positively associated with critical thinking.

4 No extra gain is not no effect

No additional benefit does not mean AI had no effect: reflective use was higher in the AI condition and positively associated with critical thinking, and the authors state that impact depended on pedagogical framing and metacognitive regulation.

The paper concludes that findings did not support the view of generative AI as a consistent learning outcome, a statement about consistency rather than about whether anything happened.

5 The sample limits

N = 87, three parallel classes, one institution, business informatics students: not generalizable to all disciplines or institutions, and not randomized. No evidence emerged for a Matthew effect, meaning AI did not intensify performance inequalities in this sample, which cannot be extended past it.

Sample size also bounds the search itself. An effect too small to detect at this N would be reported as no additional benefit, a reason to read the null as inconclusive rather than settled.

6 The bottom line

Reported result: knowledge gains in every condition, control included, with no additional AI benefit. Reported limit: 87 students in intact classes at one institution — enough to challenge a blanket claim, not to settle it. The authors point at framing and reflection, not the tool.