This certainly isn’t something we were expecting. We’ve been told that artificial intelligence can do anything and solve all our problems. Yet, a new study has shown that students using ChatGPT to prepare for tests don’t do as well as those without access to the AI chatbot.
University of Pennsylvania ChatGPT Research
Many people break out in a sweat at the thought of taking a math test. Personally, I love math, but I know I’m not the norm. For those who don’t, it would probably be a relief to know that they would have the use of ChatGPT to prepare for their test.
A group of Turkish high school students were given access to ChatGPT to do practice math problems before taking a math test. Another group of students didn’t have the ChatGPT practice beforehand.
Image source:
Unsplash
Researchers at the University of Pennsylvania studied the results. Sure, ChatGPT helped that test group solve 48 percent more of the practice problems correctly, but when they took the math test, they scored 17 percent worse than the students without the AI help.
There was actually a third test group of Turkish students. While they also had access to ChatGPT, it was a revised version of the chatbot, with functionality that resembled a tutor. Instead of just providing the students with the answers, it provided hints to help them get to the correct answer on their own. They solved 127 percent more of the math problems correctly than those without the help. Yet, when it came time to take the test, the scores between these two groups were similar.
Tip: if you’re looking into using AI writing tools, learn some of the reasons you shouldn’t.
Researchers’ Conclusion on ChatGPT in the Classroom
The university researchers came to the conclusion that ChatGPT and other AI chatbots can “substantially inhibit learning,” as even when it was designed to act as a tutor, it still didn’t give students a leg up. But not only did they say it didn’t help, they titled the research paper, “Generative AI Can Harm Learning.”
It’s the belief of the researchers, after analyzing the questions the students asked of ChatGPT, that the students are using the AI tool as a “crutch.” They didn’t seem to be trying to figure it out on their own – they just turned around and asked the practice question to ChatGPT to get the answer. It didn’t guide them or teach them anything. Researchers compared it to being on autopilot.
Image source:
Unsplash
Added to that, the students put too much faith in ChatGPT. Those who used it believed that it didn’t cause then to learn less, though it clearly did. Those who used the tutor version of ChatGPT believed they had done much better than they did.
Additionally, ChatGPT wasn’t always correct, only getting the right answers about half the time. The computations were wrong about 8 percent of the time, and the way it went about solving the math problems was wrong about 42 percent of the time. Those errors didn’t show up in the tutor version, but that’s because the chatbot was programmed with the correct answers.
It’s notable that this was only in one location with one group of students, yet it did involve nearly 1000 students, freshman to junior year of high school last fall. All students were assigned the same practice problems, then took the test. They went through four cycles of this.
Older generations will see this as a “dumbing down” of our youth. I already noticed when my own kids were in school that they didn’t have to learn handwriting because of the influence of computers, and they also didn’t learn long division. Perhaps AI chatbots will add to this even more. AI tools can also be very beneficial. Check out these AI tools for students and AI tools that can help students with disabilities.
Image credit: Unsplash
Laura Tucker –
Staff Writer
Laura has spent more than 20 years writing news, reviews, and op-eds, with the majority of those years as an editor as well. She has exclusively used Apple products for the past 35 years. In addition to writing and editing at MTE, she also runs the site’s sponsored review program.
Subscribe to our newsletter!
Our latest tutorials delivered straight to your inbox
Sign up for all newsletters.
By signing up, you agree to our Privacy Policy and European users agree to the data transfer policy. We will not share your data and you can unsubscribe at any time. Subscribe