The Use and Impact of AI on Students’ Achievement in Mathematics Courses
Abstract
Artificial intelligence (AI) programs are now being widely used and their impact has been proven to enhance student achievement. The aim of the study is to observe the use and impact of AI on students’ achievement in Mathematics and Statistics courses. The population in this study was all students of the Faculty of Information Technology, Universitas Advent Indonesia, taking or have taken Mathematics and Statistics courses from the lecturer (the first author of this paper), totalling 191 students. The study showed that 48% of students admitted that AI could improve their academic performance, while around 44% of students became more active and satisfied in learning mathematics and statistics. However, the students’ achievements were not permanent, as 71.3% of students admitted that their achievements were not lasting. The results indicate, based on experimental research comparing the results of quizzes (which were discussed after the tests) with mid or final exam results (derived from quizzes), that there was no improvement in test scores and even no significant impact (and correlation) between quiz results and mid or final exams. In fact, the majority (60.6%) of students were unsure, disagreed, or even strongly disagreed whether the impact of AI is more positive than negative on their learning. The results of this study suggest that AI is best used as a learning tool not merely to achieve learning outcomes, but assist deep learning to improve critical thinking.
References
A. Limbong, “Pengaruh model pembelajaran daring akibat pandemi COVID-19 terhadap prestasi belajar mahasiswa UNAI pada semester genap 2019/2020,” TeIKa, vol. 10, no. 2, pp. 161–168, Oct. 2020, doi: 10.36342/teika.v10i2.2386.
A. Limbong, “Dampak pandemi COVID-19 terhadap kinerja dosen UNAI pada tahun ajaran 2019/2020,” TeIKa, vol. 11, no. 1, pp. 29–37, Apr. 2021, doi: 10.36342/teika.v11i01.2476.
A. Limbong, “Dampak pandemi COVID-19 terhadap kinerja Universitas Advent Indonesia (UNAI),” TeIKa, vol. 11, no. 2, pp. 99–107, Oct. 2021, doi: 10.36342/teika.v11i2.2604.
Y. Copur-Gencturk, J. Li, A.S. Cohen, and C.H. Orril, “The impact of an interactive, personalized computer-based teacher professional development program on student performance: A randomized controlled trial,” Comput. Educ., vol. 210, pp.1–15, Mar. 2024, doi: 10.1016/j.compedu.2023.104963.
OpenAI, “Playground, 2022,” www.OpenAI.com. Access date: 2-Jun-25. [Online]. Avaiable: https://beta.openai.com/playground
M. Slepankova, Possibilities of artificial intelligence in education: an assessment of the role of AI Chatbots as a communication medium in higher education (dissertation), 2021, Available at: http://urn.kb.se/resolve?urn=urn:nbn:se:lnu:diva-108427 (Accessed: June 1, 2025).
H.B. Essel et al., “The impact of a virtual teaching assistant (chatbot) on students’ learning in Ghanaian higher education,” Int. J. Educ. Technol. High. Educ., vol. 19, pp. 1–19, Nov. 2022, doi: 10.1186/s41239-022-00362-6.
A. Kerly and S. Bull, “The potential for chatbots in negotiated learner modeling: A wizard-of-oz study”, in Intell. Tutoring Syst. (ITS 2026)., M. Ikeda, K.D. Ashley, T.W. Chan, Eds. 2006, pp. 443–452, doi: 10.1007/11774303_44.
Y. Wang, S. Petrina, and F. Feng, “VILLAGE—Virtual immersive language learning and gaming environment: immersion and presence,” Br. J. Educ. Technol., vol. 48, no. 2, pp. 431–450, Mar. 2017, doi: 10.1111/bjet.12388.
S. Tegos, S. Demetriadis, and A. Karakostas, “Promoting academically productive talk with conversational agent interventions in collaborative learning settings,” Comput. Educ., vol. 87, pp. 309–325, Sep. 2015, doi: 10.1016/j.compedu.2015.07.014.
Y.K. Dwivedi et al., “Opinion Paper: So what if ChatGPT wrote it?” multidisciplinary perspectives on opportunities, challenges, and implications of generative conversational AI for research, practice, and policy,” Int. J. Inf. Manag., vol. 71, pp. 11–63, Aug. 2023, doi: 10.1016/j.ijinfomgt.2023.102642.
Z. Xiao et al., “If I hear you correctly: Building and evaluating interview Chatbots with active listening skills,” in CHI '20: Proc. 2020 CHI Conf. Hum. Factors Comput. Syst., 2020, pp. 1–14, doi: 10.1145/3313831.3376131.
Y. Lee, G.J. Hwang, and P.Y. Chen, “Impacts of an AI chatbot on college students’ after-class review, academic performance, self-efficacy, learning attitude, and motivation,” Educ. Technol. Res. Dev., vol. 70, no. 5, pp. 1843–1865, Oct. 2022, doi: 10.1007/s11423-022-10142-8.
S. Cunningham-Nelson, W. Boles, L. Trouton and E. Margerison, “A review of chatbots in education: Practical steps forward,” in 30th Annu. Conf. Australas. Assoc. Eng. Educ. (AAEE 2019), 2019, pp. 1–9.
S. Sinha, S. Basak, Y. Dey, and A. Mondal, “An educational chatbot for answering queries,” in Emerg. Technol. Model. Graph., J. Mandal and D. Bhattacharya, Eds. 2020, pp. 55–60.
H. Hien et al., “Intelligent assistants in higher-education environments: The fit-ebot, a chatbot for administrative and learning support,” in SoICT '18: Proc. 9th Int. Symp. Inf. Commun. Technol., 2018, pp. 69–76, doi: 10.1145/3287921.328793.
K. Mabunda and A. Ade-Ibijola, “Pathbot: An intelligent chatbot for guiding visitors and locating venues,” in 2019 6th Int. Conf. Soft Comput. Mach. Intell. (ISCMI), 2019, pp. 160–168, doi: 10.1109/ISCMI47871.2019.9004411.
X. Pham et al., “Chatbot as an intelligent personal assistant for mobile language learning,” in ICEEL '18: Proc. 2018 2nd Int. Conf. Educ. E-Learn., 2018, pp. 16–21, doi: 10.1145/3291078.3291115.
N. Sandu and E. Gide, “Adoption of AI-chatbots to enhance student learning experience in higher education in India,” in 2019 18th Int. Conf. Inf. Technol. Based High. Educ. Train. (ITHET), 2019, pp. 1–5, doi: 10.1109/ITHET46829.2019.8937382.
F. AlDhaen, “The use of artificial intelligence in higher education – Systematic review,” in COVID-19 Challenges to University Information Technology Governance.Cham, Switzerland: Springer, 2022.
D. Baidoo-Anu and L.O. Ansah, “Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning,” J. AI, vol. 7, no. 1 pp. 52–62, Jan.-Dec. 2023, doi: 10.2139/ssrn.4337484.
G. Murtarellia, A. Gregory, and S. Romentia, “A conversation-based perspective for shaping ethical human–machine interactions: The particular challenge of chatbots,” J. Bus. Res., vol. 129, pp. 927–935, May 2021, doi: 10.1016/j.jbusres.2020.09.018.
F. Miller, J. Katz, and R. Gans, “AI x I = AI2: the OD imperative to add inclusion to the algorithms of artificial intelligence,” OD Practitioner, vol. 5, no. 1, pp. 6–12, Jan. 2018.
S. Akgun and C. Greenhow, “Artificial intelligence in education: Addressing ethical challenges in K-12 settings,” AI Ethics, vol. 2, pp. 431–440, Aug. 2022, doi: 10.1007/s43681-021-00096-7.
L.K. Fryer et al., “Stimulating and sustaining interest in a language course: An experimental comparison of chatbot and human task partners,” Comput. Hum. Behav., vol. 75, pp. 461–468, Oct. 2017, doi: 10.1016/j.chb.2017.05.045.
S. Bushwick, and M. Mukerjee, “ChatGPT explains why AIs like ChatGPT should be regulated,” SCIAM. Access date: 07-Jul-2025. [Online]. Available: https://www.scientificamerican.com/article/chatgpt-explains-why-ais-like-chatgpt-should-be-regulated1/
A. Joshi, S. Kale, S. Chandel, and D.K. Pal, “Likert scale: Explored and explained,” Curr. J. Appl. Sci. Technol., vol. 7, no. 4. pp. 396–403, Feb. 2015, doi: 10.9734/BJAST/2015/14975.
R.B. Burns, Introduction to Research Methods, London, United Kingdom: SAGE, 2000.
© Jurnal Nasional Teknik Elektro dan Teknologi Informasi, under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
1.png)
