AI-CHATGPT, STUDENTS’ ENGAGEMENT AND PROBLEM-SOLVING ABILITY IN UNDERGRADUATE PHYSICS: AN EXPERIMENTAL STUDY

Mokuolu Adewunmi Olanrewaju, Kolawole Richard Ojo

Abstract


The integration of Artificial Intelligence (AI) into science education can transform instructional methods by providing supporting, interactive and personalized learning experiences. This study examines the effects of AI-ChatGPT on students’ engagement and problem-solving ability in physics at the undergraduate level. A quasi-experimental design was adopted, with a study sample of 115 physics students divided into an experimental group (ChatGPT-Integrated Instruction, CGPT-II) and a control group (Traditional Lecture Method, TLM). Data were collected using Pre-Engagement Scale, Post-Engagement Scale, Pre-Problem Solving Test, and Post-Problem-Solving Test, with reliability coefficient values, α = 0.76, 0.78, 0.79, and 0.81, respectively. Collected data were analyzed using descriptive, t-test and Analysis of Covariance (ANCOVA) statistics at a 0.05 significance level. Results showed that students exposed to CGPT-II demonstrated significantly higher engagement scores (MS = 3.98, SD = 0.62) compared to the control group (MS = 2.77, SD = 1.06) with a t-test value of t(113) = 7.49, p < 0.05. Similarly, the experimental group outperformed the control group in problem-solving ability (MS = 71.61%, SD = 6.80) compared to the control group (MS = 59.40%, SD = 8.52), yielding t(113) = 8.52, p < 0.05. ANCOVA analysis indicated a significant main effect of the teaching method on post-test engagement {F(1, 113) = 27.67, p < 0.001, partial η² = 0.20} and problem-solving ability {F(1,112) = 69.48, p< 0.001, partial η² = 0.382} scores between the experimental and control groups. These findings indicate that ChatGPT significantly enhanced both engagement (behavioral, emotional, and cognitive) and problem-solving ability of students in physics. It is therefore recommended that school authorities should organize training workshops and seminars for science lecturers on the use of AI-ChatGPT, and support them with the required facilities.

 

Article visualizations:

Hit counter


Keywords


ChatGPT, physics, engagement, problem-solving ability, artificial intelligence

Full Text:

PDF

References


Ahmad, M., Zeshan, F., & Ali, S. (2022). Artificial intelligence in education: A review of current applications, challenges, and future directions. Education and Information Technologies, 27(3), 3171–3195. https://doi.org/10.1007/s10639-021-10774-3.

Alanazi, A., Osman, K., & Halim, L. (2025). Enhancing physics problem-solving skills through guided discovery and scaffolding strategies. LUMAT: International Journal on Math, Science and Technology Education, 12(4), 5. https://doi.org/10.31129/LUMAT.12.4.2329.

Almogren, A. S., Al-Rahmi, W. M., & Dahri, N. A. (2024). Acceptance of ChatGPT in higher education.Heliyon, 10. https://doi.org/10.1016/j.heliyon.2024.e31887.

Anjarani, S., Furqon, M., & Rakhmawati, F. (2024). ChatGPT to foster students’ engagement in writing class: An intervention study. Veles: Journal of English Language Education, 8(3), 630–640. http://dx.doi.org/10.29408/veles.v8i3.27687.

Boudouaia, A., Mouas, S., & Kouider, B. (2024). A study on ChatGPT-4 as an innovative approach to enhancing English as a foreign language writing learning. Journal of Educational Computing Research. https://doi.org/10.1177/07356331241247465.

Bitzenbauer, P. (2023). ChatGPT in physics education: a pilot study on easy-to-implement activities. Contemp. Educ. Technol., 15(3). https://doi.org/10.30935/cedtech/13176.

Burkholder, E. W., Miles, J. K., Layden, T. J., Wang, K. D., Fritz, A. V., & Wieman, C. E. (2020). Template for teaching and assessment of problem solving in introductory physics. Physical Review Physics Education Research, 16(1). https://doi.org/10.1103/PhysRevPhysEducRes.16.010123.

Chan, C. K. Y., & Hu, W. (2023). Students’ perceptions of generative AI. International Journal of Educational Technology in Higher Education, 20(1), 411. https://doi.org/10.1186/s41239-023-00411-8.

Darmawansah, D., Rachman, D., Febiyani, F., & Hwang, G.-J. (2025). ChatGPT-supported collaborative argumentation: Integrating collaboration script and argument mapping to enhance EFL students’ argumentation skills.Education and Information Technologies, 30(3), 3803–3827. https://doi.org/10.1007/s10639-024-12986-4.

Deep, P. D., Martirosyan, N., Ghosh, N., & Rahaman, M. S. (2025). ChatGPT in ESL higher education: Enhancing writing, engagement, and learning outcomes. Information, 16(4), 316. https://doi.org/10.3390/info16040316.

Deng, R., Jiang, M., Yu, X., Lu, Y., & Liu, S. (2024). Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies. Computers & Education, 227. https://doi.org/10.1016/j.compedu.2024.105224.

Dou, R., Brewe, E., Potvin, G., Zwolak, J. P., & Hazari, Z. (2018). Understanding the development of interest and self-efficacy in active-learning undergraduate physics courses. International Journal of Science Education, 40(13), 1587–1605. https://doi.org/10.1080/09500693.2018.1488088.

Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., & Wright, R. (2023). Opinion paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71. https://doi.org/10.1016/j.ijinfomgt.2023.102642.

Fredricks, J. A., Filsecker, M., & Lawson, M. A. (2016). Student engagement, context, and adjustment: Addressing definitional, measurement, and methodological issues. Learning and Instruction, 43, 1–4. https://doi.org/10.1016/j.learninstruc.2016.02.002.

Gan, W., Ouyang, J., Li, H., Xue, Z., Zhang, Y., Dong, Q., Huang, J., Zheng, X., & Zhang, Y. (2024). Integrating ChatGPT in orthopedic education for medical undergraduates: Randomized controlled trial. J. Med Internet. Res., 26. https://doi.org/10.2196/57037.

Gjerde, V., Paulsen, V. H., Holst, B., & Kolstø, S. D. (2022). Problem solving in basic physics: Effective self-explanations based on four elements with support from retrieval practice. Physical Review Physics Education Research, 18(1). https://doi.org/10.1103/PhysRevPhysEducRes.18.010136.

Guo, Y., Li, X., & Cunningham, G. (2025). One year in the classroom with ChatGPT: Pedagogical insights and student outcomes. Frontiers in Education, 10. https://doi.org/10.3389/feduc.2025.1574477.

Haindl, P., & Weinberger, G. (2024). Students’ experiences of using ChatGPT in an undergraduate programming course. IEEE Access, 12, 43519–43529. https://doi.org/10.1109/access.2024.3380909.

Hamid, H., Zulkifli, K., Naimat, F., Yaacob, N. L. C., & Ng, K. W. (2023). Exploratory study on student perception of the use of chat AI in process-driven problem-based learning. Currents in Pharmacy Teaching and Learning, 15(12), 1017–1025. https://doi.org/10.1016/j.cptl.2023.10.001.

Hays, L., Jurkowski, O., & Sims, S. K. (2024). ChatGPT in K-12 education. TechTrends, 68(2), 281–294. https://doi.org/10.1007/s11528-023-00924-z.

Heung, Y. M. E., & Chiu, T. K. F. (2025). How ChatGPT impacts student engagement from asystematic review and meta-analysis study. Computers & Education: Artificial Intelligence, 8. https://doi.org/10.1016/j.caeai.2025.100361.

Hsu, M. H. (2024). Mastering medical terminology with ChatGPT and Termbot. Health Educ. J., 83(4), 352–358. https://doi.org/10.1177/00178969231197371.

Jahic, I., Ebner, M., & Schon, S. (2023). Harnessing the power of artificial intelligence and ChatGPT in education – A first rapid literature review. In EdMedia Innovate Learning. Association for the Advancement of Computing in Education. https://www.learntechlib.org/p/222670/.

Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., &Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103. https://doi.org/10.1016/j.lindif.2023.102274.

Kim, J., Yu, S., Detrick, R., & Li, N. (2025). Students’ perspectives on AI writing. Education and Information Technologies, 30(1), 1265–1300. https://doi.org/10.1007/s10639-024-12878-7.

Küchemann, S., Steinert, S., Revenga, N., Schweinberger, M., Dinc, Y., Avila, K. E., & Kuhn, J. (2023). Can ChatGPT support prospective teachers in physics task development? Physical Review Physics Education Research, 19(2). https://doi.org/10.1103/PhysRevPhysEducRes.19.020128.

Lo, C. K., Hew, K. F., & Jong, M. S. (2024). The influence of ChatGPT on student engagement: A systematic review and future research agenda. Computers & Education, 219. https://doi.org/10.1016/j.compedu.2024.105100.

Lu, J., Zheng, R., Gong, Z., & Xu, H. (2024). Supporting teachers’ professional development with generative AI: The effects on higher-order thinking and self-efficacy. IEEE Transactions on Learning Technologies, 17, 1279–1289. https://doi.org/10.1109/tlt.2024.3369690.

Mai, D. T. T., Da, C. V., & Hanh, N. V. (2024). The use of ChatGPT in teaching and learning: A systematic review through swot analysis approach. Frontiers in Education, 9. https://doi.org/10.3389/feduc.2024.1328769.

Maurya, R. K. (2024). A qualitative content analysis of ChatGPT’s client simulation role-playfor practising counselling skills. Counselling and Psychotherapy Research, 24(2), 614-630. https://doi.org/10.1002/capr.12699.

Montenegro-Rueda, M., Fernández-Cerero, J., & Fernández-Batanero, J. (2023). Systematicreview of ChatGPT in education. Computers, 12(8), 153. https://doi.org/10.3390/computers12080153.

Nassiri, S. H., Khamis, N., & Bunyamin, M. A. H. (2025). Advancing physics and science education through innovative technologies and pedagogical strategies: A review. International Journal of Education, Psychology and Counselling, 10(60). https://doi.org/10.35631/IJEPC.1060052.

Nesi, Y. M. D., Kusairi, S., & Nafisah, A. W. L. (2022). Analysis of student perceptions of problem-solving learning and peer assessment. Momentum: Physics Education Journal, 6(1), 73–85. https://doi.org/10.21067/mpej.v6i1.6005.

Nugroho, A., Putro, N., &Syamsi, K. (2023). The potentials of ChatGPT for language learning: Unpacking its benefits and limitations. Register Journal, 16(2), 224–247. https://doi.org/10.18326/rgt.v16i2.224-247.

Rahman, M. M., &Watanobe, Y. (2023). ChatGPT for education and research: Opportunities, threats, and strategies. Applied Sciences, 13(9). https://doi.org/10.3390/app13095783.

Reeve, J., & Cheon, S. H. (2021). Autonomy-supportive teaching: Its malleability, benefits, and potential to improve educational practice. Educational Psychologist, 56(1), 54–77. https://doi.org/10.1080/00461520.2020.1862657.

Shahzad, M. F., Xu, S., & Javed, I. (2024). Adoption of ChatGPT in higher education. International Journal of Educational Technology in Higher Education, 21, 46. https://doi.org/10.1186/s41239-024-00478-x.

Shams, M. A., Chandio, S. A., Hussain, A. K., Kazmi, S. S. A., & Marwat, A. (2025). Effects of ChatGPT usage on student engagement and learning outcomes. Journal for Social Science Archives, 3(1), 546–556. https://doi.org/10.59075/jssa.v3i1.140.

Simangunsong, I. T., Uskenat, K., Damanik, D. P., Simangunsong, I. P., &Purba, A. A. (2024). Improvement of high school students’ physics problem-solving skills through problem-based learning assisted by student worksheets (LKPD). Lensa: Jurnal Kependidikan Fisika,12(2), 275–286. https://doi.org/10.33394/j-lkf.v12i2.13544.

Sirnoorkar, A., Laverty, J. T., & Bergeron, P. D. O. (2023). Sensemaking and scientific modeling: Intertwined processes analyzed in the context of physics problem solving. Physical Review Physics Education Research, 19(1). https://doi.org/10.1103/PhysRevPhysEducRes.19.010118.

Song Y., Huang L., Zheng L., Fan M., & Liu, Z. (2025). Interactions with generative AI chatbots: unveiling dialogic dynamics, students’ perceptions, and practical competencies in creative problem-solving. Int. J. Educational Technology Higher Education,22(12), 1-30. https://doi.org/10.1186/s41239-025-00508-2.

Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), 5. https://doi.org/10.1186/s40561-023-00237-x.

Urban, M., Děchtěrenko, F., Lukavský, J., Hrabalová, V., Svacha, F., Brom, C., & Urban, K. (2024). ChatGPT improves creative problem-solving performance in university students: An experimental study. Computers & Education, 215. https://doi.org/10.1016/j.compedu.2024.105031.

Wang, Y., Chen, X., & Sun, L. (2024). The role of artificial intelligence in promoting student engagement and problem-solving in STEM education. Journal of Educational Technology & Society, 27(1), 45–59.

Wang, J. & Fan, W. (2025). The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis. Humanities and Social Sciences Communications, 12, 621. https://doi.org/10.1057/s41599-025-04787-y.

Werth, A., Oliver, K. A., West, C. G., & Lewandowski, H. J. (2022). Assessing student engagement with teamwork in an online CURE (course-based undergraduate research experience). Physical Review Physics Education Research, 18(2). https://doi.org/10.1103/PhysRevPhysEducRes.18.020128.

Wu, T. T., Lee, H. Y., Li, P. H., Huang, C. N., & Huang, Y. M. (2023). Promoting self-regulation progress and knowledge construction in blended learning via ChatGPT-based learning aid. Journal of Educational Computing Research, 61(8), 3–31. https://doi.org/10.1177/07356331231191125.

Yang, T. C., Hsu, Y. C., & Wu, J. Y. (2025). The effectiveness of ChatGPT in assisting high school students in programming learning: Evidence from a quasi-experimental research. Interactive Learning Environments, 33(6), 3726–3743. https://doi.org/10.1080/10494820.2025.2450659.

Yilmaz, R., & Karaoglan Yilmaz, F. G. (2023). The effect of generative artificial intelligence (AI)-based tool use on students’ computational thinking skills, programming self-efficacy and motivation. Computers & Education: Artificial Intelligence, 4. https://doi.org/10.1016/j.caeai.2023.100147.

Záková, K., Urbano, C., & Cruz-Correia, R. (2025). ChatGPT’s influence on higher education. Education and Information Technologies, 30(1), 649–692. https://doi.org/10.1007/s10639-024-13184-y.

Zhou, W., & Kim, Y. (2024). Innovative music education: An empirical assessment of ChatGPT-4’s impact on student learning experiences. Education and Information Technologies, 29, 20855–20881. https://doi.org/10.1007/s10639-024-12705-z.




DOI: http://dx.doi.org/10.46827/ejes.v13i2.6542

Refbacks

  • There are currently no refbacks.


Copyright (c) 2026 Mokuolu Adewunmi Olanrewaju, Kolawole Richard Ojo

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2015-2026. European Journal of Education Studies (ISSN 2501 - 1111) is a registered trademark of Open Access Publishing Group. All rights reserved.

This journal is a serial publication uniquely identified by an International Standard Serial Number (ISSN) serial number certificate issued by Romanian National Library (Biblioteca Nationala a Romaniei). All the research works are uniquely identified by a CrossRef DOI digital object identifier supplied by indexing and repository platforms. All authors who send their manuscripts to this journal and whose articles are published on this journal retain full copyright of their articles. All the research works published on this journal are meeting the Open Access Publishing requirements and can be freely accessed, shared, modified, distributed and used in educational, commercial and non-commercial purposes under a Creative Commons Attribution 4.0 International License (CC BY 4.0).