The Influence of Prompt Quality on Higher-Order Thinking Skills in AI-Supported Learning: The Mediating Role of Cognitive Engagement and the Moderating Role of AI Literacy
Keywords:
Prompt Quality, Higher-Order Thinking Skills (HOTS), Generative Artificial Intelligence, Prompt Engineering;, Artificial Intelligence in EducationAbstract
The rapid adoption of generative Artificial Intelligence (AI), including ChatGPT, Gemini, Claude, and Microsoft Copilot, has transformed higher education by supporting writing, problem solving, programming, and independent learning. However, the effectiveness of AI-assisted learning depends not only on AI capabilities but also on the quality of prompts created by learners. Despite its importance, empirical evidence linking Prompt Quality to Higher-Order Thinking Skills (HOTS) remains limited. This study examines the influence of Prompt Quality on HOTS among university students using generative AI, with Cognitive Engagement as a mediator and AI Literacy as a moderator. A quantitative explanatory study with a cross-sectional survey design was conducted involving 312 undergraduate and graduate students who regularly used generative AI for academic purposes. Data were collected using a five-point Likert-scale questionnaire and analyzed using Partial Least Squares Structural Equation Modeling (PLS-SEM) with SmartPLS 4. The findings indicate that Prompt Quality has a significant positive effect on HOTS (β = 0.46, p < 0.001). Prompt Quality also significantly enhances Cognitive Engagement, which partially mediates its relationship with HOTS, while AI Literacy strengthens this relationship. The model explained 64.8% of the variance in HOTS, demonstrating strong predictive capability. Prompt Quality is an essential educational competency that promotes higher-order thinking in AI-supported learning. Integrating prompt engineering and AI literacy into higher education can enhance students' critical thinking, creativity, and problem-solving skills.
Downloads
References
Baranwal, D. (2022). A systematic review of exploring the potential of teachable agents in English learning. Pedagogical Research, 7 (1).
Bucy, E. P., & Tao, C.-C. (2007). The mediated moderation model of interactivity. Media Psychology, 9(3), 647–672.
Chidongo, M. (2013). Teachers’ questioning techniques in mathematics at grade 11 leval: The case of four selected Secondary Schools in Petauke District.
Cho, M. (2019). The effects of prompts on L2 writing performance and engagement. Foreign Language Annals, 52(3), 576–594.
Davis, E. A. (2003). Prompting middle school science students for productive reflection: Generic and directed prompts. The Journal of the Learning Sciences, 12(1), 91–142.
Education, D. (2018). Transforming Teaching & Learning.
Hu, Y.-H. (2022). Effects and acceptance of precision education in an AI-supported smart learning environment. Education and Information Technologies, 27(2), 2013–2037.
Huang, K., Chen, C.-H., Wu, W.-S., & Chen, W.-Y. (2015). Interactivity of question prompts and feedback on secondary students’ science knowledge acquisition and cognitive load. Journal of Educational Technology & Society, 18(4), 159–171.
Hutson, J., Jeevanjee, T., Vander Graaf, V., Lively, J., Weber, J., Weir, G., Arnone, K., Carnes, G., Vosevich, K., & Plate, D. (2022). Artificial intelligence and the disruption of higher education: Strategies for integrations across disciplines. Creative Education, 13(12).
Kaiser, F. G., Hübner, G., & Bogner, F. X. (2005). Contrasting the theory of planned behavior with the value‐belief‐norm model in explaining conservation behavior 1. Journal of Applied Social Psychology, 35(10), 2150–2170.
Kim, H. J., Yi, P., & Hong, J. I. (2020). Students’ academic use of mobile technology and higher-order thinking skills: The role of active engagement. Education Sciences, 10(3), 47.
Kimberlin, C. L., & Winterstein, A. G. (2008). Validity and reliability of measurement instruments used in research. American Journal of Health-System Pharmacy, 65(23), 2276–2284.
Lemos, M. S., Queirós, C., Teixeira, P. M., & Menezes, I. (2011). Development and validation of a theoretically based, multidimensional questionnaire of student evaluation of university teaching. Assessment & Evaluation in Higher Education, 36(7), 843–864.
Miao, F., & Holmes, W. (2021). AI and education: A guidance for policymakers. Unesco Publishing.
Naveh, G., Tubin, D., & Pliskin, N. (2010). Student LMS use and satisfaction in academic institutions: The organizational perspective. The Internet and Higher Education, 13(3), 127–133.
Novalinda, R., Budhipradipta, C. M., & Fadillah, R. (2016). Digital and AI-Driven Transformation of Learning in Vocational Education: Implementing an Innovative Instructional Model. Journal of Education Technology, 10(1), 46–58.
Nye, B. D., Graesser, A. C., & Hu, X. (2014). AutoTutor and family: A review of 17 years of natural language tutoring. International Journal of Artificial Intelligence in Education, 24(4), 427–469.
Orús, C., Gurrea, R., & Flavián, C. (2017). Facilitating imaginations through online product presentation videos: effects on imagery fluency, product attitude and purchase intention. Electronic Commerce Research, 17(4), 661–700.
Pasham, S. D. (2022). Enabling Students to Thrive in the AI Era. International Journal of Acta Informatica, 1(1), 31–40.
Ramayah, T., Cheah, J., Chuah, F., Ting, H., & Memon, M. A. (2018). Partial least squares structural equation modeling (PLS-SEM) using smartPLS 3.0. An Updated Guide and Practical Guide to Statistical Analysis, 1(1), 1–72.
Renaud, R. D., & Murray, H. G. (2007). THE VALIDITY OF HIGHER-ORDER QUESTIONS AS A PROCESS INDICATOR OF EDUCATIONAL QUALITY: RENAUD AND MURRAY. Research in Higher Education, 48(3), 319–351.
Roscoe, R. D., & McNamara, D. S. (2013). Writing Pal: Feasibility of an intelligent writing strategy tutor in the high school classroom. Journal of Educational Psychology, 105(4), 1010.
Schwittay, A. (2021). Creative universities: Reimagining education for global challenges and alternative futures. Policy Press.
Van der Brugge, E. (2018). The use of argument mapping in improving critical thinking. Unpublished Doctoral Dissertation]. University of Melbourne. Http://Hdl. Handle. Net/11343/214519.
Vidergor, H. E. (2018). Effectiveness of the multidimensional curriculum model in developing higher-order thinking skills in elementary and secondary students. The Curriculum Journal, 29(1), 95–115.
Wana, P. R., Taufika, R., & Melisa, R. (n.d.). Social Sciences & Humanities Open.
Woolf, B. P. (2010). Building intelligent interactive tutors: Student-centered strategies for revolutionizing e-learning. Morgan Kaufmann.
Wu, S. P. W., Van Veen, B., & Rau, M. A. (2020). How drawing prompts can increase cognitive engagement in an active learning engineering course. Journal of Engineering Education, 109(4), 723–742.
Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., & Ba, J. (2022). Large language models are human-level prompt engineers. The Eleventh International Conference on Learning Representations.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Haireo Iraklis
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
