Structured Generative AI Co-Tutoring for Learning Outcomes and Motivation in Information Systems

Mei Ratnasari, Amalyanda Azhari, Diyah Trinovita, Widi Andewi, Eva Nurchurifiani

Abstract


Generative AI (GenAI) offers interactive explanations, feedback, and personalized support in higher education, yet its educational value depends on how such support is structured. Evidence remains limited regarding whether structured GenAI co-tutoring can simultaneously improve independent learning outcomes and learning motivation in concept-oriented Information Systems courses. This study examined the effectiveness of structured GenAI co-tutoring compared with conventional instruction in an undergraduate Information Systems Concepts course in Indonesia. A quasi-experimental nonequivalent control group pretest–posttest design involved 60 students from two intact classes, with 30 students in each condition. Learning outcomes were measured using parallel forms of the Information Systems Concepts Achievement Test, while learning motivation was assessed using the EVC Light scale. Data collection comprised pre-intervention assessment, the instructional intervention, and post-intervention assessment; achievement tests were completed without GenAI access to capture independently demonstrated learning. ANCOVA was used to compare posttest outcomes while controlling for corresponding pretest scores. The experimental group achieved a significantly higher adjusted ISCAT-B score than the control group, with an adjusted mean difference of 7.89 points, F(1, 57) = 27.65, p < .001, partial η² = .327. Learning motivation was also significantly higher in the experimental group, with an adjusted EVC Index difference of 0.053, F(1, 57) = 13.53, p < .001, partial η² = .192. The study extends evidence on structured GenAI support by examining independently assessed conceptual learning alongside expectancy–value–cost motivation in Information Systems education. In practice, lecturers can adopt a sequence of independent attempts, guided AI feedback, verification, and revision, followed by assessment without AI assistance.

Keywords


Conceptual learning; Generative AI; Information Systems education; Learning motivation; Learning outcomes

Full Text:

PDF

References


A. Al-Abri, “Exploring ChatGPT as a Virtual Tutor: A Multi-Dimensional Analysis of Large Language Models in Academic Support,” Educ. Inf. Technol. (Dordr)., Vol. 30, no. 12, pp. 17447–17482, Aug. 2025, DOI: 10.1007/s10639-025-13484-x.

R. Deng, M. Jiang, X. Yu, Y. Lu, and S. Liu, “Does ChatGPT Enhance Student Learning? A Systematic Review and Meta-Analysis of Experimental Studies,” Comput. Educ., Vol. 227, p. 105224, Apr. 2025, DOI: 10.1016/j.compedu.2024.105224.

W. Andewi, W. Waziana, D. Wibisono, K. A. Putra, T. Hastomo, and I. B. Oktarin, “From Prompting to Proficiency: A Mixed-Methods Analysis of Prompting with ChatGPT Versus Lecturer Interaction in an EFL Classroom,” Journal of Studies in the English Language, Vol. 20, No. 2, pp. 210–238, 2025, Accessed: Sep. 01, 2025. [Online]. Available: https://so04.tci-thaijo.org/index.php/jsel/article/view/282318

H. Bastani, O. Bastani, A. Sungu, H. Ge, Ö. Kabakcı, and R. Mariman, “Generative AI without Guardrails can Harm Learning: Evidence from High School maThematics,” Proceedings of the National Academy of Sciences, Vol. 122, No. 26, Jul. 2025, DOI: 10.1073/pnas.2422633122.

W. Huang, J. Jiang, R. B. King, and L. K. Fryer, “Chatbots and Student Motivation: A Scoping Review,” International Journal of Educational Technology in Higher Education, Vol. 22, No. 1, p. 26, Apr. 2025, DOI: 10.1186/s41239-025-00524-2.

F. J. Agbo, C. Olivia, G. Oguibe, I. T. Sanusi, and G. Sani, “Computing Education using Generative Artificial Intelligence Tools: A Systematic Literature Review,” Computers and Education Open, Vol. 9, p. 100266, Dec. 2025, DOI: 10.1016/j.caeo.2025.100266.

A. C. M. Yang, J.-Y. Lin, C.-Y. Lin, and H. Ogata, “Enhancing Python Learning with PyTutor: Efficacy of a ChatGPT-based Intelligent Tutoring System in Programming Education,” Computers and Education: Artificial Intelligence, Vol. 7, p. 100309, Dec. 2024, DOI: 10.1016/j.caeai.2024.100309.

S. Choi and H. Kim, “The Impact of a Large Language Model-based Programming Learning Environment on Students’ Motivation and Programming Ability,” Educ. Inf. Technol. (Dordr)., Vol. 30, No. 6, pp. 8109–8138, Apr. 2025, DOI: 10.1007/s10639-024-13107-x.

D. Sun, A. Boudouaia, C. Zhu, and Y. Li, “Would ChatGPT-Facilitated Programming Mode Impact College Students’ Programming Behaviors, Performances, and Perceptions? An Empirical Study,” International Journal of Educational Technology in Higher Education, Vol. 21, No. 1, p. 14, Feb. 2024, DOI: 10.1186/s41239-024-00446-5.

M. Gerlich, “From Offloading to Engagement: An Experimental Study on Structured Prompting and Critical Reasoning with Generative AI,” Data (Basel)., Vol. 10, No. 11, p. 172, Oct. 2025, DOI: 10.3390/data10110172.

M. D. H. Rahiem, “Generative AI in Higher Education in Indonesia: Patterns of use and Learning Impact,” Social Sciences & Humanities Open, Vol. 13, p. 102672, Jun. 2026, DOI: 10.1016/j.ssaho.2026.102672.

H. Crompton and D. Burke, “The Educational Affordances and Challenges of ChatGPT: State of the Field,” TechTrends, Vol. 68, No. 2, pp. 380–392, Mar. 2024, DOI: 10.1007/s11528-024-00939-0.

N. Wang, X. Wang, and Y.-S. Su, “Critical Analysis of the Technological Affordances, Challenges and Future Directions of Generative AI in Education: A Systematic Review,” Asia Pacific Journal of Education, Vol. 44, No. 1, pp. 139–155, Jan. 2024, DOI: 10.1080/02188791.2024.2305156.

Y. Qian, “Pedagogical Applications of Generative AI in Higher Education: A Systematic Review of the Field,” TechTrends, Vol. 69, No. 5, pp. 1105–1120, Sep. 2025, DOI: 10.1007/s11528-025-01100-1.

H.-Y. Lee, P.-H. Chen, W.-S. Wang, Y.-M. Huang, and T.-T. Wu, “Empowering ChatGPT with Guidance Mechanism in Blended Learning: Effect of Self-Regulated Learning, Higher-Order Thinking Skills, and Knowledge Construction,” International Journal of Educational Technology in Higher Education, Vol. 21, No. 1, p. 16, Mar. 2024, DOI: 10.1186/s41239-024-00447-4.

T. K. F. Chiu, “A Classification Tool to Foster Self-Regulated Learning with Generative Artificial Intelligence by Applying Self-Determination Theory: A Case of ChatGPT,” Educational Technology Research and Development, Vol. 72, No. 4, pp. 2401–2416, Aug. 2024, DOI: 10.1007/s11423-024-10366-w.

M. Lepp and J. Kaimre, “Does Generative AI Help in Learning Programming: Students’ Perceptions, Reported use and Relation to Performance,” Computers in Human Behavior Reports, Vol. 18, p. 100642, May 2025, DOI: 10.1016/j.chbr.2025.100642.

P. Bassner, B. Lenk-Ostendorf, R. Beinstingel, T. Wasner, and S. Krusche, “Less Stress, Better Scores, Same Learning: The Dissociation of Performance and Learning in AI-Supported Programming Education,” Computers and Education: Artificial Intelligence, Vol. 10, p. 100537, Jun. 2026, DOI: 10.1016/j.caeai.2025.100537.

D. T. K. Ng, C. W. Tan, and J. K. L. Leung, “Empowering Student Self‐Regulated Learning and Science Education through ChatGP: A Pioneering Pilot Study,” British Journal of Educational Technology, Vol. 55, No. 4, pp. 1328–1353, Jul. 2024, DOI: 10.1111/bjet.13454.

J. L. Pallant, J. Blijlevens, A. Campbell, and R. Jopp, “Mastering Knowledge: The Impact of Generative AI on Student Learning Outcomes,” Studies in Higher Education, Vol. 51, No. 4, pp. 714–735, Apr. 2026, DOI: 10.1080/03075079.2025.2487570.

Febriyanti, E. Nurchurifiani, T. Hastomo, A. Wicaksono, and S. S. Nasution, “AI Literacy of Pre-Service English Teachers: Proficiency, Strategies, and Barriers,” JEELS (Journal of English Education and Linguistics Studies), Vol. 13, No. 1, pp. 155–186, Apr. 2026, DOI: 10.30762/jeels.v13i1.6293.

Q. Xia, W. Li, Y. Yang, X. Weng, and T. K. F. Chiu, “A Systematic Review and Meta-Analysis of the Effectiveness of Generative Artificial Intelligence (GenAI) on Students’ Motivation and Engagement,” Computers and Education: Artificial Intelligence, Vol. 9, p. 100455, Dec. 2025, DOI: 10.1016/j.caeai.2025.100455.

Y. M. E. Heung and T. K. F. Chiu, “How ChatGPT Impacts Student Engagement from a Systematic Review and Meta-Analysis Study,” Computers and Education: Artificial Intelligence, Vol. 8, p. 100361, Jun. 2025, DOI: 10.1016/j.caeai.2025.100361.

L. Zhang and J. Xu, “The Paradox of Self-Efficacy and Technological Dependence: Unraveling Generative AI’s Impact on University Students’ Task Completion,” Internet High. Educ., Vol. 65, p. 100978, Apr. 2025, DOI: 10.1016/j.iheduc.2024.100978.

P. Schoeffel, V. F. C. Ramos, C. Cechinel, and R. S. Wazlawick, “The Expectancy-Value-Cost Light Scale to Measure Motivation of Students in Computing Courses,” Informatics in Education, Jun. 2021, DOI: 10.15388/infedu.2022.04.

W. R. Shadish, T. D. Cook, and D. T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin, 2002.

J. W. Creswell and J. D. Creswell, Research design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage publications, 2018.

A. J. Vickers and D. G. Altman, “Analysing Controlled Trials with Baseline and Follow up Measurements,” BMJ, Vol. 323, No. 7321, pp. 1123–1124, Nov. 2001, DOI: 10.1136/bmj.323.7321.1123.

A. P. A. & N. C. on M. in E. American Educational Research Association, Standards for Educational and Psychological Testing. American Educational Research Association, 2014.

S. Sireci and M. Faulkner-Bond, “Validity Evidence based on Test Content,” Psicothema, Vol. 1, No. 26, pp. 100–107, Feb. 2014, DOI: 10.7334/psicothema2013.256.

T. K. Koo and M. Y. Li, “A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research,” J. Chiropr. Med., Vol. 15, No. 2, pp. 155–163, Jun. 2016, DOI: 10.1016/j.jcm.2016.02.012.

D. Lakens, “Calculating and Reporting Effect Sizes to Facilitate Cumulative Science: A Practical Primer for t-tests and ANOVAs,” Front. Psychol., Vol. 4, 2013, DOI: 10.3389/fpsyg.2013.00863




DOI: https://doi.org/10.32520/stmsi.v15i9.6875

Article Metrics

Abstract view : 0 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.