Exploring student perspectives on AI-generated feedback using a Socratic method chatbot

Authors

DOI:

https://doi.org/10.47408/jldhe.vi37.1724

Keywords:

generative AI, student feedback, student perspectives, socratic method

Abstract

The integration of Artificial Intelligence (AI) in educational settings has opened new avenues for enhancing student learning. This study investigated the use of a generative AI chatbot, trained to provide feedback using the Socratic Method, in a Business Management programme. Recent literature highlights the transformative potential of AI in education, particularly in fostering personalised learning experiences and supporting critical thinking (Gökçearslan et al., 2024; Lee and Moore, 2024; Mustafa et al., 2024). Understanding student perspectives on AI-generated feedback is crucial for optimising its use in learning development. This study aimed to evaluate the effectiveness of AI feedback in promoting critical thinking and its acceptance among students. Previous research has shown that AI chatbots can enhance learning by providing timely and relevant feedback, though challenges such as limited interaction and potential for misleading guidance remain (Banihashem et al., 2024; Gökçearslan et al., 2024; Guo et al., 2024). A qualitative approach was employed, utilising a focus group with n=14 final-year undergraduate students on a Business Management pathway. The generative AI tool was piloted to provide feedback on student drafts for summative coursework. The quality of feedback was assessed based on its accuracy, relevance, timeliness, and effectiveness in fostering critical thinking. Data was analysed using thematic analysis, a method well-suited for identifying and interpreting patterns within qualitative data (Nowell et al., 2017; Braun and Clarke, 2022). The Socratic Method, known for its effectiveness in promoting critical thinking through questioning, was employed as the feedback mechanism (Buckingham Shum, 2024). The study revealed that students found AI-generated feedback useful and relevant for improving their work and identifying knowledge gaps, thereby promoting deep learning. The Socratic Method used by the AI encouraged deeper engagement with their work, unlike the straightforward answers typically provided by other chatbots. However, students preferred tutor feedback.

Author Biography

Nurun Nahar, University of Greater Manchester

Nurun Nahar is an Assistant Professor at Greater Manchester Business School, specialising in technology-enhanced learning and student partnerships where she is also a lead for these areas. She led institution-wide projects on AI and digital literacy, including for pre-arrival students, co-creating enhanced learning experiences with learners. Nurun volunteers on ALT and JISC committees focusing on digital assessment, learning content management, and blended learning. At the University of Greater Manchester, she is responsible for strategically steering Students as Partners approach within the Greater Manchester Way Curriculum Model. She is also founder of TIRIgogy, a CPD platform amplifying staff and student voices.

References

Banihashem, S. K., Kerman, N. T., Noroozi, O., Moon, J. and Hendrik, D. (2024) ‘Feedback sources in essay writing: peer-generated or AI-generated feedback?’, International Journal of Educational Technology in Higher Education, 21(23). Available at: https://doi.org/10.1186/s41239-024-00455-4

Braun, V. and Clarke, V. (2022) ‘Conceptual and design thinking for thematic analysis’, Qualitative Psychology, 9(1), pp.3-26. Available at: https://doi.org/10.1037/qup0000196

Buckingham Shum, S. (2024) ‘Generative AI for critical analysis: practical tools, cognitive offloading and human agency’, Joint Proceedings of LAK 2024 Workshops: 14th International Learning Analytics and Knowledge Conference. Kyoto, Japan 18-22 March. Available at: https://ceur-ws.org/Vol-3667/GenAILA-paper4.pdf (Accessed: 10 September 2025).

Grace, J. C. (2025) ‘Chalkboards to chatbots: helping faculty harness AI for the future of higher education’, Regis University Student Publications, 1178.

Fakour, H. and Imani, M. (2025) ‘Socratic wisdom in the age of AI: a comparative study of ChatGPT and human tutors in enhancing critical thinking skills’, Frontiers in Education, 10. Available at: https://doi.org/10.3389/feduc.2025.1528603

Gökçearslan, S., Tosun, C. and Erdemir, Z. G. (2024) ‘Benefits, challenges, and methods of Artificial Intelligence (AI) chatbots in education: a systematic literature review’, International Journal of Technology in Education, 7(1), pp.19-39. Available at: https://doi.org/10.46328/ijte.600

Guo, K., Pan., M., Li, Y. and Lai, C. (2024) ‘Effects of an AI-supported approach to peer feedback on university EFL students’ feedback quality and writing ability’, The Internet and Higher Education, 63. Available at: https://doi.org/10.1016/j.iheduc.2024.100962

Lee, S. S. and Moore, R. L. (2024) ‘Harnessing generative AI (GenAI) for automated feedback in higher education: a systematic review’, Online Learning, 28(3). Available at: https://doi.org/10.24059/olj.v28i3.4593

Li, Q., Hong, J., Xie, C., Tan, J., Xin, R., Hou, J., Yin, X., Wang, Z. et al. (2024) ‘LLM-PBE: assessing data privacy in large language models’, Proceedings of the VLDB Endowment (PVLDB), 17(11), pp.3201-3214. https://doi.org/10.14778/3681954.3681994

Marín, R. Y., Caro, O. C., Rubio, Y. D. C. M., Tuesta, J. N. A., Bardales, E. S., Rituay, A. M. C. and Santos, R. C. (2025) ‘Artificial intelligence as a teaching tool in university education’, Frontiers in Education, 10, pp.1-11. Available at: https://doi.org/10.3389/feduc.2025.1578451

Mustafa, M. Y., Tlili, A., Lampropoulos, G., Huang, R., Jandrić, P., Zhao, J., Salha, S., Xu, L. et al. (2024) ‘A systematic review of literature reviews on artificial intelligence in education (AIED): a roadmap to a future research agenda’, Smart Learning Environments, 11(59). Available at: https://doi.org/10.1186/s40561-024-00350-5

Nowell, L. S., Norris, J. M., White, D. E. and Moules, N. J. (2017) ‘Thematic analysis: striving to meet the trustworthiness criteria’, International Journal of Qualitative Methods, 16(1). Available at: https://doi.org/10.1177/1609406917733847

Ren, X., and Wu, M. L. (2025) ‘Examining teaching competencies and challenges while integrating artificial intelligence in higher education’, TechTrends, 69, pp.519-528. Available at: https://doi.org/10.1007/s11528-025-01055-3

Schwarz, D. (2024) ‘Report: impact of AI on academic workloads and practice’ [Preprint]. Available at: https://doi.org/10.13140/RG.2.2.28126.24641

Downloads

Published

30-09-2025

How to Cite

Nahar, N. (2025). Exploring student perspectives on AI-generated feedback using a Socratic method chatbot. Journal of Learning Development in Higher Education, (37). https://doi.org/10.47408/jldhe.vi37.1724

Issue

No. 37 (2025): Special Edition, ALDinHE Conference Proceedings and Reflections

Section

Conference Proceedings

License

Copyright (c) 2025 Journal of Learning Development in Higher Education

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).