Augmenting Feedback on Students' Code with GenAI

Primary supervisor

Roberto Martinez-Maldonado
Banner

Are you ready to dive into the future of education and revolutionise how software projects are assessed? Join this innovative project aimed at creating cutting-edge learning analytics capabilities within the Faculty of IT at Monash University. This project seeks to provide automated support for teaching staff in augmenting the marking of software development and design assignments, specifically software projects submitted to the FIT-based GIT lab platform, using advanced large language models (LLMs).

By leveraging the power of GenAI, this project will transform the feedback process by analysing code revisions over time, accessing the progression of student submissions in GIT, and augmenting the qualitative analysis of submitted code for teachers and students as part of the feedback process.We will identify key teaching needs in assessment and feedback, and apply Learning Analytics and AI techniques to generate comprehensive summaries of student work. These summaries will highlight the adherence to essential design principles such as SOLID principles and design patterns, ensuring that students' code not only works but is also well-structured and maintainable.

To get a better understanding of SOLID principles and for prompt examples, check out SOLID Programming Principles.

The outcome of this project will be the development of an AI-powered teaching and feedback tool that will significantly enhance the Bachelor programs in IT, Computer Science, and Software Engineering. By streamlining the marking and feedback provision processes, we aim to improve teaching efficiency, assessment consistency, and ultimately elevate the course viability and reputation of our programs.

 

Student cohort

Double Semester

Aim/outline

Potential Aims for the Project:

  1. Automate feedback generation: Develop an AI-powered tool that uses advanced large language models (LLMs) to automatically generate detailed feedback on software development and design assignments submitted to the FIT-based GIT lab platform.

  2. Analyse code revisions over time: Implement mechanisms to analyse code revisions over time, providing insights into the progression and improvement of student submissions in GIT.

  3. Enhance qualitative analysis: Augment the qualitative analysis of submitted code to support teachers in providing more comprehensive and constructive feedback to students.

  4. Identify teaching needs: Identify and address key teaching needs in assessment and feedback by applying Learning Analytics and AI techniques to generate comprehensive summaries of student work.

  5. Promote adherence to design principles: Ensure that student code adheres to essential design principles, such as SOLID principles and design patterns, by highlighting these aspects in the feedback process.

  6. Enhance assessment consistency: Ensure consistency in assessment and feedback across different instructors and courses, therefore improving the reliability of the evaluation process.

  7. Develop code summaries: Create AI-generated summaries of student work that provide a holistic view of their performance, identifying strengths and areas for improvement.

What you will learn in this project

  • Application of large language models (LLMs) in educational contexts
  • Techniques for analysing and summarising code revisions over time
  • Integration of AI and Learning Analytics to enhance teaching and assessment
  • Practical experience with GIT and software project management
  • Development of AI-powered tools for educational purposes
  • Insights into the challenges and solutions in automated code assessment

URLs/references

Examples of prompts for analysing code: 

https://gptprompter.wiki/programming/principles/solid/

Martins, G. F., Firmino, E. C., & De Mello, V. P. (2024, May). The Use of Large Language Model in Code Review Automation: An Examination of Enforcing SOLID Principles. In International Conference on Human-Computer Interaction (pp. 86-97). Cham: Springer Nature Switzerland.

Required knowledge

  • Proficiency in Python, Java or similar programming languages
  • Familiarity with machine learning and AI concepts
  • Experience with GIT and version control systems
  • Understanding of software design principles to some extent (SOLID, design patterns)
  • Basic knowledge of GenAI and prompting