Hacking HE: Lessons from a GenAI assessment hackathon

The rapid evolution of Generative AI has prompted many conversations in HE in the last few years. It has demanded that academic staff update both their practice and their assessments. We know that traditional assessments such as written essays can now be created in seconds, presenting critical issues for validity and academic integrity. Many academics now face an uncomfortable question: if GenAI can complete an assessment successfully, what exactly is that assessment measuring?

Recently, we facilitated a Generative AI Assessment Hackathon for academic colleagues at LJMU, creating a space for experimentation, peer learning, and confidence building. The premise was simple: bring a real assessment, test it against common AI tools, and work collaboratively in small, cross-disciplinary groups to redesign it where necessary. At the heart of the event was the premise that we need to move beyond detection to design.

Experimenting with AI tools is key for colleagues who are still learning about the tools themselves. The Hackathon began with participants stress-testing their own assessments using GenAI tools, exploring what a student might realistically produce, and then presenting this to colleagues. This begins with a question: could GenAI complete this task, and how well could it do it?

The next step is to hack the assessment. Working in groups and supported by facilitators, participants were presented with a series of questions designed to encourage reflection on their assessment design and provide challenge. Academics from different disciplines offered their perspectives and ideas on how the learning objectives of the course could be met and tested, whilst reducing the risk of misuse or overuse of AI. New ideas could be tested and retested.

Although the Hackathon was facilitated by colleagues experienced in AI technologies, many of the most valuable insights came not from the facilitators but from supportive conversations between colleagues and the sharing of practice. The Hackathon also created space for reflecting on some of those deeper questions facing academia today. If AI can complete assessments to a reasonable standard, what exactly are we able to measure? What evidence would demonstrate genuine understanding today? Where is the learning value located: in the final product, the process, the decision-making, or the application of knowledge? Are the outputs students have traditionally submitted still indicators of learning, if indeed they ever were?

As AI tools move on at pace, it is likely that there is no such thing as a single solution to ensuring integrity and validity. Some of the options available that may, for now at least, reduce the risk of AI misuse bring with them their own pedagogical challenges. Assessment Hackathons can offer a practical way to support assessment redesign and share good practice on an ongoing basis as the AI evolution continues.

What emerged at our event was curiosity, creativity, and a renewed focus on what assessment is actually for, and what it is really testing. The future of assessment will not be protected by trying to outrun technology or seeking to detect misuse; the target will only continue to move, as each new model development will present another challenge. Working together to share ideas and experiences will help teaching and learning staff to continually evolve alongside the tech.

Please get in touch if you would like to know more about running your own assessment hackathon.