Including GenAI in examination

If you decide to allow students to use Generative AI in any part of their examination, then there are different ways it could be incorporated. This is a starter list and you will probably think of many more: 

  • Generate ideas for their work
  • Get tips on improving the presentation of their work
  • Actually improve the presentation of their work by rewriting it
  • Produce text, images or video to submit for assessment
  • Creating banks of test questions for self-study, based on your lecture slides, handouts, or online material
  • As a conversational partner to develop ideas or practice professional conversations

These are all possibilities now, and so you need to think about what is acceptable to you in relation to different tasks. In the previous pages, we have posed many questions for you to think about in relation to GenAI tools and examination: what does a valid, secure, and fair examination look like in my subject? What skills must students be able to demonstrate without GenAI tools? What value might there be to using GenAI tools in examination? Teachers on every programme, and probably every course, will answer these questions differently, and so come to different decisions about what is permitted and what is not.

You may find the AI Assessment Scale (Perkins et al 2024) useful in articulating what is acceptable to you. It ranges from a complete ban to pushing the boundaries of AI use in relation to the assessment/examination task.

 

AIAS Perkins et al.png

 

Note that they are talking generally about AI, and not just the Generative AI tools we have been focusing on. Many commonly used software tools already have some form of AI in them (grammar checkers, for instance) and so we need to give students examples of what we mean when we give them rules or instructions about ‘AI’.  

Whilst you might want to adapt this scale for your own purposes, I think it is useful in helping to codify different approaches, and the additional text in bold provides a model for explaining what is acceptable in any particular examination. You can usefully apply it by thinking about a particular examination task which would be difficult to move to a controlled situation ilke a written or oral examination. For example, let’s consider a takehome examination which asks the students to research a particular political decision/biological process/musical genre/building technique/business case and compare it with another situation or suggest what would happen if the example was applied to a different situation. This type of examination is really useful for developing critical thinking and research skills, and needs the student to spend time with it.

No AI

The examination will be carried out under controlled conditions, that is under actual or digital supervision. This also means that the timescale needs to be constrained, because we can’t supervise over more than a few hours at a time.

AI planning

The students could use an AI tool to get started with ideas and structure of the work. They could also use it to research ideas. You could ask for a summary of what the students used it for and their opinion on its value in relation to the final submission.

AI collaboration

The students could use an AI tool as a way to manage the task, getting feedback on the structure, content and presentation of the submission. You could ask for a summary of what the students used it for and their opinion on its value in relation to the final submission.

Full AI

Anything goes. For this kind of task, this might mean pasting the instructions into a GenAI tool and then directly using the output, since GenAI tools are able to generate outputs in a range of styles and lengths. You would need to think about how you would be certain that the students have learned anything, and you might need to consider reviewing the learning outcomes. You could also ask them to explain why they did or didn’t use particular parts of the outputs and what they think they have learned.

AI Exploration

I think that the authors mean experimentation with the tools in relation to the examination task, so you could be asking students not for a traditional output, but for something which shows how AI gets used in your subject area to solve problems and create new ideas and products. Would that be something you would like to see students able to do? I think you would need to look at the learning outcomes to make sure that the task reflects what you want students to learn.

 

I’ve picked the kind of task which you can find in most, if not all, university programmes, but you can try something similar for any of your existing examination tasks: laboratory work, maths problems, computer coding, performances and presentations, multiple choice examinations, and so on. If you allow substantial use of AI tools, you should also consider whether you can expect more complex work from the students, and this might mean updating the learning outcomes.

On the next page is a four step process for implementing your decisions about how students can use GenAI in examinations.