Essential skills

One important concern in relation to assessment is that the proliferation of GenAI tools, whether or not we condone their use, may lead to students not practising skills which are considered essential to the discipline or profession. Many people worry that students will no longer develop the writing or numeracy skills we have traditionally expected in academic work, for instance. We need to think carefully about what we value in student activities and what we want students to prioritise. 

There is some evidence that the ready availability of plausible AI tools can lead to a reduction in concentration – a not-peer-reviewed study showed that high-quality AI analysis of CVs was correlated with reduced effort by the experienced human recruiters reviewing the analysis, whereas they spent more time working on lower-quality AI outputs, actually leading to better overall performance (Dell’Acqua, 2021). It is important to note that these recruiters were very experienced, so used to making complex judgements, and therefore not directly comparable with students. It may be that they already had the high level skills we expect a university education to develop, and that the outcomes might have been different for inexperienced students. Some of the same authors published a similar study, but using newer tools and with business consultants (Dell'Acqua et al, 2023, also still a preprint). This study found that consultants were generally more productive when using AI tools. Again, these were experienced knowledge workers, not students. There is a strong argument that you need to know what you don’t know in order to evaluate and use GenAI tools effectively. You need to develop that professional judgement and critical thinking, which is arguably one of the key aims of university teaching. The discussion we need to have now is whether the use of GenAI tools would hinder development of these skills. 

This is an area where universities need to get very involved in discussions about professional rigour and what skills and knowledge people need at the end of a course. We are used to thinking about the impact of technological tools on different disciplines such as the effect of the introduction of calculators on the ways mathematics is now taught. Children still learn simple mathematical operations without calculators, but more complex procedures assume their use. What is non-negotiable in your subject? What must a student be able to do without the help of digital tools? It will be useful to make lists of key tasks and discuss this with colleagues. For instance: 

Read and summarise an academic paper  (But can the students use digital tools to find the paper and type the summary? Can they use digital tools to check grammar and spelling?)

Write a coherent essay or report (same questions as the previous example)

Carry out a complex calculation (Can the students use a calculator?) 

Prepare for a client interview (Can the students use digital search tools to check legal or professional guidelines?) 

Write a specific piece of computer code  (Can the students use digital tools to check or debug the code? Can they use a code library for any part of it?)

Develop an argument for oral presentation (Can the students use digital tools to create presentation slides?)

Analyse experimental or clinical outputs (Can the students use digital tools to compare previous examples?)

Most practical skills are still not susceptible to GenAI shortcuts, but some types of clinical or practical analysis may well start to see some bespoke tools being developed. This is something to keep under review with colleagues, both here and in your broader professional networks. Create your own red lines for student skills development, so that others don't do it by default.