T he ISE webinar on 25^th May hosted an insightful discussion around the pros and perils of AI in assessment and selection. Jayne Cullen from 106 listened in and watched the exclamatory chat unfold… here are some key takeaways…

Cibyl’s research into students’ use of ChatGPT

• The research had an unusually high completion rate by men; an inference that this is more on the male radar
• 1 in 10 students hadn’t heard of ChatGPT (echoing our own 106 Comms student focus group findings) with the highest usage amongst black and final-year students
• Students demonstrated nervousness about using ChatGPT for exams but were willing to engage with the tool for research, revision and other assignments

“It gives me some background and a starting point but for advanced economics, it’s wrong most of the time… it takes away from the reason why I went to Uni which is to develop my initiative and to learn how to learn..”

Michael Webb – talked about what’s happening in the world of education

The question we all had was how are universities dealing with ChatGPT? The current Higher Education position is that we have to embrace it, shifting attention away from the concern about whether students should use it, and focusing instead on how they use it. HE are starting to focus on the role that AI detection could play in assessments. However, tools for detection are not 100% accurate in themselves; too often delivering false positives.

The rapid rise of AI technology has reignited a longstanding debate and sparked a contentious question: Is there a better way to assess individuals? In the HE realm, there has been a tendency to heavily rely on essays as a primary assessment tool. The emergence of AI has the potential to act as a catalyst for change, and this could bring about positive outcomes.

Jamie Betts and Robert Newry talked about what is happening in the assessment space with employers

The new version of ChatGPT (version 4) excels in problem-solving and delivers a 99^th percentile result in verbal reasoning tests, hits the 90^TH in Numerical tests and can ace SJT’s – basically, it can pass any assessment you throw at it! It’s not just online tests we should be concerned about… there is now a plug-in on ChatGPT that listens to video interview questions… and suggests answers… (based on your LI profile that you have fed in upfront!)

OMG, WOW, Scary stuff… were just a few expressions thrown around in the webinar.

Employers have been attempting to circumvent the use of ChatGPT by implementing assessments that prevent users from copying and pasting. However, the future holds a potential workaround with discreet browser plug-ins that enable running ChatGPT alongside real-time assessments. This development opens up possibilities for utilising the power of AI while maintaining integrity in assessment processes.

Looking ahead, the expert panel delved into the crucial topic of ethical AI, emphasising the importance of transparency and auditability. As AI continues to evolve and demonstrate its capabilities, the discussion revolved around the essential skill sets that will be in demand in the future. Interestingly, many panellists highlighted the unique abilities inherent to human beings, such as creativity and empathy, as key skill sets that will remain invaluable.

ChatGPT…an obstacle to inclusion and diversity?

The current version of ChatGPT has been widely utilised and research has shown increased usage among black students and minority groups, highlighting its positive impact. However, the upcoming release of the next version behind a paywall poses a significant challenge to the social mobility agenda. This move may hinder access and limit the benefits that these underrepresented groups have been experiencing.

Another concern that persists in machine learning systems is the presence of inherent bias. These systems learn from the data they are provided, which means biases present in the input data can be embedded in the outputs they generate. Facial recognition technology serves as an example of this issue, as it has been known to unfairly target certain groups and ethnicities. This perpetuates biases and raises serious ethical concerns.

While some organisations have made slight adjustments to their assessments, acknowledging the relevance of technology skills in candidates’ daily roles, others, like law firms, have outright banned its use and opted for supervision instead. The adoption of invasive proctoring methods can lead to heightened anxiety, particularly among female, neurodiverse, and disabled students. This approach is not the preferred path for employers, as it can negatively impact the well-being and performance of individuals from these groups.

In summary…  

This is an exciting time for innovation!!! The future of assessments will no longer be text-based and we will be seeing the use of audio, visual and task-based assessments more and more. There are many challenges we have yet to overcome with AI overall and specifically with ChatGPT, how to integrate its use in assessments and applications, and how we can overcome the inherent systematic bias that will be inputted and outputted. It is clear that there is a real need for guidance and regulation.

Food for thought

Feel free to use ChatGPT to rewrite your careers website (double checked and edited by your team) BUT  do NOT use it to help you make key decisions eg screening applications.

106 Communications specialises in Student Marketing communications, building employer brands that resonate with early talent and the wider market. As a team of experts, we are on hand to help organisations navigate this complex and ever-changing assessment landscape.

Do join 106 on 22^nd June where we will be continuing this discussion around the impact of AI and discussing practical solutions. To sign up please email our Early Careers Consultant, Jayne.